Operate and monitor ML/AI models and agentic systems in production. Build AI observability, logging, tracing, and evaluation pipelines. Monitor LLM outputs, detect drift and model degradation, and maintain data/feature pipelines. Develop CI/CD, model versioning, experiment tracking, and automate alerts and incident remediation while collaborating with data scientists and platform teams.
Key Responsibilities AI/ML Model Operations
- Deploy, manage, and monitor machine learning and AI models in production environments.
- Implement model performance monitoring including accuracy, latency, and inference metrics.
- Detect and mitigate concept drift, data drift, and model degradation.
AI Observability
- Design and implement AI observability frameworks to track model behavior and reliability.
- Monitor LLM outputs, hallucination rates, and response quality.
- Implement logging, tracing, and evaluation pipelines for AI systems.
Agentic Systems Monitoring
- Monitor agent-based AI workflows and autonomous systems.
- Track agent actions, tool usage, decision paths, and execution outcomes.
- Implement guardrails, safety monitoring, and failure detection for AI agents.
Data Pipeline Monitoring
- Monitor and maintain data ingestion, transformation, and feature pipelines.
- Ensure data quality, schema consistency, and pipeline reliability.
- Detect and resolve pipeline failures and anomalies.
Infrastructure & Automation
- Build and maintain CI/CD pipelines for ML models and AI systems.
- Manage model versioning, experiment tracking, and reproducibility.
- Automate monitoring alerts, incident response, and remediation.
Collaboration
- Work closely with data scientists, ML engineers, platform teams, and product teams.
- Support continuous improvement of AI system reliability and governance
-
Compensation, Benefits and Duration
Minimum Compensation: USD 50,000
Maximum Compensation: USD 177,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Photon Chicago, Illinois, USA Office
11 East Adams Street Suite 1100, Chicago, IL, United States, 60603
Similar Jobs
Cloud • Information Technology • Security • Software • Cybersecurity
Lead the enterprise data platform strategy and architecture, driving Snowflake/dbt-based platform evolution, self-service Data Mesh and medallion models. Build AI-ready pipelines, RAG systems, and observability/cost frameworks while managing a central data team, supporting federated BI, and executing hands-on technical work.
Top Skills:
AiopsAutomated TestingCi/CdCortexData MeshDbtGitGraph RagMatillion Data Productivity Cloud (Matillion Dpc)Medallion ArchitecturePythonRetrieval-Augmented Generation (Rag)SnowflakeSnowparkSQLStreamlit
Aerospace • Hardware • Information Technology • Security • Software • Cybersecurity • Defense
Design and develop FPGA systems using VHDL and vendor toolchains. Create testbenches and perform simulation with QuestaSim, perform lab verification and debugging, manage configurations with Git, provide architectural input, and mentor/lead a team of engineers on communications-related FPGA projects.
Top Skills:
GitIntel QuartusLinuxMentor Graphics QuestasimVhdlWindowsXilinx Vivado
Aerospace • Hardware • Information Technology • Security • Software • Cybersecurity • Defense
Lead and deliver enterprise-scale IT projects (infrastructure modernization, cloud migration, cybersecurity, compliance) across the project lifecycle. Manage schedules, EVM, budgets, vendors, resources, risks, stakeholder engagement, change management, and benefits realization. Drive agile execution and executive communications.
Top Skills:
AgileCloud MigrationEarned Value Management (Evm)ExcelMicrosoft ProjectPowerPointServicenow SpmWord
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
