Plenful Logo

Plenful

Staff Machine Learning Engineer

Posted 15 Days Ago
Be an Early Applicant
Hybrid
Chicago, IL, USA
Senior level
Hybrid
Chicago, IL, USA
Senior level
Set technical direction for production AI agents, LLM and vision-language model post-training, evaluation, MLOps, and data improvement systems. Architect agent orchestration, retrieval, observability, and failure recovery; establish quality and release standards; and ship measurable model improvements. This hands-on individual contributor will co-own the ML roadmap, lead technical reviews, mentor engineers, support hiring, and implement privacy-conscious, HIPAA-aware practices.
The summary above was generated by AI
About Plenful

Plenful is on a mission to transform healthcare operations from the inside out. Fresh off our $50M Series B and backed by Notable Capital, Bessemer Venture Partners, TQ Ventures, Susa/Kivu Ventures, and other leading investors, we’re building the category-defining AI automation platform that healthcare teams rely on to operate smarter, faster, and more efficiently. Our technology empowers healthcare operators across hospital and health systems, pharmacies and payors to eliminate manual work, reduce administrative burden, and improve compliance, all while unlocking critical revenue to fund programs for their in-need patient populations.


Built by healthcare operators for healthcare operators, Plenful is driven by a deep understanding of the challenges facing today’s care teams. We’re passionate about equipping healthcare workers with world-class tools that deliver real, measurable impact, and we’re proud to serve 100+ leading health systems, pharmacies, and healthcare organizations across the country. If you’re excited to help shape the future of healthcare, we’d love to meet you. Apply now to join our growing team.

About the role

Healthcare's back office still runs on faxes, PDFs and phone calls. AI can take that work off people's plates, but only if it is accurate enough to trust with patient and financial data.

At Plenful, we build agents and models that do that work in production, with people in the loop wherever judgment matters. As a Staff Machine Learning Engineer, you set the technical direction for how we build, train and measure AI across the company, and you stay hands-on while you do it. This is an individual contributor role with no direct reports and broad influence.

What you'll do
You set the direction for how our agents are built
  • Define the architecture for production agents: tool use, retrieval, state and context management, routing, orchestration, tracing and failure recovery.

  • Turn what works into patterns other engineers reuse.

  • Make the calls on build versus buy for models and platform pieces.

You own our post-training strategy
  • Decide where supervised fine-tuning, preference optimization, distillation and reinforcement learning pay off, and where they do not.

  • Train and ship LLMs and vision-language models for healthcare tasks, and build the in-house capability to keep doing it.

  • Prove every gain with evaluation that holds up in production.

You define what good looks like
  • Set the AI quality strategy: automated graders, calibrated LLM judges, regression suites, human review, failure taxonomies and offline-to-online validation.

  • Make eval results the basis for release decisions.

You build loops that improve themselves
  • Design the systems that turn production failures and user feedback into better datasets, evals and models.

  • Set the standard for dataset versioning, observability and MLOps so every team's work is reproducible and measurable.

You raise the bar for everyone
  • Co-own the 12-month ML roadmap with engineering and product leaders, balancing near-term wins against longer bets.

  • Lead technical reviews, mentor Senior engineers, and help us hire.

  • Build privacy in by design: minimum necessary data, de-identified or synthetic data where it works, and HIPAA-aware practices throughout.

What we're looking for
  • 8+ years in ML or software engineering, including 3+ years leading LLM or agentic systems in production.

  • Deep applied post-training experience. You have run supervised fine-tuning, preference optimization, distillation or RL end to end and shipped a model that moved a production metric.

  • You have designed an evaluation strategy for LLM or agent systems and seen it drive real release decisions.

  • You have architected production agent systems and can name what broke and what you changed. Experience with frameworks like LangGraph, RAG and vector search (for example Pinecone) is expected.

  • Strong Python and PyTorch, and comfort with distributed training and serving (for example vLLM, FSDP or DeepSpeed).

  • Real MLOps depth: Docker, Kubernetes, MLflow or equivalent, CI/CD, and experiment and dataset tracking.

  • You influence without authority. Your design docs, reviews and mentoring make other engineers better.

You stay current, and others rely on it
  • You use new models and coding agents the week they ship, and colleagues ask you whether a new thing is real.

  • You turn papers and releases into experiments within days and report back with numbers.

  • You have proof: open source, write-ups, talks, or evals other people use.

Nice to have
  • Healthcare data experience: EHR, FHIR, claims, prior authorization.

  • Publications or open-source work in LLM evaluation, post-training or agents.

  • Experience interviewing and hiring ML engineers.

  • Work in another regulated, high-stakes domain.

What your first 90 days look like

By day 30, you have reviewed our agents, evals and training pipeline and written down the three biggest gaps. By day 90, you have shipped a post-training or evaluation improvement that moved a production metric, and published the plan the team follows for the next two quarters.

Why You'll Love Working Here
  • 🚀 Mission-Driven, World-Class Team — Join an exceptional group of professionals aligned around a meaningful mission and committed to making an impact

  • 📈 Opportunities for Growth — Strengthen your expertise through collaboration with experienced, high-performing leaders across the organization

  • 🏢 Flexible Hybrid Work Environment — We're remote-first, with meaningful office presence in San Francisco and New York. R&D roles follow a hybrid model, with two days per week in our San Francisco office

Benefits & Perks
  • 🏥 Healthcare Coverage — Full medical, dental, and vision insurance for you and participation for your family

  • 💰 401(k) with Company Match — Plenful matches 50% of your first 3% contributed

  • 📊 Equity — Every full-time employee shares in our success

  • 🌴 Unlimited PTO — Take the time you need, when you need it

  • 🍽️ Daily Lunch Stipend — $100/week to cover your midday meals

  • 💪 Wellness Stipend — $100/month to support your health and well-being

  • 🚇 Commuter Benefits — $100/month for SF and NYC-based employees

  • 👶 Parental Leave — Paid leave to support growing families

Similar Jobs

15 Days Ago
Remote or Hybrid
Oak Brook, IL, USA
149K-254K Annually
Expert/Leader
149K-254K Annually
Expert/Leader
Automotive • Hardware • Internet of Things • Mobile • Software • App development • PropTech
Lead development and production deployment of machine learning systems for time-series anomaly detection in connected-device environments. Build streaming and MLOps pipelines, tune alerts, monitor model drift, and integrate AI systems with LLM-based assistants. Guide architecture, tooling, observability, and production readiness while mentoring and managing applied AI and full-stack engineers. Collaborate across engineering, product, data, and research teams.
Top Skills: ArimaCi/CdDartsFlinkGluontsGruIsolation ForestKafkaKatsLlmsLstmMerlionNumpyOne-Class SvmPandasProphetPythonPyTorchScikit-LearnSparkTcnTensorFlowTransformersVector Databases
28 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
234K-349K Annually
Expert/Leader
234K-349K Annually
Expert/Leader
eCommerce • Healthtech • Kids + Family • Retail • Social Media
Own the strategy, architecture, development, deployment, monitoring, and continuous improvement of production personalization and recommendation systems. Build custom embeddings, ranking models, and shared representations across feeds, search, and recommendations. Lead ambiguous problems from concept to measurable customer impact, establish AI evaluation standards, partner with product and data teams, make cross-team technical decisions, and coach senior engineers.
Top Skills: AirflowAWSAws SagemakerBedrock AgentcoreClaude CodeCoderabbitDatadogDbtDeep LearningDevinGithub ActionsHexIncident.IoIterableKotlinKubernetesLangchainLaunchdarklyLinearMatrix FactorizationMaximMlflowMySQLPackwerkPandasPythonPyTorchRuby on RailsRankingReactRetrievalScikit-LearnShopifySidekiqSigmaSnowflakeSwiftTerraformTypescriptWardenWarpstreamWeaviateXgboost
8 Days Ago
In-Office or Remote
United States
220K-329K Annually
Senior level
220K-329K Annually
Senior level
Artificial Intelligence • Consumer Web • Digital Media • Machine Learning • Software
Lead the technical direction of Quora’s recommendation ecosystem, including feed retrieval, ranking, re-ranking, optimization, data pipelines, feature engineering, model training, and production integration. Drive measurable improvements through experimentation, apply generative AI, mentor engineers, and partner across product, data science, and infrastructure teams.
Top Skills: C++Data PipelinesEmbeddingsGenerative AiLarge Language Models (Llms)Machine LearningOnline ExperimentationPythonRecommendation Systems

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account