Harris healthcare Logo

Harris healthcare

Staff AI Engineer - QA

Posted Yesterday
Remote
Hiring Remotely in North Carolina, USA
120K-160K Annually
Senior level
Remote
Hiring Remotely in North Carolina, USA
120K-160K Annually
Senior level
Build and own end-to-end AI quality and evaluation infrastructure, including datasets, automated test harnesses, replay systems, severity-weighted metrics, monitoring, and adversarial testing. Establish accuracy and safety standards for healthcare AI systems, collaborate with clinical and product leaders, mentor engineers, and guide systems from prototype through production.
The summary above was generated by AI

Harris Computer is hiring a Staff AI Engineer (QA) to join their, expanding team! We’re seeking a seasoned expert to pioneer the quality and reliability of our cutting-edge AI and machine learning solutions for healthcare. As a Staff AI Engineer focused on Quality Assurance, you will be the technical authority responsible for ensuring our AI systems are robust, fair, accurate, and safe.


This is a unique opportunity to define the future of AI quality in a domain where it matters most, mentoring a talented team and establishing the gold standard for testing intelligent systems that impact patient care.


This is a full-time, remote position open to candidates in the U.S. or Canada.


What your impact will be:

  • Build the core AI system end-to-end, from the service layer and data pipelines to the evaluation infrastructure, and own its architecture.
  • Design and build the evaluation infrastructure, including dataset management, automated test harnesses, and replay of AI systems against historical workflows.
  • Build and maintain evaluation datasets with clinician-reviewed correct answers, ensuring they evolve as our systems change.
  • Create error metrics weighted by severity, so mistakes with clinical or financial impact are prioritized accordingly.
  • Build production monitoring for accuracy, overrides, and safety issues, with actionable alerts for the engineering team.
  • Write adversarial tests for AI systems, including hidden or malicious instructions in documents and invented content.
  • Work with product and clinical leads to define accuracy thresholds and the criteria for expanding or pulling back an AI capability.
  • The candidate must possess a curiosity and willingness to actively adopt and leverage emerging AI tools to improve workflows, solve problems, and drive efficiency along with being comfortable using a range of AI-enabled tools (such as copilots, chat-based AI, and automation solutions) as part of everyday work.

What we’re looking for:

  • 7 or more years in software or machine learning engineering, with at least 3 years focused on evaluation, testing, or quality for AI systems.
  • A track record of designing and building complete systems from scratch, taking them from prototype to production.
  • A track record of building and running evaluation systems that teams use in production, not just for one-off analyses.
  • At least 2 years of experience evaluating large language model (LLM) systems in a production environment.
  • Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis.
  • 1 year + of experience designing evaluation sets for document extraction, classification, or structured output.
  • Deep knowledge of explainable AI (XAI) techniques and AI-powered tools (e.g., SHAP, LIME, Claude).
  • Clear written communication, including design documents, and the ability to explain complex trade-offs to clinicians, engineers, and executives.

What would make you stand out:

  • Experience working within the healthcare or another highly regulated industry.
  • Familiarity with AI-specific testing techniques like metamorphic testing, adversarial testing, or data validation with tools like Great Expectations.
  • A history of mentoring engineers and influencing technical direction across multiple teams.

What we offer:

  • Plenty of opportunities to grow your career
  • Comprehensive medical, dental, and vision benefits
  • 3 weeks of vacation plus 5 personal days to recharge
  • Employee stock ownership, RRSP program, 401k matching
  • A chance to give back through community involvement
  • Flexible work arrangements to suit your lifestyle

Salary range: $120,000-160,000 USD depending on experience and qualifications.


About Harris Computer:

Harris provides mission critical software solutions for the Public Sector, Healthcare, Utilities and Private Sector verticals throughout North America, Europe, Asia and Australia.
Working for Harris is the perfect opportunity to fulfill your professional goals as well as achieve your personal dreams!
Our employees enjoy a casual work environment that offers comfort while providing superior service to our customers. We offer a comprehensive benefit package as well as other additional “Perks”!

  • We empower our employees to make a difference
  • We have an award-winning culture
  • We offer opportunity to learn
  • We are financially strong and we are owned by the largest software company in Canada (CSI)
  • We have fun!

#LI-remote

Similar Jobs

42 Minutes Ago
Remote or Hybrid
United States
156K-263K Annually
Senior level
156K-263K Annually
Senior level
Artificial Intelligence • Cloud • Sales • Security • Software • Cybersecurity • Data Privacy
Lead technical strategy and architecture across multiple platform domains supporting multi-tenant SaaS and distributed systems. Drive scalable shared services, API compatibility, tenant isolation, reliability, SLOs, and incident response. Establish AI-assisted engineering workflows with verification and security controls, mentor Staff-level engineers, and collaborate with senior leadership on cross-domain initiatives involving authentication, API gateways, data pipelines, and MCP Gateway services.
Top Skills: Ai Code Review ToolsAmazon BedrockApi GatewaysAWSClaude CodeCursorDistributed CachingFoundation ModelsKafkaKubernetesLlmsMcpNosql DatabasesRelational DatabasesVertex Ai
An Hour Ago
In-Office or Remote
Chicago, IL, USA
288K-330K Annually
Expert/Leader
288K-330K Annually
Expert/Leader
Blockchain • Fintech • Payments • Financial Services • Cryptocurrency • Web3
Leads Circle’s global strategy and commercial results for AI-agent partnerships and commerce. Owns Arc’s agentic ecosystem roadmap, adoption, budget, partnerships, incentives, marketplace growth, and product priorities. Builds and manages the function, develops ecosystem programs, influences executive decisions, and represents Circle with AI labs, platforms, enterprises, developers, and strategic partners. The role establishes Circle’s infrastructure, USDC, and Arc as foundational financial systems for AI-driven commerce.
Top Skills: Agent MarketplaceAgent WalletsAIAPIsArcBlockchainCircle Agent StackCircle SkillsDigital AssetsNanopaymentsProgrammable PaymentsStablecoinsUsdc
An Hour Ago
Easy Apply
Remote or Hybrid
US
Easy Apply
135K-223K Annually
Mid level
135K-223K Annually
Mid level
Marketing Tech • Social Media • Software • Analytics • Business Intelligence
Build and deploy AI agents, automations, and integrations for sales, customer success, renewals, and revenue operations. Develop LLM applications with evaluation, monitoring, and safety guardrails; integrate solutions with Salesforce, Gong, and other GTM systems; and collaborate with Salesforce IT, business analysts, and enablement teams. Own use cases from prototype through production, improve data quality and processes, support AI governance, and measure adoption and business impact.
Top Skills: Ai AgentsAi Evaluation ToolingApexAPIsFunction CallingGongLlmsPythonRetrieval-Augmented GenerationSalesforceSalesforce FlowsTool CallingWorkflow Automation Frameworks

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account