Designs and integrates AI-enabled modernization solutions while establishing quality assurance frameworks for AI-generated code, skills, agents, and prototypes. Develops test plans, acceptance criteria, regression suites, automated testing, security scans, and quality gates. Collaborates with DevOps teams on reliable multi-agent workflows, validates prototypes for production readiness, and monitors accuracy, security, compliance, and data-sensitivity risks.
AI Quality Engineer
Full-Time | 100% Remote (U.S.)
Overview
The AI Quality Engineer supports a federal customer by helping design and implement AI-enabled solutions, developing and integrating AI capabilities, and ensuring that skills, agents, and AI-generated prototypes meet quality, security, and compliance standards as they move toward production. This work supports broader modernization goals, including automating and streamlining servicing operations, reducing manual processing, improving data quality, aligning with the customer’s modernization standards, and establishing a roadmap toward the eventual retirement of legacy platforms. Success requires a bachelor’s degree (or equivalent experience), demonstrated experience in software quality assurance and test engineering, hands-on experience validating AI-generated code and prototypes, familiarity with CI/CD pipelines and quality gates, strong analytical skills, and excellent written and verbal communication.
Key Functions
- Design and implement AI-enabled solutions for customer modernization use cases, developing and integrating AI capabilities into existing systems and workflows.
- Establish and enforce a quality assurance framework — test plans, acceptance criteria, and regression suites — for validating AI-generated code, skills, and agents before they are promoted through the innovation pipeline.
- Guide the development of reusable AI skills and agents for customer modernization tasks, with particular emphasis on automated testing, security scanning, data sensitivity, and scalability.
- Collaborate with DevOps engineers to create and orchestrate long-running, multi-agent workflows, ensuring reliability, observability, and recoverability in production environments.
- Support creation of the innovation pipeline that moves AI-generated (“vibe-coded”) prototypes from sandbox concepts into secure, enterprise-grade applications.
- Support Value Engineers’ efforts by reviewing and quality-checking prototype outputs for accuracy, reliability, and adherence to standards prior to pipeline promotion.
- Continuously monitor AI-assisted development outputs for accuracy, reliability, and compliance drift, feeding findings back to Value Engineers and AI Engineers to improve prompts, skills, and agents over time.
Minimum Qualifications
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent professional experience.
- Demonstrated experience in software quality assurance, test engineering, or a related discipline, including test plan design and defect management.
- Hands-on experience testing or validating AI-generated code, prototypes, or “vibe-coded” solutions for accuracy and reliability.
- Experience with automated testing tools and frameworks (e.g., Selenium) across web and enterprise applications.
- Experience with one or more of the following: Salesforce, AWS (including Lambda and Redshift), Java, or PostgreSQL, or a comparable enterprise technology stack.
- Familiarity with CI/CD pipelines and quality gates within modern software delivery practices.
- Strong analytical skills, with the ability to identify security, data-sensitivity, and compliance risks in AI-assisted solutions.
- Excellent written and verbal communication skills, with the ability to work across product, engineering, data, and business SME teams.
- Must be a U.S. citizen and eligible to obtain and maintain a Public Trust clearance / favorable suitability determination.
Preferred Qualifications or Skills
- Direct experience supporting Federal clients.
- Experience with FedRAMP authorization processes.
- Mortgage, fintech, or other regulated industry experience (preferred, not required — willingness to learn quickly is essential).
- Familiarity with AI safety, governance, and compliance.
- Ability to use or teach tools such as ChatGPT, Claude, Gemini, Notion, Airtable, Zapier, LangChain, or similar.
- Experience in startups, consulting, internal strategy, learning & development (L&D), or innovation teams.
Position Details
- Employment type: Full-Time
- Location: 100% Remote (US-based)
- Hours: 8am–5pm ET
- Eligibility: Must be a U.S. citizen and suitable for, and eligible to obtain and maintain, a Public Trust clearance
Similar Jobs
Fitness
Own quality control, regulatory alignment, and performance reporting for AI-powered clinical extractors. Analyze accuracy, consistency, data quality, provenance, and performance drift; guide extraction logic and clinical interpretation; support FHIR and CSV data exports; and contribute to governance, de-identification, validation, and client reporting. Collaborate with Data Science, Engineering, and Clinical Informatics while serving as a clinical subject-matter expert, particularly in oncology and real-world evidence.
Top Skills:
AWSCsvFhirGCPGitGitHealth Information Exchanges (Hie)LoincOmopPythonQualified Health Information Networks (Qhin)RRxnormSnomedSQL
Fintech • Financial Services
Designs and maintains automated UI, API, end-to-end, and integration tests across React, Angular, and .NET applications. Validates data flows with SQL Server or Azure SQL, integrates test suites into Azure DevOps quality gates, and manages test data for parallel execution. Uses AI tools to accelerate test generation while critically reviewing coverage and assertion quality. Documents defects in Jira, contributes to test planning and acceptance criteria, and collaborates with engineers and offshore QE teams.
Top Skills:
AngularAsp.Net CoreAzure DevopsAzure SqlC#ChatgptClaudeGithub CopilotJIRANunitPlaywrightReactRest ApisSQL ServerTypescriptXunit
Professional Services • Consulting
Own the company’s AI-powered quality and release infrastructure as its first dedicated hire. Build agent-run test harnesses, LLM graders, regression coverage, release gates, fresh-install and upgrade verification, and natural-language UAT. Manage CI health, artifact publication, deployment pipelines, credentials, certificates, and staging environments. Write production Python code, convert escaped bugs into regression tests, and provide trusted release-readiness evidence through dashboards, runbooks, retrospectives, and operating policies.
Top Skills:
AWSDockerGithub ActionsHelmKubernetesLlmsPlaywrightPythonSnowflake Native AppsTypescript
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
