Designs and develops AWS data pipelines and automated document-processing workflows for structured and unstructured data. Builds chunking, embedding, vector indexing, semantic search, knowledge bases, and RAG integrations. Develops AI agents using Amazon Bedrock and Claude technologies, deploys them through AgentCore, and maintains GitHub Actions CI/CD pipelines. Optimizes systems for scalability, reliability, security, and performance. Requires remote work in western U.S. time zones and travel to client workshops or meetings.
Sr. Data Engineer (GenAI & AWS) Remote/Hybrid with Travel
The key is time zone coverage and so we can expand our search across the West Coast - remote/hybrid but necessarily located in the western time zones. You must be comfortable to travel to the client site for workshops/meetings.
- Senior Data Engineer (GenAI / AWS)
Overall experience: 12-20+ years Cloud Platform: AWS Primary Language: Python
Key Responsibilities
- Design and develop data pipelines using AWS Step Functions, Lambda, S3, Textract, and Comprehend.
- Build automated document-processing workflows for extracting, transforming, and processing structured and unstructured data.
- Implement document chunking, embeddings, vector indexing, and semantic search.
- Develop and maintain vector databases and knowledge bases to support Retrieval-Augmented Generation (RAG).
- Integrate knowledge bases with AI agents for contextual information retrieval.
- Build AI agents using Claude Agent SDK and deploy them on Amazon Bedrock AgentCore Runtime.
- Develop CI/CD pipelines using GitHub Actions to automate builds, testing, and deployments.
- Optimize data pipelines for scalability, reliability, security, and performance.
Must-have vs. good-to-have skills
Must-have
Good-to-have
Python, SQL
TypeScript
AWS Step Functions, Lambda, S3
AWS Glue
Amazon Textract and Comprehend
Document AI / OCR
Vector databases and embeddings
OpenSearch, Pinecone, pgvector
Document chunking and indexing
Advanced retrieval and reranking
RAG and knowledge-base integration
LangChain / LlamaIndex
Amazon Bedrock
Claude Agent SDK
GitHub Actions CI/CD
Terraform / AWS CDK
AWS IAM and API integrations
AgentCore Runtime
Important: If this individual must personally build and deploy the Claude agents, move Claude Agent SDK and AgentCore Runtime into the must-have category.
Similar Jobs
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Senior CRM sales leader responsible for selling ServiceNow CRM solutions to enterprise healthcare and life sciences customers. Builds executive relationships, develops territory and account plans, leads discovery and value-based selling, coordinates virtual sales teams, conducts customer business reviews, and identifies expansion opportunities. Requires extensive software or solutions sales experience, CRM or front-office SaaS expertise, strong enterprise sales performance, and willingness to travel up to 50%.
Top Skills:
AICRMSaaSServicenow Crm
Artificial Intelligence • Machine Learning • Software • Defense
Build full-stack geospatial applications and platform components for defense and intelligence users. Develop interactive mapping tools with TypeScript, React, Tailwind, Node.js, and PostgreSQL; design reusable map components; ingest, analyze, and visualize high-volume geospatial data; integrate agentic AI analyses; and collaborate with customers, mission owners, designers, and engineers to create intuitive operational workflows.
Top Skills:
AIAWSGeospatial DataNode.jsPostgresPythonReactTailwind CssTypescriptWeb-Based Mapping Technologies
Information Technology • Cybersecurity
Lead and develop a team of Technical Account Managers delivering post-sale guidance to cybersecurity partners. Establish standards for deployment, adoption, partner health, retention, and expansion; coach team members; coordinate with Sales, Support, Product, and Marketing; resolve technical and account issues; communicate with technical and executive audiences; and build AI-enabled workflows for account reviews, coaching, prioritization, and customer communication.
Top Skills:
Artificial IntelligenceCybersecurity
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
