Top Senior Data Engineer Jobs in Chicago, IL

Build and maintain a production data platform integrating RMS, Azure SQL, HubSpot, and vendor APIs. Develop pipelines, data models, quality gates, APIs, system-of-record write-backs, performance improvements, and explainable lineage. Build production AI-agent workflows with evaluation, guardrails, and monitoring. Own features end to end, contribute to applications, support incident response, and collaborate with product managers and traders.

Design, build, deploy, and maintain AWS-based data and AI/ML infrastructure for enterprise agentic AI platforms. Develop data lake pipelines, observability systems, agent-enablement tooling, and production MLOps/LLMOps capabilities. Own features end-to-end, contribute to architecture and design decisions, optimize reliability, scalability, security, and cost, and collaborate with AWS Professional Services, client teams, and junior engineers.

Build and operate reliable ETL and ELT pipelines across lakehouse and transactional systems. Develop data models, enforce governance, integrate diverse data sources, and support analytics, APIs, and predictive tools. Monitor performance, data quality, lineage, cost, and uptime while collaborating with data science, AI, product, and innovation teams.

Designs, builds, and maintains scalable ETL/ELT pipelines, cloud data infrastructure, storage, and batch or real-time processing workloads. Optimizes queries, PySpark jobs, pipelines, performance, and cloud costs. Partners with BI, analytics, data science, and AI teams to develop dimensional data marts supporting reporting and machine learning. Implements governance, data quality validation, lineage tracking, security controls, and automated infrastructure and code deployments using CI/CD practices.

Designs, builds, and optimizes scalable end-to-end data pipelines and curated data products using Snowflake, SQL, and Python. Translates business needs into reliable technical solutions, ensuring data quality, governance, performance, and cost efficiency. Troubleshoots complex issues, contributes to architectural decisions, uses AI-assisted engineering tools, and collaborates with engineering, analytics, product, and business stakeholders. The role also includes code reviews, documentation, mentoring, and influencing data engineering best practices.

Build and maintain automated ETL/ELT data pipelines, improve data quality and monitoring, optimize Postgres and BigQuery schemas and queries, develop internal tooling, support data governance (HIPAA), participate in on-call rotation, and collaborate across engineering teams to scale data systems.

Designs, builds, and maintains enterprise data pipelines, ETL processes, integrations, cloud data infrastructure, and complex data models. Supports analytics partners, develops data warehouse solutions, tunes SQL and database performance, troubleshoots production issues, and delivers end-to-end data engineering projects. The role collaborates with architects, analysts, developers, and business stakeholders, applies big data technologies, supports agile delivery, and mentors team members.

Build and operate a governed Snowflake enterprise data warehouse, including ingestion, orchestration, dbt transformations, infrastructure as code, security controls, reliability checks, and CI/CD. Partner with data owners and BI teams during migration from legacy systems, ensure compliance for sensitive financial data, support reconciliation and incident resolution, and mentor engineers. The role requires hands-on ownership of pipelines and platform components across development and production environments.

Design and build a durable data storage platform that preserves source disagreements, provenance, uncertainty, historical versions, and evidence. Develop result and evidence stores, lineage tracking, versioned rules and configurations, reproducible decision records, and point-in-time entity views. Collaborate with entity-resolution and machine-learning teams to support probabilistic outputs. The role requires production data engineering experience, strong Python and SQL, PostgreSQL expertise, U.S. citizenship, and an active Secret clearance.

Designs and implements scalable software solutions, develops maintainable features, resolves bugs, optimizes performance, and writes automated tests. Participates in architecture discussions, code reviews, Agile planning, CI/CD, observability, and adoption of AI-assisted engineering tools. Collaborates with QA, Product Management, and DevOps to deliver software across web, desktop, cloud, and IoT environments. The role is remote for candidates in the Seattle or Los Angeles metropolitan areas, with quarterly travel to Los Angeles.

Designs and builds scalable data platforms and pipelines for scientific, clinical, manufacturing, and process-engineering data. Responsibilities include developing batch and streaming ETL/ELT systems, integrating Databricks, Snowflake, cloud platforms, MES, SCADA, and laboratory systems, and enforcing governance and GxP compliance. The role provides hands-on technical leadership, establishes data engineering standards, and collaborates with scientists, process engineers, and platform teams. It requires three days onsite weekly in Indianapolis and unrestricted U.S. work authorization.

Own and evolve end-to-end data pipeline architecture across ingestion, transformation, modeling, and serving. Lead platform improvements for cost efficiency, reliability, access control, observability, orchestration, CI/CD, and developer experience. Drive complex cross-team initiatives, resolve production issues, influence architectural decisions, mentor engineers, and build automation incorporating AI into data engineering workflows.

Build and scale resilient ingestion, catalog, and derived data pipelines powering core infrastructure datasets. Own data contracts, schema governance, quality gates, automated testing, observability, and pipeline reliability. Modernize legacy systems into scalable platform architecture, including a graph database layer. Partner with global engineering, data science, ML, intelligence, and trust teams to support matching, confidence scoring, and automated workflows.

Leads architecture and hands-on development of scalable data pipelines, governed data models, semantic layers, analytics products, and AI workflows. Writes production SQL and Python, establishes engineering standards, mentors engineers, and drives design reviews. Builds agentic and LLM solutions using retrieval and managed AI services while ensuring data quality, observability, reliability, privacy, and regulatory compliance for PHI and genomic data. Partners with stakeholders, platform teams, and governance forums to deliver trustworthy business solutions.

Designs and develops complex cloud-native data solutions, scalable batch and real-time pipelines, APIs, data models, and enterprise data platforms. Implements data governance, quality, lineage, security, observability, CI/CD, automated testing, and operational reporting capabilities. Collaborates with architects, product owners, analysts, governance, security, and platform teams on technology strategy and cloud modernization. Mentors engineers, leads technical design discussions, presents recommendations, and supports AI-ready data products and emerging development tools.

Build and maintain production data ingestion, transformation, and pipeline systems for a governed federal data and AI platform. Develop data products, contribute to Terraform infrastructure and CI/CD, monitor quality and reliability, troubleshoot issues, and apply governance and access controls. Support data science and ML teams while operating within FedRAMP, NIST 800-171, and CUI requirements. Collaborate with engineers and document pipelines and processes.

Designs, builds, and operates scalable data pipelines, warehouses, lakes, and integrations supporting analytics, products, and production AI/ML systems. Responsibilities include data modeling, schema design, validation, monitoring, orchestration, real-time processing, governance, and observability. The role contributes to architecture decisions, collaborates across analytics, AI, and product teams, communicates technical tradeoffs, and owns data engineering work end to end.

Designs and maintains scalable Databricks data pipelines on AWS, covering ingestion, transformation, streaming, data quality, and analytics-ready dataset delivery. Builds medallion architecture layers using Spark, Python, SQL, Delta Lake, and AWS services. Responsibilities include source-to-target mapping, unit testing, CI/CD deployment, anomaly analysis, and compliance with FISMA High and multi-tenant governance requirements. Collaborates in Agile teams and uses AI automation tools to accelerate pipeline development and validation.

Designs and operates data infrastructure for a clinical trial platform. Builds pipelines that extract, normalize, validate, and link information from clinical documents and publications into Aurora and GraphDB. Develops embeddings, vector search, RAG workflows, data-serving APIs, lineage tracking, and regulatory audit trails. The role also requires AWS-based orchestration, clinical data quality controls, biomedical knowledge graph modeling, and deployment automation using infrastructure-as-code and containerization.

Designs and maintains scalable data pipelines, warehouse models, and analytics solutions. Builds natural-language data interfaces using GenAI and LLM techniques, optimizes SQL, improves data quality and observability, and supports distributed data platforms on AWS or GCP. Partners with engineering, analytics, business, and product teams, contributes to shared codebases, establishes development standards, and participates in production on-call and incident response.

Manage the health, performance, reliability, and cost of customer Snowflake and Databricks environments. Responsibilities include customer onboarding, account and warehouse setup, permissions, data ingestion, query optimization, cost reporting, incident response, escalation support, and automation using Python and Terraform. The role also advises customers and internal teams on data platform best practices and participates in on-call rotations.

Build and maintain data lakes, data warehouses, and modern data pipelines for a multi-tenant SaaS platform. Design end-to-end data processes, write pipeline and infrastructure code, conduct design reviews, optimize performance, monitor systems, support data governance, and troubleshoot scalability and security issues. Collaborate with technical and business stakeholders, mentor junior engineers, evaluate technologies, document platform architecture, automate testing, and participate in on-call rotation.

Own the architecture, roadmap, and operation of Clarium’s AWS-based data platform. Build reliable batch and near-real-time pipelines, model healthcare supply chain data, optimize Postgres and Snowflake performance, and establish standards for testing, deployment, quality, and governance. Partner cross-functionally to create durable data models, lead multi-quarter initiatives, manage sensitive data controls, and mentor engineers.

Leads and mentors the Data Engineering team while remaining hands-on in designing, building, and maintaining scalable data pipelines, warehouses, integrations, and infrastructure. Establishes architecture, reliability, quality, and performance standards; guides technical delivery; and resolves complex issues. Partners with Finance and business stakeholders to translate data needs into practical solutions, sets technical direction, evaluates technologies, and communicates strategy, risks, and recommendations to leadership.

Leads and delivers complex data engineering projects, designing scalable data infrastructure, pipelines, warehouses, and data quality solutions. Recommends tools and architectures, optimizes data movement and performance, collaborates across technical teams, mentors engineers, and translates technical concepts for clients and nontechnical stakeholders. The role may also involve real-time processing, data modeling, transformation, security, and MLOps.