Top Senior Data Engineer Jobs in Chicago, IL

Designs and operates batch, streaming, and real-time data platforms and pipelines using Spark, Kafka, Iceberg, Airflow, and cloud infrastructure. Owns data lake evolution, data quality, observability, reliability, governance, event instrumentation, schema management, and privacy controls for PII, retention, deletion, and residency. Leads complex initiatives end-to-end, troubleshoots production issues, mentors engineers, and develops AI agent harnesses that enforce data engineering standards and quality gates.

Build and lead the data engineering ecosystem supporting analytics, BI, and data science. Design scalable pipelines, cloud infrastructure, Snowflake data warehouses, governance controls, validation frameworks, and machine learning workflows. Collaborate with engineering, data science, product, and business stakeholders on architecture and technology decisions. Develop AI agents and secure enterprise AI integrations, participate in on-call and incident management, document standards, and mentor data engineers.

Builds and maintains scalable data pipelines, ETL infrastructure, data warehouses, and analytics tools. The role assembles complex datasets, automates processes, optimizes data delivery architecture, and supports cross-functional data initiatives. Collaborates with executive, product, data, and design stakeholders to resolve technical issues and improve analytics systems. Requires proficiency in SAS, Python, Java, MATLAB, SQL, NoSQL, Hadoop, MapReduce, Hive, Presto, UNIX, and Linux, with JavaScript, R, and basic machine learning preferred.

Designs and maintains scalable cloud data architectures, pipelines, ETL processes, data warehouses, databases, and data models. Ensures data quality, accuracy, security, availability, and consistency across sources. Develops SQL queries for analytics and reporting, and documents pipelines, schemas, architectures, and models. The role requires expertise with Python, Java, Scala, R, SQL, relational and NoSQL databases, cloud platforms, and Snowflake.

Designs and delivers large-scale, governed data pipelines and infrastructure across the full data lifecycle. Responsibilities include coding in Python and SQL, optimizing Airflow, dbt, AWS, Kafka, and Iceberg/Parquet workflows, implementing data quality and observability, contributing to architecture and tool evaluations, mentoring engineers, collaborating with stakeholders, and participating in on-call support.

Designs and manages scalable data pipelines, contracts, transformation layers, and analytics models. The role focuses on data governance, observability, lineage, configuration validation, and collaboration with product engineering teams. It requires processing large datasets, integrating cloud and on-premises industrial technologies, and implementing streaming and message-queuing solutions, ideally in cybersecurity or industrial environments.

Belvedere Trading seeks a Senior Data Engineer to enhance trading performance through data availability and reliability, designing ETL pipelines and data architectures in a fast-paced environment.

Designs and operates an AWS-native lakehouse supporting analytics, reporting, visualization, and AI/ML. Builds batch and streaming ingestion pipelines using Python and PySpark, Apache Iceberg data structures, governance and metadata services, quality controls, security, observability, and CI/CD automation. Optimizes performance, scalability, reliability, and cost while collaborating with cross-functional technical teams and documenting architecture, operations, and secure configurations.

Designs, builds, and optimizes modern Azure and Databricks data platforms for clients. Responsibilities include client discovery, requirements workshops, architecture discussions, batch and streaming ingestion, lakehouse and dimensional modeling, data transformation, governance, monitoring, DevOps, security, and solution reviews. The role also communicates recommendations to technical stakeholders and leadership while delivering scalable analytics and reporting solutions.

Lead and mentor a data engineering team to design, develop, test, and support cloud-based big data and streaming solutions. Build full-stack data pipelines using Java/Scala/Python, RDBMS/NoSQL, Redshift/Snowflake, and cloud platforms. Collaborate with product managers, perform unit tests and code reviews, and promote engineering best practices and tech community engagement.

Build, deploy, and operate production-grade data pipelines and data products for healthcare audiences. Design transformations, data models, and governed views using Python, SQL, Airflow, S3, Snowflake, and EMR. Implement data-quality, monitoring, and privacy-by-design controls for PHI/PII. Partner with product, analytics, and platform teams to onboard sources, support audience discovery, segmentation, activation, measurement, and troubleshoot production issues.

Build, optimize, and maintain data pipelines, ETL processes, data architectures, AI solutions, and analytics platforms. Develop cloud-based solutions using Azure Functions and containerization, implement CI/CD workflows, integrate databases and APIs, and ensure data quality, monitoring, reliability, and documentation. Troubleshoot production issues, collaborate with cross-functional teams, and apply modern data engineering, AI, cloud, and DevOps practices.

Build modern data and AI analytics solutions using ThoughtSpot, Snowflake, semantic data layers, and governed data assets. Develop ETL processes, AI prompts, intelligent dashboards, and interactive analytics experiences for actuarial teams. Partner with business and analytics stakeholders to support pricing, reserving, underwriting, and portfolio decisions while surfacing trends, explaining business drivers, and enabling actionable recommendations.

As the founding engineer of the Data Intelligence team, you will design semantic layers, develop AI tooling, and automate data processes to drive business insights.

Lead design and scaling of the data platform and analytics pipelines. Build and maintain ETL/ELT workflows, integrations, serverless data services, and reliable data models. Mentor teammates, improve automation, observability, and data governance to support analytics and operational insights.

Lead a team of engineers to design and implement data solutions using cloud technologies, optimizing data pipelines and collaborating with Agile teams to deliver impactful data-driven products.

Lead and mentor data engineering teams to design, build, test, and support cloud-based, full-stack data solutions. Work with big data, streaming, and data warehousing technologies (Redshift/Snowflake), collaborate with product managers, and ensure high-quality, performant code and systems.

Design, build, and maintain scalable batch and real-time data pipelines and data lake architecture. Improve observability and SLOs, optimize ETL/ELT, develop dbt workflows, support event-driven architectures, integrate financial APIs, and collaborate with analytics and ML teams to deliver reliable, model-ready data products.

Designs, builds, and maintains scalable data pipelines, data marts, enterprise data platforms, and ETL processes. Integrates diverse data sources, improves data quality and performance, and supports secure analytics infrastructure using Microsoft Fabric and Azure. Develops Tableau and Power BI dashboards, collaborates with cybersecurity and cross-functional teams, manages project deliverables, documents processes, and trains team members on data organization and interpretation.

Own and evolve the data infrastructure powering identity and fraud products. Build and operate scalable ETL/ELT pipelines, ensure data quality and observability, optimize storage and access, support product squads, respond to production incidents, mentor engineers, and drive data platform innovation.

Lead the Data Platform Engineering team owning ETL/reverse-ETL pipelines, replication, and production datastores. Define architecture, roadmap, standards (IaC, observability, schema contracts), mentor engineers, drive incident response and durable fixes, and partner on cross-team technical decisions and operational reliability for data correctness, completeness, and timeliness.

Designs and maintains healthcare data pipelines, ETL workflows, databases, data models, and analytical frameworks. Analyzes claims and payer data to identify trends and insights, while ensuring data quality and integrity. Creates reports, dashboards, and visualizations for nontechnical stakeholders and collaborates with healthcare policy experts, analysts, and other stakeholders to translate requirements into data solutions. The role also improves data infrastructure and analytics capabilities.

The Senior Data Engineer will design and implement scalable data models in BigQuery using dbt for analytics and reporting, ensuring data governance and optimal performance.

Lead design and operation of large-scale data pipelines and data APIs. Build Spark/PySpark workflows on Databricks, optimize job performance, manage data quality and observability, and develop MCP servers and AI-agent integrations. Mentor engineers, define standards, support production incidents and on-call rotations, and collaborate with stakeholders to deliver scalable data platform products.

Lead design and delivery of scalable, secure cloud data pipelines and products. Translate requirements into architecture, apply advanced data modeling (Data Vault 2.0/dimensional), embed data quality, observability, security, and DataOps/CI-CD practices, optimize performance and cost, mentor engineers, and enable AI/ML-ready datasets within an agile DevSecOps model.