Top Senior Data Engineer Jobs in Chicago, IL

Build and maintain Databricks-based data platforms, including ingestion, transformation, storage, governance, data modeling, and serving pipelines. Establish medallion architecture standards, canonical data models, quality controls, lineage, schema evolution, and reliable batch or incremental processing. Improve pipeline observability, scalability, idempotency, and recoverability while moving curated data to systems such as ClickHouse. Collaborate across application and analytics teams to create durable, governed production datasets.

Designs and operates scalable streaming and batch data pipelines using Azure Data Explorer, Kafka, Cribl, and Azure Data Factory. Develops data models, warehouses, governance, security controls, curated datasets, and analytics dashboards in Power BI and Tableau. Partners with technical and business stakeholders to deliver data solutions, optimize analytical workloads, ensure data quality, and support decision-making. Mentors data engineers and contributes to architecture, data security, and emerging technology adoption.

Develop and optimize enterprise Oracle SQL and PL/SQL solutions, including data models, database designs, APIs, ETL processes, batch jobs, and performance tuning. Collaborate with cross-functional teams to translate requirements into scalable implementations, support testing and CI/CD, troubleshoot production issues, and document solutions. Provide technical leadership, mentor engineers, participate in architecture and code reviews, and drive engineering best practices across data modernization initiatives.

Design, build, and operate scalable ETL/ELT pipelines and backend data services using Go. Integrate relational, NoSQL, blockchain, API, web, and social data sources; develop NLP pipelines for text extraction and classification; and support production APIs, dashboards, alerts, and intelligence systems. Collaborate with engineering, data science, and investigations teams on architecture, data quality, lineage, validation, monitoring, and client-facing products.

Leads data engineering and data science teams, setting strategy and delivering scalable pipelines, data architecture, models, analytics infrastructure, and data governance. Partners with product, analytics, and business stakeholders to prioritize initiatives, assess upstream changes, maintain data quality, and support Snowflake-based warehousing. This hands-on leadership role includes mentoring staff, contributing to technical codebases, and advancing data science and engineering solutions.

Build and operate scalable healthcare data pipelines, dbt models, event-driven services, APIs, and Google Cloud infrastructure. Model and govern multimodal EHR, genomic, and imaging data for AI agents across federated hospital networks. Ensure data quality, lineage, observability, security, HIPAA compliance, and production reliability while contributing to TypeScript application services and agentic workflows.

Design, build, and maintain scalable batch and streaming data pipelines using Python, SQL, Kafka, and GCP. Develop REST and GraphQL data services, CI/CD automation, observability, and data-quality frameworks. Integrate ML and NLP models into production workflows, including continuous retraining and real-time inference. Support structured and unstructured healthcare datasets, distributed systems, microservices, and reliable data infrastructure while collaborating with data science and MLOps teams.

Build and configure a government Databricks workspace, develop ingestion and integration pipelines from enterprise and legacy systems, and prepare data for reporting. Implement Unity Catalog governance, metadata management, auditing, and lineage; configure MLflow workflows; assess security controls for sensitive data; and create architecture documentation, data dictionaries, and SOPs to support long-term sustainment.

Lead the design and optimization of healthcare data pipelines integrating EHR, payer, HIE, and vendor data. Build AWS-based ETL/ELT infrastructure, reusable integration tooling, data quality monitoring, and ML enablement foundations. Ensure HIPAA-compliant handling of clinical data, including security controls, audit logging, encryption, and de-identification. Provide technical leadership, mentor engineers, influence platform architecture, and lead complex partner integrations involving FHIR, HL7v2, CCDA, and healthcare terminology standards.

Design and lead scalable, secure, highly available database infrastructure for multi-terabyte SQL Server environments. Set technical direction for data architecture, performance, real-time pipelines, security, and growth. Troubleshoot complex production issues, optimize queries and indexes, guide cross-functional projects, mentor engineers, review changes, reduce technical debt, and evaluate emerging technologies. Monitor database health using Splunk, Datadog, and Grafana while influencing engineering and Site Reliability strategy.

The Staff Data Reliability Engineer ensures data integrity, quality, freshness, and availability across enterprise pipelines. Responsibilities include defining data SLOs, implementing observability, building resilient Informatica and Python/PySpark pipelines, automating validation and recovery, eliminating operational toil, leading data incidents and postmortems, creating runbooks, and applying AIOps, machine learning, and DataOps practices to detect anomalies and self-heal data workflows.

Build scalable, secure data pipelines, models, platforms, and governance systems that enable AI applications to access trusted enterprise data. Partner with analytics, marketing, and product teams to translate requirements into technical solutions, independently deliver projects through production, and support sensitive-data security and access controls. Preferred experience includes LLM architectures, cloud data platforms, infrastructure as code, regulated environments, and technical leadership.

Design and scale lakehouse architecture, streaming and batch pipelines, and reliable data platforms using Kafka, Spark, Airflow, Iceberg, Databricks, and related technologies. Build Medallion-layer data systems, manage open table formats, optimize distributed queries, monitor platform reliability, resolve data quality issues, and collaborate with analysts, data scientists, and product teams.

Designs and operates scalable batch and streaming data pipelines and AI-ready data products across Databricks, Snowflake, Kubernetes, and cloud services. Develops governed data assets, semantic models, search and retrieval capabilities, and data quality processes. Supports Snowflake Cortex and Databricks initiatives, production troubleshooting, orchestration, performance optimization, CI/CD, and observability. Partners with business and technical teams on roadmaps and adoption, evaluates AI-assisted engineering tools, mentors engineers, and establishes reusable platform patterns.

Leads data engineering teams in designing, developing, and supporting scalable batch and streaming data solutions. Responsibilities include data pipeline development, architecture, governance, security, platform optimization, production support, technical mentorship, stakeholder collaboration, and delivery leadership. The role requires hands-on engineering with Python or similar languages, modern data technologies, cloud platforms, DevOps practices, and people-management responsibilities.

Lead the design, development, deployment, and operation of scalable data pipelines, APIs, services, and healthcare analytics applications. Translate business and analytical requirements into reliable production solutions, establish technical standards and architecture, troubleshoot issues, improve performance, document designs, coordinate platform changes, and provide technical leadership and mentorship to data engineers.

Design, build, and iterate data engineering pipelines, dashboards, and reports to support analytics and data science. Partner with stakeholders to translate requirements into engineering solutions, promote self‑service analytics, define best practices, and support implementation using Databricks, Snowflake, and Azure components.

Own and build the company’s data warehouse architecture, curated and semantic layers, data models, quality standards, and migration automation. Develop Redshift-based pipelines using Python, PostgreSQL, Fivetran, dbt, AWS, Airflow, and Terraform. Establish metrics governance, observability, infrastructure-as-code, AI-enabled enrichment, anomaly detection, metadata generation, and structured clinical-record ingestion. Partner with engineering, product, and commercial teams to define metrics and reliable platform change contracts while mentoring data engineers.

Build and own cloud-based data pipelines and governed data products supporting Employee Benefits Sales and Underwriting. Design ELT solutions, integrate complex data, improve data quality, and implement CI/CD and enterprise data engineering practices. Architect hybrid cloud solutions using AWS, Hadoop, Spark, Kafka, Snowflake, and related technologies. Influence architecture and tooling decisions, document technical guidance, mentor teams, evaluate emerging technologies, and contribute to strategic roadmaps.

Build and maintain scalable data platforms and systems that transform raw market data into validated analytical datasets, including bars, residuals, curves, volatility surfaces, and greeks. Collaborate with trading, research, and external vendor teams to understand datasets and workflows, implement calculations, document technical solutions, and ensure reliable delivery through strong testing and validation practices.

Build and maintain centralized, validated data platforms covering reference, fundamental, market, index, exchange, instrument, and security master datasets. Analyze vendor data, design scalable data models, implement quality controls, resolve processing issues, and partner with quantitative researchers and traders. Provide technical leadership and mentorship while maintaining testing and system design standards across the Core Data team.

Design, build, and maintain ETL/ELT pipelines and API integrations to ingest, cleanse, transform, and analyze infrastructure and cloud data. Produce analytics, dashboards, and executive reports; automate workflows and support near-real-time processing. Follow SDLC/DevOps practices (Git, CI/CD), ensure data governance, quality, lineage, and support incident, release, and compliance activities.

Designs and maintains ELT pipelines, warehouse integrations, data models, and data quality processes. Collaborates with data scientists, analysts, engineers, and stakeholders to deliver scalable analytical solutions. Responsibilities include data integration, governance, security, performance optimization, exploratory analysis, visualization, and cloud-based API ingestion. Requires strong SQL, DBT, Snowflake, Azure Data Factory, cloud, and scripting experience, plus leadership and communication skills.

Designs, builds, and maintains scalable Azure data pipelines and ELT frameworks using Databricks, SQL, Python, and PySpark. Implements Azure DevOps CI/CD, monitoring, security, infrastructure automation, and production support. Collaborates with BI teams on Power BI datasets, documents data solutions, participates in Agile delivery, and mentors junior engineers. The role requires strong data modeling, structured and semi-structured data expertise, performance tuning, and data governance awareness.

Develops scalable data and software systems using Python and Apache Spark. Designs APIs, data models, ETL workflows, orchestration, and data interactions while optimizing performance, scalability, integrity, and security. Builds user interfaces, contributes to automated testing, applies Agile practices, and supports CI/CD delivery. The role requires strong software engineering, distributed computing, analytical, and communication skills, with AWS, R, Databricks, and SAS experience preferred.