Responsible for maintaining and enhancing data warehouses and pipelines, developing scalable ETL processes, data analysis, and reporting through visualization tools like PowerBI.
Job Overview
We are seeking a skilled Data Engineer to join our team and drive our data infrastructure forward. In this role, you will primarily focus on maintaining and enhancing our data warehouse and pipelines (80%) while also contributing to data analysis and reporting initiatives (20%). You'll work closely with cross-functional stakeholders to build robust data solutions and create actionable insights through compelling visualizations.
Key Responsibilities
Data Engineering
- Infrastructure Management: Maintain, enhance, and optimize existing data warehouse architecture and ETL pipelines.
- Pipeline Development: Design and implement scalable ETL/ELT processes ensuring data quality, integrity, and timeliness.
- Performance Optimization: Monitor and improve pipeline performance, troubleshoot issues, and implement best practices.
- Documentation: Create and maintain comprehensive documentation for data engineering processes, architecture, and configurations.
Data Analysis & Reporting
- Stakeholder Collaboration: Partner with business teams to gather requirements and translate them into technical solutions.
- Report Development: Build and maintain PowerBI dashboards and reports that drive business decisions.
- Data Modeling: Develop new data models and enhance existing ones to support advanced analytics.
- Insight Communication: Transform complex data findings into clear, actionable insights for various departments.
Required Qualifications
Technical Skills
- Programming & Query Languages: Strong proficiency in Python, SQL, and PySpark.
- Big Data Platforms: Experience with cloud data platforms including Snowflake, BigQuery, and Databricks. Databricks experience highly preferred.
- Orchestration Tools: Proven experience with workflow orchestration tools (Airflow preferred).
- Cloud Platforms: Experience with AWS (preferred), Azure, or Google Cloud Platform.
- Data Visualization: Proficiency in PowerBI (preferred) or Tableau.
- Database Systems: Familiarity with relational database management systems (RDBMS).
Development Practices
- Version Control: Proficient with Git for code management and collaboration.
- CI/CD: Hands-on experience implementing and maintaining continuous integration/deployment pipelines.
- Documentation: Strong ability to create clear technical documentation.
Experience & Communication
- Professional Experience: 3+ years in data engineering or closely related roles.
- Language Requirements: Fluent English communication skills for effective collaboration with U.S. based team members.
- Pipeline Expertise: Demonstrated experience building and maintaining production data pipelinesk
Similar Jobs
Security • Cybersecurity
Design and maintain data pipelines, models, and transformation layers for analytics. Build data contracts, observability and lineage tooling, collaborate with product engineering, and optimize data structures for performance and scalability.
Top Skills:
Cloud-Native ArchitecturesData ContractsData Observability FrameworksDimensional ModelingDockerGoIncremental Data ModelsJvm (Java/Scala/Kotlin)KubernetesLineage TrackingMessage QueuingNode.jsPythonStream Processing
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Design, build, and operate enterprise data pipelines and Lakehouse data products on Azure. Lead migrations, implement medallion architectures, optimize Spark and distributed SQL workloads, enforce CI/CD and observability, and support production incident response while partnering with architects, data scientists, and stakeholders.
Top Skills:
ArmAzure Ad / Entra IdAzure Data FactoryAzure Data Lake Storage Gen2Azure DatabricksAzure DevopsAzure Key VaultAzure MonitorAzure SqlAzure Synapse AnalyticsBashBicepDelta LakeKqlLog AnalyticsMicrosoft FabricPandasParquetPowershellPrivate EndpointsPysparkPytestPythonSpark SqlSsisSynapse Dedicated Sql PoolT-SqlTeradataTerraformUnity CatalogVnet
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Lead design and operation of large-scale data pipelines and data APIs. Build Spark/PySpark workflows on Databricks, optimize job performance, manage data quality and observability, and develop MCP servers and AI-agent integrations. Mentor engineers, define standards, support production incidents and on-call rotations, and collaborate with stakeholders to deliver scalable data platform products.
Top Skills:
Apache IcebergApi GatewayAWSAws LambdaAws Rds/AuroraAzureDatabricksDatadogDbtFastapiFivetranGCPGoogle BigqueryLlms/Ai AgentsMcp ServersMs Sql ServerMySQLOraclePostgresPysparkPythonS3SecretsmanagerSnowflakeSnsSparkSplunkSQLSqs
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
