Design, develop, and optimize enterprise-scale data platforms, pipelines, data lakes, warehouses, and analytical datasets on Azure and Snowflake. Build ETL/ELT processes using ADF, Databricks, PySpark, SQL, Python, and Snowflake. Manage data models, governance, security, quality, CI/CD, cloud migrations, and performance optimization. Collaborate with stakeholders, troubleshoot production issues, document solutions, and support Agile delivery.
- Azure Data Engineer with 6+ years of experience designing, developing, and optimizing enterprise-scale data platforms using Microsoft Azure and Snowflake
- Proven expertise in building scalable data pipelines, data warehouses, and analytics solutions leveraging Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Snowflake, Python, SQL, and Spark
- Experienced in data modeling, ETL/ELT development, cloud migration, performance tuning, data governance, and implementing CI/CD practices
- Strong background in integrating data from multiple sources and enabling business intelligence and advanced analytics solutions
Key Skills: - Azure Data Factory (ADF)
- Azure Databricks
- Azure Synapse Analytics
- Snowflake Data Cloud
- Python, PySpark, SQL
- Azure Data Lake Storage (ADLS Gen2)
- Azure Functions
- Delta Lake
- Data Warehousing & Data Modeling
- ETL / ELT Development
- CI/CD (Azure DevOps, Git)
- Performance Tuning & Optimization
- Data Governance & Security
- Power BI Integration
- Agile & Scrum Methodologies
What you will do:
- Design, develop, and maintain scalable data engineering solutions on Azure and Snowflake platforms
- Build and optimize ETL/ELT pipelines using Azure Data Factory, Databricks, and Snowflake to ingest, transform, and load data from various structured and unstructured sources
- Develop and maintain enterprise data warehouses, data marts, and analytical datasets supporting reporting, BI, and advanced analytics initiatives
Implement data transformation and orchestration processes using PySpark, SQL, Python, and Snowflake stored procedures - Design and manage cloud-based data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake architecture
- Collaborate with business stakeholders, data analysts, and architects to gather requirements and translate them into scalable data solutions
- Monitor and optimize data pipeline performance, ensuring reliability, scalability, and cost efficiency
- Implement data quality frameworks, validation rules, and monitoring processes to ensure accurate and trusted data
- Manage Snowflake objects including databases, schemas, warehouses, tasks, streams, and data-sharing capabilities
- Develop and maintain dimensional and normalized data models following industry best practices
- Implement security controls, access management, encryption, and governance policies across Azure and Snowflake environments
- Support cloud migration and modernization initiatives from on-premises or legacy platforms to Azure and Snowflake
- Automate deployment processes using Azure DevOps, Git repositories, and CI/CD pipelines
- Troubleshoot production issues, perform root cause analysis, and provide timely resolution for data-related challenges
- Participate in Agile ceremonies, code reviews, and technical design discussions to ensure high-quality solution delivery
- Create technical documentation, architecture diagrams, operational runbooks, and knowledge-transfer materials
Preferred Certifications: - Microsoft Certified: Azure Data Engineer Associate (DP-203)
- SnowPro Core Certification
Similar Jobs
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Build and operate low-latency market data systems for institutional trading, including feed handlers, normalization pipelines, venue connectivity, and distribution services. Develop high-throughput services, improve reliability and performance through observability and incident response, participate in on-call support, and collaborate with engineering and product teams. The role requires production backend engineering experience, market data infrastructure expertise, Java or C++, messaging frameworks, and exchange connectivity protocols.
Top Skills:
AeronC++FixItch/OuchJavaMulticastSbe
.png)
Artificial Intelligence • Cloud • Payments • Software • Business Intelligence • Generative AI • Automation
Define and govern enterprise-scale data architecture across batch, streaming, warehouse, lakehouse, transactional, and AI use cases. Establish standards for data quality, lineage, access, cataloging, governance, observability, and SLAs. Architect AI-enabled workflows, resolve complex architecture issues, influence roadmaps, and mentor engineers through hands-on technical leadership. The role requires 15+ years of software, data engineering, or architecture experience and expertise in large-scale data platforms and modeling.
Top Skills:
AIBatch ProcessingBigQueryData CatalogsData WarehousesDbtFeature StoresGCPLakehousesOlapOltpStreaming ArchitecturesVector Stores
Fintech • News + Entertainment • Software • Database • Financial Services
Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.
Top Skills:
Amazon EcsAmazon KinesisAmazon RedshiftAmazon S3Amazon SqsApache AirflowApache FlinkAWSAws GlueAws LambdaBeautifulsoupCi/CdDatabricksDockerGreat ExpectationsKafkaMonte CarloMwaaPythonScrapySnowflakeSQLTerraform
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
