Similar Jobs
Fintech • Information Technology • Software • Financial Services
The Data Engineer will build real-time data pipelines for pricing algorithms, collaborate with teams, and contribute to batch data workflows.
Top Skills:
Cloud-Based Distributed Data InfrastructureFlinkKafkaPythonSQL
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
As a Data Engineer II, you will build and optimize data pipelines in Databricks, ensure high-quality data delivery, and support generative AI applications. You will collaborate with data scientists and AI engineers to provide reliable data infrastructures and meet business needs.
Top Skills:
DatabricksDbtGainsightGongOutreachPythonRdsRedshiftSalesforceSnowflakeSparkSQL
.png)
Food • Software
The Senior Data Engineer handles ChowNow's data platform, collaborating with teams to enhance data availability and insights, supporting internal and customer-facing products.
Top Skills:
AWSDbtPythonSnowflakeSQL
hatch I.T. is partnering with Packaged Agile to find a Data Engineer. See details below:
About the Role:
As a Data Engineer, you will play a crucial part in architecting and implementing data solutions that drive business value and enable informed decisionmaking. As a key member of their team, you will collaborate closely with cross-functional teams to understand business requirements, design robust data pipelines, and leverage cutting-edge technologies such as Databricks, Python, and various AWS services. If you thrive in a dynamic and collaborative environment and are driven by the opportunity tosolve complex challenges, we encourage you to apply and join their team at Packaged Agile.
They are proud to be a United States Small Business Administration (SBA) HUBZone certified company. Though not a requirement for this position, we strongly prefer candidates who live in an SBA HUBZone.
About the Company:
Packaged Agile is a boutique, agile, and digital services-focused coaching and consulting company. We help our clients deliver measurable value to their customers and aspire to bring "good agile" to clients across the public and private sectors.
We hire passionate people who believe in agility, understand how to meet people where they are, and treat them with respect while helping them succeed. And we want to hire people that see their continuous growth and improvement as a way of life.
Responsibilities:
- Collaborate with Artificial Intelligence and Machine Learning team members to operationalize data pipelines and ML tasks.
- Provide day-to-day support for deploying Python-native ML pipelines.
- Implement efficient data ingestion strategies and change data capture mechanisms, leveraging Databricks and AWS services, to ensure real-time and accurate data updates for analytics and decision-making processes.
- Utilize Databricks to design, develop, and maintain scalable data pipelines, enabling efficient extraction, transformation, and loading (ETL) processes.
- Leverage Python programming expertise to enhance data processing scripts, ensuring robustness, efficiency, and maintainability in AWS Lambda environments.
- Execute data engineering tasks to facilitate AI/ML capabilities.
- Communicate results effectively to diverse audiences through presentations.
- Assist in architectural leadership, technical support, and advisement services to ensure integration of identity management system technologies meets security requirements.
- Provide support to leadership engaging with senior-level executives at a public facing Federal agency, offering subject matter expertise in security architecture and other key domains.
- Identify and address problems and inefficiencies, implementing solutions.
- Identify and optimize data bottlenecks, leveraging automation where applicable.
- Establish and manage data lifecycle policies, including retention backups/restores
Requirements:
- US Citizenship with the ability to receive a Public Trust clearance or higher
- Must be eligible to obtain a Public Trust Clearance.
- Minimum 7 years of experience in Software Development focused on data proficiency in designing and maintaining data architectures to meet business needs.
- Experience in acquiring, processing, and optimizing datasets.
- Ability to support AI/ML teams by enhancing feature engineering code.
- Strong familiarity with Spark, including regular updates.
- Skilled in creating, managing, and optimizing Spark Structured Streaming jobs, leveraging technologies like Delta Live Tables and/or DBT.
- Competence in developing and managing ksqlDB and Kafka Streams queries/code.
- Experience maintaining and updating Python-based data processing scripts executed on AWS Lambdas.
- Commitment to conducting unit tests for all Spark, Python data processing, and Lambda code.
- Bachelor’s degree in computer science or a related field.
- Strongly preferred: candidates who live in an SBA HUBZone
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
