Design, build, and optimize scalable AWS data pipelines and serverless solutions for utilities datasets. Orchestrate workflows (Step Functions/Airflow), implement ETL/ELT with PySpark/Python, model data for analytics, ensure performance, reliability, security, and compliance while collaborating with cross-functional stakeholders.
❋ Why Scalepex?
Scalepex is a dynamic services firm specializing in providing solutions for premium brands like Nike, Pepsi, Toyota, Virgin and Walgreens. Our mission is to connect prominent market leaders with top-tier professionals from around the world, fostering collaboration, efficiency, and growth.
❋ Take your portfolio to the next level by working with one of our fastest growing clients.
Join the Innovation Frontier at Scalepex!
About the Role
We are seeking an experienced AWS Data Engineer with a strong background in building scalable data solutions and expertise in utilities-related datasets. The ideal candidate will have at least 5 years of experience in data engineering, a deep understanding of distributed systems, and proficiency with AWS services and tools like Step Functions, Lambda, Glue, and Redshift. This role will focus on designing, developing, and optimizing data pipelines to support analytics and decision-making in the utilities industry.
Key Responsibilities
- Design and Build Data Pipelines: Develop scalable, reliable data pipelines using AWS services (e.g., Glue, S3, Redshift) to process and transform large datasets from utility systems like smart meters or energy grids.
- Workflow Orchestration: Use AWS Step Functions to orchestrate workflows across data pipelines; experience with Airflow is acceptable but Step Functions is preferred.
- Data Integration and Transformation: Implement ETL/ELT processes using PySpark, Python, and Pandas to clean, transform, and integrate data from multiple sources into unified datasets.
- Distributed Systems Expertise: Leverage experience with complex distributed systems to ensure reliability, scalability, and performance in handling large-scale utility data.
- Serverless Application Development: Use AWS Lambda functions to build serverless solutions for automating data processing tasks.
- Data Modeling for Analytics: Design data models tailored for utilities use cases (e.g., energy consumption forecasting) to enable advanced analytics
- Optimize Data Pipelines: Continuously monitor and improve the performance of data pipelines to reduce latency, enhance throughput, and ensure high availability.
- Ensure Data Security and Compliance: Implement robust security measures to protect sensitive utility data and ensure compliance with industry regulations.
Requirements
Required Qualifications
- Minimum of 5 years of experience in data engineering
- Proficiency in AWS services such as Step Functions, Lambda, Glue, S3, DynamoDB, and Redshift.
- Strong programming skills in Python with experience using PySpark and Pandas for large-scale data processing.
- Hands-on experience with distributed systems and scalable architectures.
- Knowledge of ETL/ELT processes for integrating diverse datasets into centralized systems.
- Familiarity with utilities-specific datasets (e.g., smart meters, energy grids) is highly desirable.
- Strong analytical skills with the ability to work on unstructured datasets.
- Knowledge of data governance practices to ensure accuracy, consistency, and security of data.
- Strong experience in AWS data engineering
- Ability to work independently
- Ability to work with a cross-functional teams, including interfacing and communicating with business stakeholders
- Professional oral and written communication skills
- Strong problem solving and troubleshooting skills with experience exercising mature judgement
- Excellent teamwork and interpersonal skills
- Ability to obtain and maintain the required clearance for this role
Similar Jobs
Artificial Intelligence • Enterprise Web • Software • Design • Generative AI
Build and maintain GTM automation, data modeling, and integrations across SFDC, Marketo, Clay, and Snowflake. Develop predictive scoring models, optimize automated outreach and lifecycle workflows, manage automation cost efficiency (LLM and enrichment usage), and partner with marketing, sales, partnerships, and revenue ops to translate business needs into scalable technical solutions and documentation.
Top Skills:
APIsClayLlmMakeMarketoMcp IntegrationsN8NSalesforce (Sfdc)SnowflakeZapier
Artificial Intelligence • Enterprise Web • Software • Design • Generative AI
Own and grow strategic technology partnerships end-to-end: sourcing, negotiation, integration scoping, cross-functional alignment, co-marketing/co-selling, KPI tracking, and ecosystem strategy to drive product integrations and influenced revenue.
Top Skills:
Adobe ExchangeAffinityAPIsAws MarketplaceCmsCrossbeamHubspot App MarketplaceLoomModel Context Protocol (Mcp)No-CodeSalesforceSalesforce AppexchangeSlackWebflow
Beauty • Robotics • Design • Appliances • Manufacturing
The Senior Manager, Marketing Analytics will lead AI-driven marketing analytics, develop performance metrics, and collaborate across teams to enhance data-driven decision-making.
Top Skills:
AIData VisualizationDbtEltETLFivetranMlSnowflake
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
