EXL Logo

EXL

Senior Data Engineer

Reposted One Month Ago
Remote or Hybrid
Hiring Remotely in United States
94K-154K Annually
Senior level
Remote or Hybrid
Hiring Remotely in United States
94K-154K Annually
Senior level
Design, build, and operate production-grade, event-driven data pipelines (Kafka/Flink) on GCP to deliver model-ready features. Optimize BigQuery SQL and Parquet performance, develop Python data workloads (Polars/Pandas), deploy ML pipeline components on Kubeflow/Vertex AI with Docker, design event store architectures, and collaborate with ML and platform teams while documenting architecture and standards.
The summary above was generated by AI

EXL is hiring a Senior Data Engineer to join a strategic AI / ML platform engagement with a leading specialty retailer. This is a hands-on build role embedded with the client's platform engineering team.

The role requires shipping production-grade data pipelines that feed real-time customer event data into machine learning workflows. The right person is comfortable owning the full lifecycle of pipeline design, build, and deployment: from streaming ingestion through event store design to model-ready feature delivery.

This is a high-visibility role with growth potential into a larger book of work as the engagement expands.

Salary Range: $93,900 - $154,200 annual base 

The posted range is the hiring range for this role — a subset of the broader range available to employees over time — and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process.

Responsibilities
What You'll Do
  • Design and operate event-driven data pipelines using Kafka consumers and Flink jobs to process high-volume customer events (clicks, purchases, returns) in near-real time.
  • Build and optimize large-scale data transformations on Google Cloud Platform — BigQuery SQL, query performance tuning, and partitioning strategy at scale.
  • Develop Python data engineering workloads using Polars or Pandas at scale, with rigorous attention to Parquet partitioning, join performance on large datasets, and memory efficiency.
  • Build, deploy, and maintain ML pipeline components on Kubeflow Pipelines (KFP) and Vertex AI; package and deploy services with Docker.
  • Design event store architecture: partitioning by customer, time-ordered event assembly across heterogeneous sources, and schema management for mixed event types.
  • Partner with ML engineers, platform engineers, and data scientists to deliver clean, performant, model-ready data products.
  • Document architecture decisions and contribute to engineering standards across the platform team.
Qualifications
Required Skills & Experience
  • 6–12 years of experience in data engineering, platform engineering, or a closely related discipline.
  • Streaming: Production experience with Kafka consumers and Flink stream processing — building, deploying, and operating streaming jobs at meaningful scale.
  • GCP Data Stack: Strong SQL on BigQuery (or an equivalent cloud warehouse), with demonstrated query optimization, cost management, and partitioning chops.
  • Python Data Engineering: Hands-on with Polars or Pandas at scale; deep working knowledge of Parquet partitioning and performance on large joins.
  • ML Pipelines: Hands-on experience building and deploying components on Kubeflow Pipelines (KFP) and/or Vertex AI Pipelines; working proficiency with Docker.
  • Event Store Design: Demonstrated experience designing event stores — partitioning by customer, time-ordered event assembly across sources, schema strategy for mixed event types (clicks, purchases, returns).
  • Communication: Strong written and verbal communication; comfortable being the senior IC voice in design conversations with client stakeholders.
Nice to Have
  • Domain experience in Retail or E-commerce — customer journey data, transaction analytics, returns and exchanges modeling.
  • Exposure to schema registry tooling (e.g., Confluent), Iceberg, or Delta Lake.
  • Experience working in client-facing or consulting engagements.
  • Google Cloud certifications (Professional Data Engineer or equivalent).
Work Arrangement & Eligibility
  • This role requires 3–4 days per week onsite in Seattle, WA. Fully remote and out-of-state candidates will not be considered.
  • EXL is open to sponsoring H1B transfers for qualified candidates.

Similar Jobs

5 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
125K-234K Annually
Senior level
125K-234K Annually
Senior level
Fintech • Mobile • Software • Financial Services
Build and maintain production data pipelines, dbt models, Airflow workflows, and governed datasets supporting People Analytics. Integrate Workday and other HR systems into reliable Snowflake data products, implement data quality monitoring, row-level security, semantic models, and AI agents. Partner with engineers, analysts, data scientists, and business stakeholders to deliver trusted data solutions, communicate technical trade-offs, enforce governance, and mentor other engineers.
Top Skills: Ai AgentsAmazon RedshiftApache AirflowCi/CdDbtEtl/EltGitlabGoogle Cloud PlatformLlmsPythonSnowflakeSQLWorkday
Yesterday
In-Office or Remote
180K-240K Annually
Senior level
180K-240K Annually
Senior level
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Build and operate distributed data and AI infrastructure for Airwallex’s Knowledge Platform. Responsibilities include architecting high-availability systems, optimizing state management and exactly-once processing, integrating streaming, vector indexing and agentic workflows, processing petabytes of data globally, and mentoring engineers through technical leadership.
Top Skills: Apache FlinkSparkClickhouseDatabricksGoJavaKafkaKotlinKubernetesOlap Engines
4 Days Ago
In-Office or Remote
CA, USA
185K-327K Annually
Senior level
185K-327K Annually
Senior level
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Design, build, and operate shared backend platforms and frameworks for data acquisition, processing, and access. Lead projects from architecture through production, develop event-driven and batch or streaming systems, and improve reliability, observability, scalability, and data quality. Collaborate with software engineers, data engineers, and data scientists, make architectural trade-offs, mentor engineers, and contribute to technical standards. The role may also involve external data integrations and analytics and machine-learning data systems.
Top Skills: AWSCloudeventsDatabricksDatahubDbtIcebergJavaKafkaKotlinPrefectSnowflakeSparkSpark Structured Streaming

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account