Waabi Logo

Waabi

Senior / Staff ML Ops Engineer

Posted 2 Days Ago
Remote or Hybrid
Hiring Remotely in Canada
184K-272K Annually
Senior level
Remote or Hybrid
Hiring Remotely in Canada
184K-272K Annually
Senior level
Build and operate a reliable ML platform for large-scale distributed GPU training. Responsibilities include Kubernetes infrastructure, developer-facing CLIs and SDKs, AWS infrastructure, data and artifact management, experiment tracking, model registries, CI/CD, observability, documentation, security guardrails, and platform adoption. The role partners closely with researchers and engineers to improve training speed, reliability, usability, and cost efficiency for autonomous transportation systems.
The summary above was generated by AI
Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With a world-class team, we're unlocking the next era of autonomous transportation with technology that's powering commercial autonomous trucks and robotaxis. Waabi is backed by and partners with world leaders in AI, automotive, logistics, and deep tech.
 
With offices in Toronto, San Francisco, Dallas, and Pittsburgh, Waabi is growing quickly and looking for diverse, innovative and collaborative candidates who want to impact the world in a positive way. To learn more visit: www.waabi.ai

You will..
  • Build and evolve our training infrastructure on Kubernetes with Infrastructure — GPU scheduling, autoscaling, multi-node distributed jobs, capacity strategy, and the operators and workflow engines that keep long-running training reliable.
  • Shape the developer-facing surface — CLIs, SDKs, job submission, templates, paved paths — designed with the teams who'll use them. Make the common case one command and keep the uncommon case possible.
  • Shorten the inner loop. Time to first training run, edit-to-signal latency, local iteration before a job hits the cluster, fast failure over slow mystery. Measure it, publish it, drive it down.
  • Evangelize best-in-class tooling and frameworks. Track what the ecosystem is shipping, evaluate honestly, and make the case with working prototypes and migration paths — or say plainly when a shiny thing isn't worth the switching cost.
  • Strengthen the data and artifact layer. Dataset versioning, sharding, and high-throughput loading of large multimodal sensor data, so jobs saturate GPUs instead of waiting on I/O.
  • Turn one-off Python into durable tooling — tested, documented, observable libraries, CLIs, and services with sane defaults, and deletions where they're overdue.
  • Make experiments legible, with the teams who live in them: experiment hygiene, dashboards researchers trust, a real model registry, and lineage from dataset to checkpoint to simulation result.
  • Ship CI/CD for models alongside autonomy and simulation, so a model change is validated the same way a code change is.
  • Build observability across the ML stack — utilization, throughput, failure modes, queue times, cost per experiment. When a job fails at 3am on node 47, the researcher should find out why without you.
  • Treat docs, onboarding, and support as product surface — golden-path guides, a new researcher productive on day two, office hours that turn repeat questions into shipped fixes.
  • Drive adoption, not just availability. Prototype with real users, watch them work, iterate. A tool nobody adopts didn't ship.
  • Make the platform boringly reliable — fewer failures, faster recovery, and none of the manual steps that quietly cost a team days.
  • Build guardrails that don't feel like walls, with Security, IT, and Infrastructure: access controls, data handling, and cost governance that hold up in an IP-sensitive environment while staying self-serve.
 
Qualifications:
  • 5+ years of software or infrastructure engineering, including tools or platforms used by other engineers and operating ML or data-intensive production systems.
  • Hands-on Kubernetes expertise — GPU scheduling, autoscaling, Helm or equivalent, networking fundamentals, and the ability to debug a cluster under load rather than restart it.
  • Excellent Python, and a track record of designing APIs and CLIs other people enjoy using.
    Practical AWS depth: object storage at scale, IAM, GPU compute, networking, cost management, and infrastructure as code (Terraform, Pulumi, or similar).
  • Distributed training in PyTorch (DDP, FSDP, or similar), plus experiment tracking and model registry tooling — from the perspective of someone who made them pleasant for others to use.
  • Fluency with containers, CI/CD, and modern build systems, including large monorepos.
  • The ability to influence without authority: evaluate a framework on its merits, pilot it credibly, and persuade skeptical senior engineers to change how they work.
  • A collaborative default — you'd rather co-own a system than draw a boundary around your part of it.
  • User empathy: you'd rather fix the third-most-interesting problem blocking ten people than the most interesting one blocking nobody.
  • Strong product instincts, strong writing, and comfort operating autonomously in ambiguous territory.
  • Passionate about self-driving technologies and frontier AI, and about what a small, world-class team can do with the right infrastructure.
 
Bonus/nice to have: 
  • Internal developer platform, research platform, or DevEx work — with a story about a tool whose adoption you grew from zero.
  • Large-scale distributed GPU training: hundreds to thousands of accelerators, NCCL, high-performance cluster networking, collective communication tuning.
  • High-throughput loading of LiDAR or camera data, and formats such as Parquet or WebDataset.
  • Workflow and scheduling systems — Argo Workflows, Ray, Flyte, Kubeflow, or Slurm.
  • Build-system depth (Bazel or similar), including remote caching in a monorepo.
  • Simulation infrastructure or large-scale batch evaluation pipelines.
  • Background in ML, robotics, or autonomous systems infrastructure.
  • Security- and IP-sensitive production environments.
  • Open-source contributions to ML infrastructure or developer tools.

The US yearly salary range for this role is: $184,000 - $272,000 USD in addition to competitive perks & benefits. Waabi (US) Inc.’s yearly salary ranges are determined based on several factors in accordance with the Company’s compensation practices. The salary base range is reflective of the minimum and maximum target for new hire salaries for the position across all US locations.  Note: The Company provides additional compensation for employees in this role, including equity incentive awards and an annual performance bonus.

Perks/Benefits:
  • Competitive compensation and equity awards.
  • Health and Wellness benefits encompassing Medical, Dental and Vision coverage (for full-time employees only).
  • Unlimited Vacation.
  • Flexible hours and Work from Home support.
  • Daily drinks, snacks and catered meals (when in office).
  • Regularly scheduled team building activities and social events both on-site, off-site & virtually.
  • As we grow, this list continues to evolve! 
 
Waabi is a technology start-up building technologies to transform the way the world moves. Join our talented team to be a part of the future and to make an impact!
 
Waabi is an equal opportunity employer. We celebrate diversity and are committed to creating a supportive, inclusive, and accessible workplace for all our employees. We seek applicants of all backgrounds and identities, across race, color, ethnicity, national origin or ancestry, age, citizenship, religion, sex, sexual orientation, gender identity or expression, military or veteran status, marital status, pregnancy or parental status, caregiver status, disability, or any other characteristic protected by law. We make workplace accommodations for qualified individuals with disabilities as required by applicable law. If reasonable accommodation is needed to participate in the job application or interview process please let our recruiting team know.

Similar Jobs

5 Hours Ago
Easy Apply
Remote
Easy Apply
181K-241K Annually
Senior level
181K-241K Annually
Senior level
Big Data • Fintech • Mobile • Payments • Financial Services
Lead technical strategy and execution for foundational Trust and Safety backend systems supporting consumer risk and credit reporting. Design and operate highly available distributed systems, improve reliability, monitoring, testing, and on-call processes, and guide cross-functional initiatives. Establish engineering standards, conduct technical planning, mentor engineers, communicate architectural decisions, and drive system quality, scalability, and sustainability across teams.
Top Skills: SparkAWSKotlinKubernetesMySQLPython
Yesterday
Remote or Hybrid
Senior level
Senior level
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Leads ServiceNow’s digital content strategy, editorial standards, AI-ready content systems, governance, and optimization across enterprise digital experiences. Develops narrative frameworks, content models, taxonomies, and publishing standards; partners with Product Marketing, Brand, UX, SEO, Analytics, and Engineering; improves discoverability, personalization, customer journeys, and conversion; and enables distributed teams through playbooks, training, audits, and governance.
Top Skills: Adobe Experience ManagerAi-Powered SearchBehavioral AnalyticsContent Management SystemsExperimentation PlatformsPersonalization EnginesSeo
Yesterday
Easy Apply
Remote
United States
Easy Apply
250K-349K Annually
Expert/Leader
250K-349K Annually
Expert/Leader
Cloud • Security • Software • Cybersecurity • Automation
Leads technical direction for GitLab’s Core DevOps services, covering CI, CD, planning, and source control. Defines architecture, quality metrics, migration strategies, and technical roadmaps; addresses systemic risk and complex design decisions; prototypes and writes code when needed; partners with Infrastructure, Security, SRE, and Product; adopts AI capabilities; mentors senior engineers; supports incident response and senior technical hiring.
Top Skills: Agentic FrameworksAIAutonomous WorkflowsCi/CdDevOpsDistributed SystemsFault ToleranceGitlabHuman-In-The-Loop ControlsLarge Language ModelsMachine LearningMulti-Tenant SystemsObservabilityResponsible Ai

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account