Operate and optimize large-scale LLM pre-training on 1,000+ GPU clusters using PyTorch, DeepSpeed, or Megatron-LM. Improve networking (InfiniBand/RDMA), memory management, checkpointing, and failure recovery. Manage SLURM/Kubernetes GPU clusters and apply systems engineering (C++, CUDA, Python) and 3D parallelism techniques.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Healthtech • Software
The Payment Selections Manager leads the identification and recovery of improper claim payments, collaborates with cross-functional teams, and improves payment integrity processes while conducting data analysis and audits.
Top Skills:
CptHcpcsIcd-10Excel
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Serve as primary legal partner to product, engineering, and design teams building stablecoins, payments, and crypto infrastructure. Provide full product lifecycle counseling on U.S. and international payments and securities regulations, resolve novel legal issues, partner cross-functionally with Compliance, Finance, and Risk, and build scalable legal processes and playbooks to enable compliant product launches.
Top Skills:
BlockchainCryptoGenerative AiStablecoins
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Lead technical product management across the full product lifecycle: define strategy, roadmaps, and specs; drive innovation and commercialization; collaborate with cross-functional teams using Agile; oversee cloud and data projects; mentor staff; manage budgets, planning, and stakeholder conflicts while upholding professional standards.
Top Skills:
AgileCloud EngineeringCustom Application DevelopmentData & AnalyticsProduct Lifecycle Management
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
