ClickUp Logo

ClickUp

Senior AI Developer

Posted 10 Days Ago
Remote or Hybrid
Hiring Remotely in United States
200K-250K Annually
Senior level
Remote or Hybrid
Hiring Remotely in United States
200K-250K Annually
Senior level
Build and own ClickUp’s AI evaluation systems, agent harness, self-improvement loops, model benchmarking, routing, and fallback strategies. Architect scalable backend and AI platform services for LLM deployment, orchestration, monitoring, logging, and automated evaluation. Improve reliability, privacy, security, and compliance while integrating AI features into the core product and collaborating cross-functionally.
The summary above was generated by AI

At ClickUp, we're building the future of work: the first truly converged AI workspace unifying tasks, docs, chat, calendar, and enterprise search, all supercharged by context-driven AI. We are an AI-native company. Every team member is expected to leverage AI daily, and we evaluate AI fluency as part of our hiring process. Join us and help redefine what's possible. 🚀

We're looking for a highly skilled and experienced Senior AI Engineer to join our ClickUp Engineering team, with a focus on AI evaluations. You'll own how we measure and improve the quality of ClickUp's AI: building the eval systems that tell us how well our AI is performing, the core agent harness, and the AI that performs well on those evals. You'll work on backend systems that make AI-powered capabilities scalable, reliable, and secure, while working hands-on with LLMs to solve real user problems and drive product innovation.

 

Key Responsibilities:

  • Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and automated scorers, regression detection, and quality tracking across AI features.

  • Build and evolve the core agent harness, and the AI that performs well on those evals, partnering closely with product and engineering to define what "great" looks like.

  • Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate automatically.

  • Benchmark new models as they're released and own model selection, routing, and fallback strategies across providers (e.g., OpenAI, Anthropic, Google, open-weight), balancing quality, latency, and cost.

  • Architect, design, and implement scalable AI platform services that support the deployment, orchestration, and lifecycle management of LLMs and other AI models.

  • Build and maintain robust APIs and backend systems that integrate AI-powered features into ClickUp's core platform.

  • Develop infrastructure for model serving, monitoring, logging, and automated evaluation to ensure high reliability and performance of AI services in production.

  • Drive best practices in AI privacy, security, and compliance, including data anonymization, secure data handling, and regulatory adherence.

  • Collaborate cross-functionally with product, frontend, and data teams to deliver seamless, reliable, and user-centric AI experiences.

 

Qualifications:

  • 5+ years of software engineering experience, including 2+ years building production LLM or ML systems.

  • Hands-on experience building LLM evaluation systems in production (golden datasets, LLM-as-judge, judge calibration, offline and online evals), tied to real product quality.

  • Strong experiment design and statistical rigor, including A/B testing, offline vs. online metrics, and separating real regressions from noisy or variable results.

  • Proven track record of applying LLMs and AI models to real-world, customer-facing product features.

  • Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and cost/latency/quality tradeoffs.

  • Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g., Kubernetes, Docker, AWS/GCP/Azure).

  • Strong programming skills in Python (preferred), Go, TypeScript, or similar languages used for backend and AI platform development.

  • Proficiency in orchestration frameworks and workflow engines (e.g., LangGraph, Airflow, Ray, or similar).

  • Familiarity with search and retrieval technologies and their integration into AI-driven applications.

  • Excellent collaboration and communication skills, with a track record of working effectively in cross-functional teams.

  • Passion for staying at the forefront of AI and applying new technologies to solve real-world problems at scale.

 

Nice to Have:

  • Background in data science, applied science, or machine learning engineering.

  • Experience with eval tooling (e.g., Braintrust, LangSmith, Weave, Arize, or in-house frameworks).

  • Experience with model routing, AI gateways, or inference optimization.

 

#LI-REMOTE

#LI-AK2
#LI-CC1

Equal Opportunity Employer

ClickUp is an Equal Opportunity Employer, and qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.

Privacy Notice

ClickUp collects and processes personal data in accordance with applicable data protection laws. You can find further details by viewing our Global Candidate Privacy Notice.

If you are a Philippine Job Applicant, please also see our Philippine Data Privacy Notice for further details.

Visa Sponsorship

Please note we are unable to sponsor or take over sponsorship of an employment visa for roles outside of engineering and product at this time. Sponsorship for engineering and product roles is not guaranteed, but is instead based on the business needs for that specific role at that time. Please reach out to the recruiter with any questions.

Fraud Alert

ClickUp Talent Acquisition will only initiate contact via an @clickup.com email or through our official careers portal on clickup.com. We will never request fees, payments, or sensitive personal information. Please disregard any offers received outside these channels and report them to [email protected].

AI Processing Notice

ClickUp may use artificial intelligence and machine learning technologies to help review and screen candidates' employment applications against role-related criteria. These tools support, but do not replace, human decision‑making. If you have questions or need an accommodation in the recruitment process, please contact us at [email protected].

Similar Jobs

2 Days Ago
In-Office or Remote
180K-240K Annually
Senior level
180K-240K Annually
Senior level
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Build and operate distributed data and AI infrastructure for Airwallex’s Knowledge Platform. Responsibilities include architecting high-availability systems, optimizing state management and exactly-once processing, integrating streaming, vector indexing and agentic workflows, processing petabytes of data globally, and mentoring engineers through technical leadership.
Top Skills: Apache FlinkSparkClickhouseDatabricksGoJavaKafkaKotlinKubernetesOlap Engines
3 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
155K-260K Annually
Senior level
155K-260K Annually
Senior level
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Build agentic AI products that enable customers to create custom detections and workflows from operational camera data. Own projects from customer discovery through design, implementation, production operation, and ongoing optimization. Develop backend services, APIs, and customer-facing experiences; partner with platform, machine learning, product, and design teams. Improve AI accuracy, reliability, performance, and cost while mentoring engineers and shaping technical architecture.
Top Skills: Agentic AiAi Coding AgentsAPIsCloud ComputingComputer VisionEdge ComputingIotLarge Language Models
3 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
162K-290K Annually
Senior level
162K-290K Annually
Senior level
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Lead Samsara’s AI initiatives across the software development lifecycle, including developer agents and SRE AI. Set strategy, select experiments, make build-versus-buy decisions, control infrastructure and token costs, scale automated incident response, and build trust in AI-driven actions. Manage and grow a team of five to ten senior engineers while partnering directly with R&D leaders and developers.
Top Skills: Ai AgentsClaudeCloud InfrastructureCodexIncident Response AutomationIotModel Context Protocol (Mcp)Sre Tooling

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account