WeLocalize Logo

WeLocalize

Expert Rater

Posted An Hour Ago
Be an Early Applicant
In-Office or Remote
Hiring Remotely in Georgia, USA
Junior
In-Office or Remote
Hiring Remotely in Georgia, USA
Junior
Rate and evaluate online advertisements against client guidelines for quality, relevance, and compliance. Identify ambiguities, refine rating standards, curate benchmark datasets for AI training, analyze rating trends, collaborate with quality and AI teams, and mentor new raters. The role requires strong analytical judgment, research skills, digital advertising knowledge, target-language fluency, excellent English, and experience in AI relevance rating, data labeling, linguistics, marketing, content, QA, or related fields.
The summary above was generated by AI

If you have a Candidate Login already, but have forgotten your password please use the steps to reset your password. If you have forgotten your email login, please contact [email protected] subject Workday Candidate Login

When creating your Workday account and entering personal information like name, address, please do not use ALL CAPS.

Thank you!

NOTICE:  For Privacy Policy please review here

Job Responsibilities:

Welo Data is an award-winning localization and AI data company. We run some of the world’s largest AI model relevance rating programs. This will play a crucial role in analyzing and improving AI-driven online advertising platforms.

This role will help establish benchmarks and guidelines for data used to train AI models, particularly in areas lacking defined standards. That means you will be shaping the future of web-based advertising and AI models.

MAIN DUTIES

The following is a non-exhaustive list of responsibilities and areas of ownership of an Expert Rater

  • Rate Client-provided ads following pre-defined guidelines
  • Evaluate advertisements based on specific quality, relevance, and compliance criteria provided by the client.
  • Ensure ratings are objective, consistent, and aligned with the predefined rating standards.
  • Identify any guideline ambiguities and apply critical thinking when assessing borderline cases.
  • Maintain a high level of accuracy and efficiency, meeting daily/weekly rating quotas.
  • Review and refine rating guidelines, identifying areas for improvement and recommending updates to enhance accuracy, consistency, and overall rating quality.
  • Curate high-quality datasets ("golden sets") by selecting and validating top-tier rated ads, which can serve as benchmarks for AI model training and human rater calibration.
  • Collaborate with cross-functional teams (e.g., quality assurance, data analysts, and AI training teams) to ensure rating processes align with evolving industry and client needs.
  • Analyse trends and patterns in ad ratings, providing insights that contribute to guideline optimizations and overall rating effectiveness.
  • Assist in training and mentoring new raters, ensuring they understand guidelines and best practices for accurate evaluations.

REQUIREMENTS

Experience

  • Have experience in AI model relevance, such as data labeling, linguistic annotation, relevance rating, etc.
  • Are analytical and data-driven with a proven track-record of data-based decision making  
  • Have a strong interest and understanding of digital marketing and advertising, especially as it relates to artificial intelligence  
  • Can effectively summarize complex information, questions, and solutions for various audiences Have experience researching and gathering insights, and understanding complex problems across a variety of topics  
  • Can defining, explain, and defend their points of view in group discussions 
  • Are comfortable reviewing and discussing a variety of topics – you do not need to have expertise on the topics, but willingness to research and understand to support consensus decisions is vital 
  • Are comfortable with ambiguity in guidelines, workflows, and processes, and are motivated to help establish those systems 

Requirements

  • Native level fluency in the target language
  • Excellent understanding of both written & spoken English
  • A 4 year university degree and/or or at least 2 years work experience in one or more of the following: AI-based relevance rating, marketing/advertising, linguistics, content creation, writing, tech-related quality assurance, data labeling, or linguistic annotation.

Additional Job Details:

Similar Jobs

5 Hours Ago
Remote or Hybrid
Entry level
Entry level
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Supports the analytics agenda by developing data models, applying analytical methods and algorithms, identifying trends and patterns, and translating findings into recommendations for business leaders. Collaborates with analytics leaders and external partners, uses KPIs and scorecards, and communicates the expected outcomes of analytics initiatives to stakeholders.
Top Skills: AlteryxExcelPythonRTableau
7 Hours Ago
Remote
Mid level
Mid level
Events • Food • Travel • Hospitality • Hotels & Resorts
Prepare and cook dishes to established recipes and presentation standards, supervise line cooks, maintain kitchen cleanliness, monitor ingredient quality and storage, support food-cost control, coach team members, and ensure compliance with health, safety, sanitation, and alcohol-awareness regulations.
7 Hours Ago
Remote
Entry level
Entry level
Events • Food • Travel • Hospitality • Hotels & Resorts
Supervise housekeeping staff and ensure hotel rooms and public areas meet cleanliness standards. Conduct inspections, manage supplies and equipment, resolve guest requests, train employees, coordinate with hotel departments, and maintain operational and performance reports.

What you need to know about the Chicago Tech Scene

With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.

Key Facts About Chicago Tech

  • Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
  • Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
  • Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
  • Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account