Evaluates AI-generated data science and quantitative reasoning outputs; develops prompts, datasets, reference solutions, and objective evaluation benchmarks. Identifies statistical errors, flawed methodologies, and unsupported conclusions while providing structured feedback to improve model accuracy. Responsibilities include dataset validation, statistical and quantitative assessment, quality assurance, technical documentation, and clear communication of analytical findings.
This role is for one of our clients
Pay: 150–200/hour
Role Type: Contractor
Location: Remote
We are seeking a detail-oriented Junior Data Scientist to contribute to the evaluation and improvement of advanced artificial intelligence systems. This opportunity is ideal for early-career data science professionals with strong analytical skills and a solid understanding of statistics, machine learning, experimentation, and quantitative reasoning.
In this role, you will evaluate AI-generated responses to data science and quantitative problems, develop expert-level prompts and datasets, and create reference solutions to assess the accuracy and reliability of AI models. You will identify statistical errors, flawed methodologies, and weaknesses in quantitative reasoning while providing structured feedback to improve model performance.
The ideal candidate combines practical data science knowledge with clear technical communication and a rigorous, evidence-based approach to problem-solving. Previous experience in AI development is not necessarily required.
Requirements
Key Responsibilities
- AI Model Evaluation: Evaluate AI-generated outputs on data science, statistics, machine learning, experimentation, and quantitative analysis tasks.
- Problem and Prompt Development: Create challenging, well-defined prompts and technical problems that assess analytical reasoning and practical data science skills.
- Dataset Preparation: Prepare and validate relevant datasets for testing AI models across statistical and machine learning use cases.
- Reference Solution Development: Develop accurate reference answers, calculations, and analytical solutions to establish reliable evaluation benchmarks.
- Statistical Validation: Identify statistical errors, incorrect assumptions, flawed methodologies, and inappropriate interpretations of data.
- Quantitative Reasoning Assessment: Assess whether AI-generated solutions demonstrate sound mathematical reasoning, appropriate analytical methods, and evidence-based conclusions.
- Structured Feedback: Provide clear, actionable feedback highlighting errors, missing steps, and opportunities to improve the accuracy and reliability of AI-generated outputs.
- Quality Assurance: Validate analytical results, review solution consistency, and ensure that evaluation criteria are objective and reproducible.
- Technical Documentation: Explain statistical concepts, analytical approaches, and evaluation findings in a clear and structured manner.
Required Qualifications
- At least one year of professional experience at a well-established company in technology, finance, research, or a related field.
- At least part of the relevant professional experience should have been gained within the past three years.
- Foundational knowledge of data science, statistics, machine learning, and quantitative analysis.
- Strong analytical and problem-solving skills, with the ability to evaluate data-driven conclusions critically.
- Understanding of statistical methods, experimental design, data interpretation, and model evaluation.
- Ability to identify incorrect calculations, methodological weaknesses, and unsupported conclusions.
- Strong written communication skills, particularly when explaining technical and statistical concepts.
- Ability to work independently, follow evaluation guidelines, and deliver accurate, high-quality assessments.
Must-Have Skills
- Data Science Fundamentals
- Statistics and Probability
- Machine Learning Fundamentals
- Quantitative Reasoning
- Data Analysis and Interpretation
- Statistical Methodology
- Analytical Problem-Solving
- Technical Writing and Documentation
Good-to-Have Skills
- Python Programming
- SQL and Data Manipulation
- Pandas, NumPy, and scikit-learn
- Experimental Design and Hypothesis Testing
- Predictive Modeling and Model Evaluation
- Dataset Creation and Validation
- AI Model Evaluation and Benchmarking
- Experience in Technology, Finance, or Research Organizations
Preferred Qualifications
- Experience working at a recognized technology company, financial institution, or research organization.
- Professional exposure to statistical analysis, machine learning projects, quantitative research, or data-driven decision-making.
- Familiarity with evaluating AI-generated answers or validating analytical solutions.
- Ability to communicate complex quantitative concepts in clear, concise English.
- Candidates based in English-speaking countries are preferred.
Similar Jobs
Events • Food • Travel • Hospitality • Hotels & Resorts
Prepares and presents dishes according to standardized recipes, supervises line cooks, maintains kitchen cleanliness and food safety, monitors ingredient quality and storage, supports food cost control, coaches team members, and ensures compliance with health, sanitation, safety, and alcohol-awareness regulations.
Top Skills:
Haccp
Events • Food • Travel • Hospitality • Hotels & Resorts
Prepares and cooks menu items under senior chef supervision, manages guest orders, executes recipes, oversees kitchen stations, inspects presentation and quality, maintains sanitation and food safety standards, monitors inventory, and requisitions supplies. The role requires at least one year of cooking experience, cuisine knowledge, teamwork, communication, and the ability to work effectively in a demanding kitchen environment.
Productivity • Software • App development • Automation
Manage and close enterprise software sales opportunities across India. Responsibilities include qualifying leads, managing the full sales cycle, identifying customer needs, partnering with solution engineers, presenting software solutions, researching markets and competitors, negotiating with business and technical executives, and achieving sales quotas.
Top Skills:
CRMDocument Processing SdkmacOSWindows
What you need to know about the Chicago Tech Scene
With vibrant neighborhoods, great food and more affordable housing than either coast, Chicago might be the most liveable major tech hub. It is the birthplace of modern commodities and futures trading, a national hub for logistics and commerce, and home to the American Medical Association and the American Bar Association. This diverse blend of industry influences has helped Chicago emerge as a major player in verticals like fintech, biotechnology, legal tech, e-commerce and logistics technology. It’s also a major hiring center for tech companies on both coasts.
Key Facts About Chicago Tech
- Number of Tech Workers: 245,800; 5.2% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: McDonald’s, John Deere, Boeing, Morningstar
- Key Industries: Artificial intelligence, biotechnology, fintech, software, logistics technology
- Funding Landscape: $2.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Pritzker Group Venture Capital, Arch Venture Partners, MATH Venture Partners, Jump Capital, Hyde Park Venture Partners
- Research Centers and Universities: Northwestern University, University of Chicago, University of Illinois Urbana-Champaign, Illinois Institute of Technology, Argonne National Laboratory, Fermi National Accelerator Laboratory
