Analytical Evaluator - AI Feedback

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities

- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .

- Provide structured written feedback to improve training data quality and downstream performance.

- Develop clear, precise, and well-evidenced written rationales beyond surface-level observations.

- Apply strict attention to detail and accurate application of structured evaluation guidelines.

- Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

- Bachelor's degree from a top-500 globally ranked university preferred.

- Strong analytical and written communication skills.

- Ability to work independently and follow detailed task guidelines.

- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.

- Ability to write clear, precise, and well-evidenced written rationales.

- Consistent and honest judgment, including the ability to give critical assessments.

- Strict attention to detail and accurate application of structured evaluation guidelines.

- Ability to work entirely without AI writing tools .

- Native English fluency required.

Application Process (Takes 20–30 mins to complete)

- Upload resume

- AI interview based on your resume

- Submit form

Resources & Support

- For details about the interview process and platform information, please check:

- For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Originally posted on Himalayas