English Audio Rater

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an English Audio Rater based in Canada.

As an English Audio Rater, you will evaluate recorded conversations between users and AI voice agents to help assess the quality of AI-generated interactions. You will listen closely to individual exchanges and evaluate each response across content and voice delivery. Your assessments will help measure whether AI responses are useful, relevant, appropriate, natural, and expressive. The role requires careful listening, consistent judgment, and the ability to explain ratings clearly and concisely. Working independently in a freelance environment, you will contribute to the evaluation of emerging voice-based AI technology. This is a focused opportunity for someone with strong attention to detail who enjoys structured evaluation and language-focused quality assessment.

Accountabilities:
- Listen carefully to recorded conversations between users and AI voice agents.

- Evaluate individual AI agent responses using two independent 1–5 rating scales.

- Assess the content of each response for helpfulness, relevance, and appropriateness.

- Assess the prosody of each response for naturalness, pacing, expressiveness, and overall voice delivery.

- Provide a short, specific, and clearly written justification for every rating.

- Apply evaluation criteria consistently across conversations and maintain reliable judgment throughout the project.

Requirements:

- Strong listening skills and excellent attention to detail.

- Ability to evaluate audio interactions objectively and provide consistent, well-reasoned ratings.

- Strong written communication skills, with the ability to express observations clearly and concisely.

- Fluency in Canadian English and familiarity with natural language use in the Canadian market.

- Ability to distinguish between the quality of an AI response's content and the quality of its vocal delivery.

- Reliable availability for approximately 10 hours per week.

- Comfortable working independently in a freelance and remote environment.

Benefits:

- US$26 per hour.

- Freelance contract.

- Approximately 10 hours per week.

- Fully remote position within Canada.

- Opportunity to contribute to the evaluation and improvement of AI voice technology.