Shop App Annotator - Fully Remote | Upto $50/hr

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Shopify Shop App Agent Annotator Type: Contract Compensation: $40–$50/hour Location: Remote Commitment: ~5 hours/week Role Responsibilities - Evaluate full shopper-agent conversations, including tool calls and their outcomes. - Identify the specific turn that received a thumbs-down from the shopper. - Classify the issue with a primary domain tag and a specific secondary tag. - Write concise comments explaining the shopper's intent, the agent's actions, and the rationale for the tags. - Work independently and asynchronously to maintain consistency in tagging across conversations. Qualifications Must-Have - Rule discipline to apply a fixed taxonomy consistently. - Comfort reading structured traces to understand agent actions. - Restraint under ambiguity to use the unknown label appropriately. - Clear, concise written English for explaining labeling decisions. Preferred - Consumer e-commerce fluency in orders, tracking, and platform responsibilities. - Prior annotation, labelling, or model-evaluation work with a defined rubric. - Experience with the Shop app or other AI shopping assistants. - Familiarity with customer-support operations and escalation paths. Application Process (Takes 20–30 mins to complete) - Upload resume - AI interview based on your resume - Submit form Resources & Support - For details about the interview process and platform information, please check: - For any help or support, reach out to: PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity. Originally posted on Himalayas