Role Overview
We are seeking detail-oriented evaluators to conduct human quality evaluations for an enterprise AI customer support product on Instagram, WhatsApp, and Messenger. You wil evaluate and benchmark AI model responses against complex evaluation rubrics using provided business knowledge bases.
Key Responsibilities
● Model Evaluation: Review and score AI-generated customer interactions across Foundational, Experiential, and Operational dimensions (e.g., Action Fidelity, Faithfulness, Halucination, Compliance, Tone, and Handoff).
● Intent & Fact Verification: Benchmark both informational (R1) and transactional (R2) customer queries against authoritative business sources (FAQs, product catalogs, SOPs) within the task UI.
● Quality Assurance: Participate in dual-review processes and daily calibration audits to ensure inter-rater agreement and establish ground-truth performance targets.
● Performance Targets: Deliver precise evaluations within a fast-paced daily turnaround cycle (~45-minute average handle time per job).
Required Qualifications & Skills
● Customer Service Background: Prior experience in customer service, cal centers, retail, or handling customer communications via email, chat, or phone (highly prioritized).
● English Proficiency: Exceptional written English skils with a strong command of tone, brand voice, grammar, and nuance.
● Analytical Precision: Ability to strictly fo low multi-tier evaluation guidelines, complex logic trees, and technical rubrics without deviation.
● Tech Adaptability: Comfort using dedicated web-based tools and labeling interfaces.
Work Environment & Schedule
● Duration: Ful-Time, 2-Month Contract (~60 working days), with potential extension based on project needs.
● Shift Alignment: Fixed shift during US night hours
Disclaimer : Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for any form of payment during any stage of the recruitment, onboarding or employment process. If you believe you've been targeted by a recruitment scam, please immediately report it to
Content Evaluator – Customer Support & LLM Benchmarking • bareilly, India
Job Title: Customer Success Manager.The CSM will play a critical role in ensuring customer satisfaction, product adoption, and long-term success.This role requires a strong blend of technical exper... Show more
Competitive salary based on experience and local market benchmarks.Lead paid acquisition across Meta and TikTok, driving app installs and subscriptions for an AI-powered productivity platform with ... Show more
Power Platform (admin + delivery); proven CoE governance experience in large enterprises (10K+ users).Hands-on CoE Starter Kit implementation and operations (install, configure, troubleshoot, upgra... Show more
Expert Contributor - Language Specialist.Must be fluent in one of the following languages:.Bengali, Thai, Korean, Vietnamese or Chinese.Your expertise teaches tomorrow's AI.HALO by Arise connects s... Show more
First Choice Business Debt Solutions is dedicated to helping small businesses, particularly Construction, Trade, and Trucking companies, overcome financial hardships caused by Merchant Cash Advance... Show more
The Account Manager (AM) will be the primary owner of commercial health, revenue retention, and account expansion at TestGrid.This role bridges the gap between customer business goals and company r... Show more
We are looking for a Video Editor with 3–5 years of hands-on experience in creating high-quality content for digital platforms.You will work closely with the marketing and design teams to deliver p... Show more
About Aelum Consulting Aelum Consulting is a ServiceNow Partner helping enterprises accelerate digital transformation through workflow automation, enterprise service management, AI-driven solution... Show more
Statusbrew is a social media management platform used by teams and agencies worldwide to schedule content, manage engagement across 20+ channels, run analytics, and moderate conversations, all from... Show more
Senior Sales Development Representative | Netcore Unbxd.Australia and New Zealand (Remote) | Sales Team | Full-time.Are you creative to adapt to the changing landscape of modern selling?.Is outboun... Show more
KEPLER22 LLC 1 is a growing company focused on leveraging digital platforms to reach and engage targeted audiences.The organization values data-driven decision-making and performance marketing to s... Show more
We are a leading digital marketing agency in Singapore and an official Amazon Advertising Partner.We manage marketplace advertising across Amazon, Shopee, and Lazada for brands across Southeast Asi... Show more
KEPLER22 LLC 1 is a growing company focused on leveraging digital platforms to reach and engage targeted audiences.The organization values data-driven decision-making and performance marketing to s... Show more
MAX Security Solutions is a global risk management and intelligence company providing monitoring, analysis, and operational support services to clients worldwide across the corporate, travel, and s... Show more
