GenAI Analyst

Job Description

Job Role : Multimodal GenAI Evaluation Analyst

Target Language : English

Company : iMerit

Position Type : Remote - Contractual; min. 20 hours per week

Position Overview :

iMerit seeks detail-oriented and analytically minded Multimodal GenAI Evaluation Analysts to perform highly nuanced evaluations of AI system outputs across different modalities : text, image, video, and multimodal interactions. Analysts will assess the accuracy, appropriateness, quality, clarity, and cultural alignment of model outputs against complex guidelines, ensuring that results align with project standards and real-world use cases. These evaluations will directly inform the development and fine-tuning of advanced large language models (LLMs), vision models (LVMs), and multimodal AI systems.

Role Responsibilities :

• Evaluate outputs generated by LLMs across multiple modalities (text, image captions, video descriptions, and multimodal prompts).
• Assess quality against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety.
• Identify subtle errors, hallucinations, or biases in AI responses.
• Apply domain expertise and logical reasoning to resolve ambiguous or unclear outputs.
• Provide detailed written feedback, tagging, and scoring of outputs to ensure consistency across the evaluation team.
• Escalate unclear cases and contribute to refining evaluation guidelines.
• Collaborate with Project Managers and Quality Leads to meet accuracy, reliability, and turnaround benchmarks.

Skills & Competencies :

Requirements :

While moderation of high-harm / high-risk material is not part of this role, candidates should be aware that occasional exposure to NSFW or otherwise sensitive content may occur due to imperfections in client-provided datasets. Applicants should indicate that they are comfortable working in environments where such incidental exposure is a possibility.

What We Offer :

Join iMerit to help shape the benchmarks and standards that ensure AI systems are accurate, safe, and culturally aware across text, vision, and multimodal applications. If rigorous analysis and innovation excite you, we encourage you to apply