Analyst

About the Role

Innodata is collaborating with a leading international conglomerate, to contract subject matter experts (SMEs) for a complex prompt data annotation project. SMEs will create complex prompts and responses, implement domain-specific evaluation criteria, and use cases, and provide evaluations and judgement of model responses. SMEs are an integral part in this impactful engagement that directly contributes to the helpfulness, reasoning, and performance of a frontier LLM model.

You will have a chance to collaborate with project managers, prompt engineers, and data scientists, to solve problems together. You will also utilize AI-driven processes to meet high standards for quality and reliability. Your work will directly contribute to improving our client's data annotation and evaluation processes, enabling them to scale more efficiently.

Key Responsibilities

• Create complex prompts in your subject domain according to detailed instructions and guidance.
• Design and implement evaluation rubrics on complex prompts of your creation, and corresponding model responses
• Provide evaluations and judgements of model responses, based on your domain knowledge
• Commit to consistent throughput and improvements over the course of the project

Technical & Required Skills

Preferred Skills and Experience

Minimum Education Requirements