Expert Rater – AI Model Evaluation & Code Quality

Position : Expert Rater – AI Model Evaluation & Code Quality

Email- nancy.singh@truelancer.com

Fill up this form- https : / / forms.gle / RELmAdij16e9jyFF8

We’re expanding our elite rater team and looking for highly skilled technical experts who love reviewing code, debugging tricky problems, and shaping the next generation of AI systems.

If you enjoy analyzing AI-generated content, evaluating technical accuracy, and working with cutting-edge tools — this role is calling your name.

🔍 About the Role

As an Expert Rater, you will evaluate AI model responses across programming, machine learning, and general technical reasoning. Your feedback will directly influence the accuracy, safety, and performance of AI systems used worldwide.

This role is perfect for software engineers, ML / AI practitioners, tech reviewers, or anyone who loves deep technical problem-solving.

🧠 What You’ll Do

Model Quality Assessment

• Evaluate AI-generated responses involving code, ML / AI concepts, and technical workflows
• Identify errors, inefficiencies, vulnerabilities, and deviations from best practices
• Ensure outputs meet strict quality guidelines

Code Annotation & Labeling

Annotate code snippets, algorithms, and documentation

Apply project-specific labeling frameworks accurately

Review & Feedback

Provide structured, actionable feedback to improve AI behavior

Spot correctness issues, edge cases, and quality gaps

Comparative Analysis

Compare multiple AI outputs and rank them based on correctness, clarity, and efficiency

Data Validation

Validate, clean, and correct datasets used in training and evaluation

Collaboration

Work alongside data scientists and engineers

Contribute to refining rating and annotation guidelines

✅ Required Qualifications

Technical Background

Strong foundation in software engineering, computer science, ML / AI, or related fields

Excellent attention to detail and passion for data quality

Programming Expertise

Must-have :

Python

Plus at least one of the following :

JavaScript, Rust, Node.js, TypeScript, C, C++, Shell

Bonus languages :

Go, Ruby, Swift, PHP, Kotlin, Rust (extra points), Shell

Web Technologies

APIs, web scraping

HTML / CSS / JavaScript

Web app development (Flask, Node.js, React, etc.)

ML & AI Knowledge

Machine Learning fundamentals

Deep Learning frameworks (TensorFlow, PyTorch, JAX, Keras)

NLP, CV, RL concepts

Understanding of transformer architectures, CNNs, RNNs, LSTMs

Data Science & Engineering

Pandas, NumPy, visualization libraries

SQL / NoSQL databases

Algorithms & Math

Strong understanding of algorithms (e.g., A

, MCTS, optimization, combinatorics)

Software Engineering Practices

Version control (Git / GitHub)

Clean code principles

Debugging & problem-solving

Ability to follow complex multi-step instructions

Soft Skills

Excellent written communication

Analytical mindset

Proactive, detail-oriented, and willing to challenge assumptions

⭐ Preferred Qualifications

Experience with LLMs or AI-driven code generation

Familiarity with multiple programming paradigms

Prior experience reviewing code (professional or academic)

Background in annotation, QA, or technical evaluation