QA Engineer - AI / ML Model Validation

About the Role

We are looking for a QA Engineer experienced in validating traditional AI / ML models to ensure the reliability, accuracy, and performance of predictive systems. The ideal candidate will collaborate closely with data scientists, MLOps engineers, and product teams to design comprehensive testing frameworks for AI models — ensuring correctness, consistency, and reproducibility across multiple datasets and environments.

Key Responsibilities

Validate and verify machine learning models (Regression, Classification, Clustering, NLP, etc.) across different stages of development and deployment.

Develop and execute test plans , test cases , and validation strategies for AI / ML pipelines.

Ensure model outputs meet defined accuracy, precision, recall, and F1-score benchmarks.

Perform data quality checks , detect anomalies, and validate preprocessing pipelines.

Validate feature engineering steps , ensuring data leakage and bias-free inputs.

Conduct A / B testing , cross-validation , and reproducibility checks .

Collaborate with Data Science teams to define acceptance criteria for models.

Validate model retraining , version control , and drift detection processes.

Document findings and prepare detailed QA reports for model performance and compliance.

Automate regression testing and pipeline validation using CI / CD and MLOps frameworks.

Required Skills & Experience

Strong understanding of machine learning lifecycle (training, validation, inference).

Experience validating supervised and unsupervised learning models .

Hands-on skills in Python and ML libraries such as scikit-learn , TensorFlow , or PyTorch .

Familiarity with data validation frameworks (e.g., Great Expectations , Deequ ).

Experience with ML experiment tracking tools (MLflow, Weights & Biases, or Kubeflow).

Working knowledge of SQL and data visualization for validation analysis.

Understanding of MLOps , CI / CD pipelines , and model deployment tools (e.g., Docker, Jenkins, GitHub Actions ).

Familiarity with statistics and data drift analysis (Kolmogorov–Smirnov test, PSI, etc.).

Strong analytical mindset and ability to identify subtle inconsistencies in model performance.