Quality Assurance Automation Lead (AI\ML)

Experience - 5-6 years

Location - Bengaluru / Hyderabad / Mangalore

AI / ML Product Testing | Automation | Test Strategy | Quality Leadership

About the Role

We are looking for a QA Lead with 5–6 years of experience in leading QA activities, designing test strategies, and ensuring the delivery of high-quality software. The ideal candidate has solid experience testing AI / ML-driven products , including LLM applications, ML pipelines, data-processing systems, and intelligent automation platforms. This role requires strong analytical thinking, hands-on testing capability, leadership skills, and a deep understanding of quality processes for AI-based systems.

You will work closely with product, data science, engineering, and DevOps teams to define and implement robust testing frameworks tailored for traditional systems as well as AI-based features.

Key Responsibilities

1. QA Strategy & Planning Lead QA efforts across the full SDLC for AI / ML and software products.

Design test strategies for :

AI / ML models and inference APIs

LLM-driven features (chatbots, RAG pipelines, automation workflows)

Microservices and backend APIs

Web / mobile applications

Define test plans, scope, timelines, and acceptance criteria.

2. AI / ML Product Testing Design and execute test scenarios for :

Model accuracy, drift, bias, hallucination testing

Prompt / response validation for LLM features

Model versioning and regression testing

RAG pipeline validation (embedding accuracy, retrieval quality)

AI-driven UX workflows

Work closely with ML engineers to understand model behaviour, datasets, and performance metrics.

3. Automation Testing Build or oversee automation frameworks for UI, API, and data validation.

Tools commonly used :

Selenium / Playwright / Cypress

Postman / Newman

PyTest / Robot Framework

JMeter / Locust for performance

Develop automated test suites for continuous integration pipelines.

4. Functional & Non-functional Testing Lead end-to-end functional testing across web / mobile platforms.

Handle performance, load, scalability, and security testing.

Ensure data validation, API correctness, and integration quality.

5. Leadership & Collaboration Lead and mentor a team of QA engineers (2–5 members).

Conduct test reviews, defect reviews, root cause analysis, and quality audits.

Work with cross-functional teams including PMs, DevOps, and ML engineers.

Contribute to sprint planning, story estimation, and release readiness.

6. Process & Quality Improvements Establish QA best practices, guidelines, and templates.

Drive continuous improvement of processes across automation, manual QA, and AI testing.

Set up dashboards and metrics for test coverage, defect trends, and release quality.

Required Skills & Experience

Core QA Skills 5–6 years of hands-on QA experience including test planning, test case design, defect management.

Strong experience in manual & automation testing .

Experience validating REST APIs , backend services, and data pipelines.

AI / ML QA Expertise Understanding of ML model lifecycle—training, testing, deployment.

Experience testing :

LLM-based features (Generative AI, chatbots, AI copilots)

Recommendation systems, predictive analytics systems

RAG or vector-search-based applications

Ability to define metrics for accuracy, bias, drift, hallucinations, and quality.

Automation Tools Hands-on with automation frameworks :

Selenium, Playwright, Cypress, PyTest, Robot , etc.API automation experience (Postman / Newman, REST Assured).

Exposure to CI / CD and automated pipeline integration.

Technical Knowledge Understanding of microservices, cloud environments, and distributed systems.

Basics of Python or Java for writing automation scripts.

Familiarity with tools like Jira, TestRail, Git, Jenkins, Docker.

Soft Skills Excellent analytical and problem-solving skills.

Strong communication, documentation, and stakeholder management.

Ability to lead testing efforts and coordinate across teams.

Nice-to-Have Experience with MLOps workflows and model monitoring.

Knowledge of cloud platforms (AWS / GCP / Azure).

Understanding of data quality validation techniques.

Exposure to security testing and compliance.