Talent.com
This job offer is not available in your country.
TEST AUTOMATION LEAD

TEST AUTOMATION LEAD

DailoqaNoida, UP, in
27 days ago
Job type
  • Quick Apply
Job description

Job Description

Role Overview

As a Test Automation Lead at Dailoqa, you’ll architect and implement robust testing frameworks for both software and AI / ML systems. You’ll bridge the gap between traditional QA and AI-specific validation, ensuring seamless integration of automated testing into CI / CD pipelines while addressing unique challenges like model accuracy, GenAI output validation, and ethical AI compliance.

Key Responsibilities

Test Automation Strategy & Framework Design

  • Design and implement scalable test automation frameworks for frontend (UI / UX) , backend APIs , and AI / ML model-serving endpoints using tools like Selenium, Playwright, Postman, or custom Python / Java solutions.
  • Build GenAI-specific test suites for validating prompt outputs, LLM-based chat interfaces, RAG systems, and vector search accuracy.
  • Develop performance testing strategies for AI pipelines (e.g., model inference latency, resource utilization).

Continuous Testing & CI / CD Integration

  • Establish and maintain continuous testing pipelines integrated with GitHub Actions, Jenkins, or GitLab CI / CD.
  • Implement shift-left testing by embedding automated checks into development workflows (e.g., unit tests, contract testing).

    • AI / ML Model Validation

  • Collaborate with data scientists to test AI / ML models for accuracy , fairness , stability , and bias mitigation using tools like TensorFlow Model Analysis or MLflow.
  • Validate model drift and retraining pipelines to ensure consistent performance in production.

    • Quality Metrics & Reporting

  • Define and track KPIs.
  • Test coverage (code, data, scenarios)
  • Defect leakage rate
  • Automation ROI (time saved vs. maintenance effort)
  • Model accuracy thresholds
  • Report risks and quality trends to stakeholders in sprint reviews.
  • Drive adoption of AI-specific testing tools (e.g., LangChain for LLM testing, Great Expectations for data validation).

    • Soft Skills

  • Strong problem-solving skills for balancing speed and quality in fast-paced AI development.
  • Ability to communicate technical risks to non-technical stakeholders.
  • Collaborative mindset to work with cross-functional teams (data scientists, ML engineers, DevOps).

    • Requirements

    Technical Requirements

    Must-Have

  • 5–8 years in test automation, with 2+ years validating AI / ML systems.
  • Expertise in :   Automation tools : Selenium, Playwright, Cypress, REST Assured, Locust / JMeter
  • CI / CD : Jenkins, GitHub Actions, GitLab
  • AI / ML testing : Model validation, drift detection, GenAI output evaluation
  • Languages : Python, Java, or JavaScript
  • Certifications : ISTQB Advanced, CAST, or equivalent.
  • Experience with MLOps tools : MLflow, Kubeflow, TFX
  • Familiarity with vector databases (Pinecone, Milvus) and RAG workflows.
  • Strong programming / scripting experience in JavaScript, Python, Java, or similar
  • Experience with API testing, UI testing, and automated pipelines
  • Understanding of AI / ML model testing, output evaluation, and non-deterministic behavior validation
  • Experience with testing AI chatbots, LLM responses, prompt engineering outcomes, or AI fairness / bias
  • Familiarity with MLOps pipelines and automated validation of model performance in production
  • Exposure to Agile / Scrum methodology and tools like Azure Boards

    • Requirements

    Technical Requirements Must-Have 5–8 years in test automation, with 2+ years validating AI / ML systems. Expertise in : Automation tools : Selenium, Playwright, Cypress, REST Assured, Locust / JMeter CI / CD : Jenkins, GitHub Actions, GitLab AI / ML testing : Model validation, drift detection, GenAI output evaluation Languages : Python, Java, or JavaScript Certifications : ISTQB Advanced, CAST, or equivalent. Experience with MLOps tools : MLflow, Kubeflow, TFX Familiarity with vector databases (Pinecone, Milvus) and RAG workflows. Strong programming / scripting experience in JavaScript, Python, Java, or similar Experience with API testing, UI testing, and automated pipelines Understanding of AI / ML model testing, output evaluation, and non-deterministic behavior validation Experience with testing AI chatbots, LLM responses, prompt engineering outcomes, or AI fairness / bias Familiarity with MLOps pipelines and automated validation of model performance in production Exposure to Agile / Scrum methodology and tools like Azure Boards

    Create a job alert for this search

    Automation Test Lead • Noida, UP, in