Gen AI Testing

Seeking a Lead Software Engineer Automation Testing with 6 8 years of experience in designing developing and executing automation frameworks for AI/ML-driven applications.

The ideal candidate brings hands-on experience with GenAI Testing LangSmith PyTest Agentic frameworks and strong AI/ML evaluation expertise.

• Strong programming expertise in Python.
• Hands-on expertise with LangSmith Agentic frameworks PyTest BDD/V&V frameworks.

AI / GenAI Testing

• Design and execute V&V strategies for classical ML and GenAI/ agentic systems.
• Define test plans tests cases and execute them
• Evaluate AI models for accuracy performance safety robustness hallucinations bias and correctness.
• For GenAI rely on LangSmith for LLM observability tracing monitoring and evaluation for offline and oline evaluation
• For classical ML UC ensure V&V tests are done during the whole life cycle of the model (ML monitoring)

Test Automation

• Build scalable automation frameworks using PyTest integrated with CI/CD pipelines.
• Develop automated test suites for APIs microservices and AI pipelines.
• Perform sanity integration functional regression and UAT testing across QA UAT and Prod environments.
• Implement automation strategies: Keyword-driven Data-driven BDD Hybrid frameworks.

Good understanding of Data Science related metrics (for both classical ML and GenAI UC)Experience with AI/Chatbot products and testing AI applications.

Experience with Git GitHub Copilot and CI/CD pipelines.

Knowledge of Azure/AWS cloud environments.

Familiarity with MySQL PostgreSQL Oracle SQL Server NoSQL databases.

Employment Type : Full Time
Experience: years
Vacancy: 1