Quality Assurance (QA) Lead MiDAS

Roles & Responsibilities :
AI Quality Strategy: Develop and own the evaluation framework for GenAI solutions focusing on Faithfulness Relevancy and Hallucination detection using LLM-as-a-judge frameworks.

• Hybrid Test Automation: Architect a dual-layered automation suite:

  • Deterministic: E2E UI (Playwright) and API testing (Pytest/Requests).

  • Probabilistic: Automated evaluation of non-deterministic LLM outputs.

• Shift-Left Integration: Embed automated quality checks directly into GitHub Workflows enabling seamless CI/CD.

Performance & Resilience: Lead JMeter-based performance testing.

Qualifications :

Educational qualification:

• Experience: 8 years in Software QA

• Problem Solving: Ability to define quality in an ambiguous non-deterministic AI landscape.

• Education: Bachelors or Masters degree in Computer Science Software Engineering or a related field.

Experience :

• 8 years in Software QA

Mandatory/requires Skills :
Automation & Tooling

• Python Mastery: Expert-level Python skills for building custom test tooling and automation scripts.

• Testing Stack: Hands-on proficiency with Pytest (API) Playwright (E2E) and JMeter (Performance).

• DevOps: Advanced experience designing and maintaining GitHub Actions/Workflows for automated test execution.

Core AI & LLM Expertise

• Learning Agility in GenAI: High capability and interest in rapidly mastering AI evaluation concepts. You should be prepared to quickly upskill in automated metrics for LLMs (such as Faithfulness Relevancy and Groundedness).

• Exposure to LLM Logic: Basic familiarity with how LLMs function (e.g. prompting context windows). You should be comfortable exploring and implementing LLM-as-a-Judge strategies where high-reasoning models help grade application-specific outputs.

• Orientation toward RAG Systems: Interest in understanding the mechanics of Retrieval-Augmented Generation (RAG). You will be responsible for defining how we validate the accuracy of data retrieved from our engineering context catalogues and vector databases.

• Data-Driven Quality Mindset: A strong desire to move beyond binary Pass/Fail results toward probabilistic quality monitoring utilizing tools like Langfuse to analyze live traces and performance trends.

Preferred Skills :

Additional Information :

Why Join MiDAS

You wont just be testing software; you will be defining the quality standards for the future of AI-First Engineering. Your work will directly impact the speed and reliability of vehicle software development globally.

Remote Work :

No

Employment Type :

Full-time

Experience: years
Vacancy: 1