This job offer is not available in your country.

AI QA Automation Engineer

Aidewiser SoltekDelhi, IN

28 days ago

Job type

Remote

Job description

Title : AI QA Automation Engineer

Location : Remote

Job Summary :

We are seeking an AI Quality Engineer with a strong automation skillset to ensure the robustness, performance, and reliability of our AI systems and services. The ideal candidate is tech-savvy, proactive, and passionate about quality at every stepfrom initial design through deployment and ongoing monitoring. You will play a key role in building and maintaining a highly automated testing infrastructure to support fast, reliable model and pipeline delivery as the company scales.

Key Responsibilities :

Testing Expertise :

Conduct comprehensive testing across all layers, including server load, integration points, and output quality.
Apply Test Driven Development (TDD) principlesanticipate, design, and define all necessary tests before the start of feature development.
Identify what needs to be tested and proactively communicate requirements before build phases.

Automation-First Approach :

Develop, maintain, and extend a fully automated testing suite that covers unit, integration, performance, and end-to-end testing.

Emphasize automation to minimize manual intervention and maximize test coverage, reliability, and repeatability.

DevOps & CI / CD Integration :

Collaborate closely with DevOps to ensure all tests (including those for model deployment and data pipelines) are tightly integrated with modern CI / CD workflows.

Streamline rapid yet safe releases through automation and timely feedback.

Automated Testing Frameworks :

Extensive hands-on experience with frameworks such as Pytest (Python testing), Playwright (end-to-end browser testing), Postman (API testing), and Langfuse (LLM output tracking / testing).

Implement and maintain robust API contract testing to ensure reliable interactions between services.

Manual & LLM Testing :

Execute manual test cases with strong attention to detail, especially for evaluating Large Language Model (LLM) output quality.

Flag issues such as hallucinations, factual inaccuracies, or unexpected edge case responses.

Continuously update manual testing strategies to adapt to evolving model behaviors and business requirements.

Monitoring, Observability & Post-Deploy Quality :

Configure, deploy, and interpret dashboards from monitoring tools like Prometheus, Grafana, and CloudWatch.

Track model health, pipeline performance, error rates, and system anomalies after deployment.

Proactively investigate and triage quality issues uncovered in production.

Core Abilities and Technical Skills :

Deep practical knowledge in test automation, performance, and reliability engineering.

In-depth experience integrating tests into CI / CD pipelines, especially for machine learning and AI model workflows.

Hands-on proficiency in automated QA tools : Pytest, Playwright, Postman, Langfuse, and similar.

Solid foundation in manual exploratory testing, particularly for complex and evolving outputs such as those from LLMs.

Expertise in monitoring, APM, and observability tools (e.g., Prometheus, Grafana, CloudWatch).

Demonstrated strong problem-solving skillsanticipate, identify, and resolve issues early.

Strong communication skills to clearly articulate requirements, quality risks, and advocate for automation-driven quality throughout the organization.

Mindset :

Automation-First : Relentless emphasis on driving automation over manual effort.

Proactive : Anticipates issues and testing needs; does not wait to be told what to test.

Quality Advocate : Champions testing best practices and designs processes to catch bugs before production.

Curious & Continuous Learner : Seeks out new tools, stays current with testing frameworks and industry best practices.

Collaborative : Partners effectively with product, engineering, and DevOps teams to deliver high-quality models and features at scale

(ref : hirist.tech)

Create a job alert for this search

Qa Automation Engineer • Delhi, IN

Related jobs

Promoted

Generative AI Engineer

Neurones IT AsiaDelhi, IN

We are looking to onboard a GenAI / ML engineer who will drive the development of GenAI activities globally and bring expertise to leverage Large Language Models (LLMs) across languages and modalitie...Show moreLast updated: 5 days ago

Promoted

AI / ML Engineer

RecroDelhi, IN

PDFs, scanned images, invoices, etc.OCR output post-processing (JSON / CSV) using.OCR technologies, intelligent document processing, and enterprise-scale AI / ML solutions. Form Recognizer / Document Inte...Show moreLast updated: 30+ days ago

Promoted

AI-Powered QA Automation Engineer – 100% Remote

Hyly.AIDelhi, IN

Remote

AI is reimagining Quality Assurance through.AI-driven automation and agentic testing tools.AI-Powered QA Automation Engineer. QA processes by building intelligent, automated testing systems.You’ll w...Show moreLast updated: 3 days ago

Promoted

Automation Engineer(Testim / CPQ)

Gokool DigitalDelhi, IN

Quality Assurance / Engineering.You will contribute to building stable, maintainable test suites that ensure high-quality business operations across CRM and ERP systems. Salesforce CPQ and NetSuite ...Show moreLast updated: 19 days ago

Promoted

Senior AI Engineer

DarGlobalDelhi, IN

We are seeking a highly skilled.AI / ML solutions that solve complex business challenges and power next-generation products. The ideal candidate has strong expertise in machine learning, deep learning...Show moreLast updated: 3 days ago

Promoted

AI Exploration Engineer

Mitchell Martin Inc.Delhi, IN

Design and execute machine learning experiments to evaluate emerging AI technologies and frameworks.Prototype and assess end-to-end AI solutions to inform product and platform strategy.Formulate hy...Show moreLast updated: 19 days ago

Promoted

QA Automation

Mount Talent Consulting Pvt Ltd.Delhi, IN

We are looking for QA Automation Engineer for one of the leading organization on the Payroll of Mount Talent Consulting.Please find below the Job Description : . Agile environment (Scrum, Kanban).Type...Show moreLast updated: 30+ days ago

Promoted

AI / ML Engineer

VeltrisDelhi, IN

Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-gene...Show moreLast updated: 17 days ago

Promoted

Quality Assurance Automation Engineer

TestGrid.ioDelhi, IN

Quality Assurance Automation Engineer.TestGrid is a leading provider of end-to-end automation cloud and on-premise testing solutions. The company focuses on simplifying the testing process with inno...Show moreLast updated: 16 days ago

Promoted

AI Engineer

MightyBotGhaziabad, IN

Join our team as an AI Engineer, where we're focused on graduating AI from interesting demos to indispensable products.You will build reliable, self-improving systems that empower subject matter ex...Show moreLast updated: 30+ days ago

Promoted

Senior AI Engineer

Milestone Technologies, Inc.Delhi, IN

AI-first, data-centric platform.You will implement agentic capabilities (intent, planner, router / composer), integrate knowledge-graph reasoning alongside a strong RAG baseline, and instrument robus...Show moreLast updated: 6 days ago

Promoted

AI Tools & Automation Expert

Office Beacon LLCDelhi, IN

Any Location / Remote (EST / PST Timezone).AI-driven tools across the company.This role involves working closely with multiple departments to identify opportunities for automation, deploying AI sol...Show moreLast updated: 25 days ago

Promoted

Quality Assurance Automation Engineer

Neurones IT AsiaDelhi, IN

We are looking to onboard three HCs for the Quality Engineer role, which is a remote position.Ensuring that software bugs are detected early in the Software Development Lifecycle, liaising with the...Show moreLast updated: 30+ days ago

Promoted

Quality Assurance Automation Engineer

BrightEdgeDelhi, IN

BrightEdge is a global leader in enterprise SEO and content performance solutions, driving AI-powered digital marketing success for the world’s top brands. Our culture is product-first, innovation-d...Show moreLast updated: 25 days ago

Promoted

Forward Deployed Engineer, Conversational AI

OmniDimensionDelhi, IN

OmniDimension is on an exhilarating journey.We're building an AI-powered platform empowering developers and enterprises across 50+ countries to effortlessly create and deploy reliable Voice AI agen...Show moreLast updated: 3 days ago

Promoted

Gen AI Engineer

SkillveraDelhi, IN

Deep expertise in LangChain, LangGraph, Langflow.Able to write UI in AngularJS / React and UI / UX practices.Hands-on experience with LLM interactions, prompt tuning, OpenAI / AWS Bedrock.C# or Java for ...Show moreLast updated: 3 days ago

Promoted

Agentic AI Engineer

InterVision SystemsDelhi, IN

We are looking for an innovative.You’ll be working at the forefront of AI, where you will architect and implement intelligent agents that can autonomously perform complex tasks, make decisions, and...Show moreLast updated: 6 days ago

Promoted

AI Engineer – Prompt Engineering & Backend Systems

Particle ExecutionDelhi, IN

We’re seeking a Senior AI Engineer who combines advanced prompt engineering expertise with solid backend development and deployment skills. You’ll work end-to-end—from crafting sophisticated AI serv...Show moreLast updated: 3 days ago

Promoted

Generative AI Engineer

GigarevDelhi, IN

At Gigarev, we harness the power of human ingenuity and advanced technology to drive business transformation and growth.Our innovative AI solutions combined with human creativity unlock new opportu...Show moreLast updated: 30+ days ago

Promoted

Lead QA Test Automation Engineer SDET

Omni ReachDelhi, IN

We are looking for 8+ years experience Lead QA Automation Engineers (Engineering background mandatory.Ability to interact with Customer. This is a full-time remote role for a Lead QA Test Automatio...Show moreLast updated: 13 days ago