Prompt Engineer

Experience : 1–4 Years

Location : Bangalore (Onsite)

About the Role :

We are seeking an Agentic Automation QA Engineer with 1–4 years of experience to test, validate, and enhance Generative AI (GenAI) and Agentic automation workflows. The ideal candidate should possess a strong understanding of software testing principles, prompt evaluation, and LLM-based application behavior.

Candidates with Java programming experience will be given preference.

Key Responsibilities :

Design, develop, and execute test cases for Agentic workflows and AI-driven automation processes.

Validate AI agent behaviors, responses, and decision-making logic to ensure correctness, stability, and reliability.

Evaluate and benchmark LLM outputs for accuracy, coherence, consistency, and alignment with defined objectives.

Collaborate with developers, data scientists, and prompt engineers to identify and resolve issues across automation pipelines.

Develop automated test frameworks or scripts for repeatable validation of GenAI features and Agentic components.

Maintain detailed documentation of test plans, scenarios, and results for continuous improvement.

Stay updated on LLM testing methodologies, Agentic frameworks, and emerging trends in AI automation QA.

Required Skills & Qualifications :

1–4 years of experience in QA, automation testing, AI validation, or NLP-based systems.

Hands-on experience with LLM testing or evaluation (e.g., GPT, Claude, Gemini, or similar models).

Strong understanding of software QA methodologies, testing types, and debugging processes.

Proficiency in Java or another programming language for automation and testing workflows.

Knowledge of GenAI concepts such as context handling, temperature tuning, and prompt behavior analysis.

Excellent analytical, problem-solving, and documentation skills.

Strong communication and collaboration skills for cross-functional teamwork.

Good to Have

Experience with Agentic AI frameworks, LLMOps, or LLM evaluation tools such as DeepEval or LangChain.

Familiarity with REST APIs, Postman, or cloud-based AI platforms (OpenAI, Anthropic, Vertex AI, etc.).

Exposure to CI / CD pipelines, automated QA tools, or AI model monitoring systems.