Talent.com
No longer accepting applications
Prompt Engineer

Prompt Engineer

Webknot Technologieskolkata, west bengal, in
10 hours ago
Job description

Experience : 1–4 Years

Location : Bangalore (Onsite)

About the Role :

We are seeking an Agentic Automation QA Engineer with 1–4 years of experience to test, validate, and enhance Generative AI (GenAI) and Agentic automation workflows. The ideal candidate should possess a strong understanding of software testing principles, prompt evaluation, and LLM-based application behavior.

Candidates with Java programming experience will be given preference.

Key Responsibilities :

  • Design, develop, and execute test cases for Agentic workflows and AI-driven automation processes.
  • Validate AI agent behaviors, responses, and decision-making logic to ensure correctness, stability, and reliability.
  • Evaluate and benchmark LLM outputs for accuracy, coherence, consistency, and alignment with defined objectives.
  • Collaborate with developers, data scientists, and prompt engineers to identify and resolve issues across automation pipelines.
  • Develop automated test frameworks or scripts for repeatable validation of GenAI features and Agentic components.
  • Maintain detailed documentation of test plans, scenarios, and results for continuous improvement.
  • Stay updated on LLM testing methodologies, Agentic frameworks, and emerging trends in AI automation QA.

Required Skills & Qualifications :

  • 1–4 years of experience in QA, automation testing, AI validation, or NLP-based systems.
  • Hands-on experience with LLM testing or evaluation (e.g., GPT, Claude, Gemini, or similar models).
  • Strong understanding of software QA methodologies, testing types, and debugging processes.
  • Proficiency in Java or another programming language for automation and testing workflows.
  • Knowledge of GenAI concepts such as context handling, temperature tuning, and prompt behavior analysis.
  • Excellent analytical, problem-solving, and documentation skills.
  • Strong communication and collaboration skills for cross-functional teamwork.

    • Good to Have

  • Experience with Agentic AI frameworks, LLMOps, or LLM evaluation tools such as DeepEval or LangChain.
  • Familiarity with REST APIs, Postman, or cloud-based AI platforms (OpenAI, Anthropic, Vertex AI, etc.).
  • Exposure to CI / CD pipelines, automated QA tools, or AI model monitoring systems.
    • Create a job alert for this search

    Engineer • kolkata, west bengal, in