Talent.com
Generative AI Application Developer
Generative AI Application DeveloperTestZeus • Bengaluru, Republic Of India, IN
Generative AI Application Developer

Generative AI Application Developer

TestZeus • Bengaluru, Republic Of India, IN
5 days ago
Job description

Location : Bangalore (Office)

Experience : 4–5 years

Role Type : Full-time

About TestZeus

TestZeus is pioneering the next generation of AI-powered software testing. We’re the team behind Hercules, the world’s first open-source testing agent. By combining large language models, multi-agent orchestration, and state-of-the-art retrieval pipelines, we deliver autonomous, zero-maintenance testing for web and API workloads. We are seeking an AI Engineer (4–5 years experience) who can design, build, and scale production-grade LLM systems. This is a hands-on product role where you’ll work closely with backend, frontend, and product teams to ship quickly, test with real users, and iterate in production. If you enjoy building RAG systems, prompt workflows, and agentic evaluation that solve real business problems, this role is for you.

Key Responsibilities

1. LLM Workflow & Prompt Engineering

  • Design, build, and maintain LLM workflows that score freeform answers, generate contextual feedback, and assist users in real time.
  • Develop and refine prompt templates, chains, and tools to improve relevance, reduce token usage, and mitigate hallucinations.
  • Implement multi-step prompt workflows for tasks like mock interviews, code reviews, and automated test guidance.

2. Retrieval & RAG Pipelines

  • Build and optimise retrieval-augmented generation (RAG) pipelines using vector stores (e.G., Pinecone, Weaviate, Elasticsearch).
  • Implement embedding generation, similarity search, and dynamic context selection that reduce hallucinations and improve answer quality.
  • Ensure low-latency, high-accuracy retrieval combined with LLM generation for highly personalised user experiences.
  • 3. LLM Evaluation & Analytics

  • Define and implement evaluation frameworks for LLM outputs : accuracy, consistency, bias, interpretability, and robustness.
  • Build automated evaluation pipelines that monitor performance over time, detect regression, and flag failure modes.
  • Instrument systems with metrics and logging to understand model behaviour in production and drive data-informed decisions.
  • 4. Agent-Based System Development

  • Build tool-augmented agents capable of evaluating coding, system design, or reasoning questions using frameworks like LangChain, AutoGen, LlamaIndex, or similar.
  • Design and experiment with agent orchestration patterns (multi-agent workflows, planners, evaluators) to improve multi-step reasoning and reliability.
  • Integrate agents with external tools and APIs (code execution, documentation search, test runners, etc.) to extend capabilities.
  • 5. Cross-Functional Collaboration & Product Impact

  • Partner with backend engineers (Go, FastAPI) and frontend engineers (React) to ship features end-to-end.
  • Work closely with product managers and designers to shape user journeys, gather feedback, and iterate quickly in production.
  • Participate in agile ceremonies (standups, sprint planning, retrospectives) and provide clear, consistent status updates.
  • 6. Research, Experimentation & Innovation

  • Stay current with state-of-the-art LLM and retrieval research, benchmarks, and open-source tools.
  • Rapidly prototype new ideas (e.G., advanced retrieval strategies, custom fine-tuning flows, new evaluation methods) and demonstrate feasibility.
  • Contribute to internal best practices, playbooks, and reusable components for LLM and agent development.
  • Qualifications & Skills Education

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent professional experience.
  • Experience

  • 4–5 years of professional software engineering experience, with a strong focus on Python.
  • At least 2 years of hands-on experience building and deploying LLM-powered applications in production (beyond toy projects).
  • Core Skills

    LLM & Prompt Engineering

  • Proven experience designing prompt workflows, tools, and chains for production use.
  • Familiarity with OpenAI, Claude, and / or open-weight LLMs (e.G., Hugging Face ecosystem).
  • Experience with LangChain, LlamaIndex, AutoGen, or equivalent frameworks for chains / agents.
  • Retrieval & RAG

  • Built at least one end-to-end RAG pipeline integrating vector search (Pinecone, Weaviate, Elasticsearch, or similar) with LLMs.
  • Understanding of embeddings, similarity search, reranking, and dynamic context selection for reducing hallucinations.
  • Evaluation Frameworks

  • Experience defining LLM output quality metrics (accuracy, consistency, bias, interpretability).
  • Implemented automated evaluation or testing pipelines to monitor LLM behaviour and aggregate performance statistics.
  • Python Engineering

  • Strong Python skills and experience building production services using frameworks like FastAPI, Flask, or similar.
  • Good understanding of data preprocessing, ETL flows, and integration patterns for AI / ML components.
  • Familiarity with testing, observability, and integration testing for AI systems.
  • Collaboration & Product Mindset

  • Demonstrated ability to work in cross-functional teams (backend, product, UX) within an Agile / Scrum environment.
  • Ability to translate research concepts and technical trade-offs to non-technical stakeholders.
  • Product mindset : you care deeply about user experience, measurable impact, and iteration speed.
  • Preferred

  • Hands-on experience with vector databases & semantic search (Pinecone, Weaviate, pgvector, Elasticsearch, etc.).
  • Domain exposure to developer tooling, QA / testing platforms, edtech, or hiring / assessment products.
  • Experience with fine-tuning LLMs or building lightweight custom models.
  • Interest in growing into a Founding AI Lead role as TestZeus scales.
  • Why Join TestZeus?

  • Real Impact : Own and shape the AI systems that power our flagship testing platform, impacting quality for thousands of users.
  • Cutting-Edge Environment : Work with SOTA LLMs, agents, and retrieval systems, and get space to prototype bold ideas.
  • High Ownership : Small, talented team where your decisions and code directly shape the product and company direction.
  • Learning & Growth : Regular tech talks, deep dives into research, and support for conferences, workshops, and open-source work.
  • Bangalore – Work From Office : Collaborate closely with the team, iterate quickly, and build a strong product culture together.
  • Application Process

    To apply, please share the following details with us :

  • Your CV
  • Your Current and Expected CTC
  • Months of experience building LLM applications and / or AI agents
  • Links to Public Work (e.G., GitHub, Medium, personal website, open-source contributions)
  • 📬 Send everything to : hiring@testzeus.com

  • We’re excited to review your application!
  • Create a job alert for this search

    Ai Application Developer • Bengaluru, Republic Of India, IN

    Related jobs
    Lead Generative AI Application Developer

    Lead Generative AI Application Developer

    Senzcraft • Bengaluru, Republic Of India, IN
    Founded by IIM Bangalore and IEST Shibpur Alumni, Senzcraft is a hyper-automation company.Senzcraft vision is to Radically Simplify Today's Work. And Design Business Process For The Future.Using int...Show more
    Last updated: 5 days ago • Promoted
    Generative AI Engineer

    Generative AI Engineer

    Recro • Bengaluru, Karnataka, India
    Hiring : AI Architect (7+ yrs) — Agentic & GenAI Systems.Full-time | AI Platforms & Architecture Team.You’ll own the design and scale of production-grade. Design production architectures on.AWS (EKS,...Show more
    Last updated: 30+ days ago • Promoted
    Generative AI Application Engineer

    Generative AI Application Engineer

    Ivanti • Bengaluru, Republic Of India, IN
    Are you an innovative AI / ML expert ready to lead the future of enterprise software?.Join Ivanti to drive the next generation of SaaS products and services, powered by cutting-edge AI.If you thrive ...Show more
    Last updated: 8 days ago • Promoted
    Generative AI Application Developer

    Generative AI Application Developer

    Recro • Bengaluru, Republic Of India, IN
    Design, build, and operate production AI agents and tool using Agentic AI frameworks in Python.Own agentic workflows end to end across planning, reasoning, tool calling, retrieval, evaluation, secu...Show more
    Last updated: 9 hours ago • Promoted • New!
    AI Application Development Engineer

    AI Application Development Engineer

    ITC Infotech • Bengaluru, Republic Of India, IN
    We’re Hiring : Gen AI Developer | Bangalore.Build APIs using Python to integrate with.Design Retrieval-Augmented Generation pipelines using. Create dynamic prompts to maximize accuracy and minimize h...Show more
    Last updated: 5 days ago • Promoted
    Applied AI Engineer

    Applied AI Engineer

    Strategic Talent Partner • Bengaluru, IN
    Design and deploy advanced multi-agent pipelines for credit analysis.Optimize inference and prompt chains using frameworks like DSPy, GEPA, and LangChain. Implement reasoning techniques (CoT, ToT, G...Show more
    Last updated: 23 days ago • Promoted
    Agentic AI Application Developer

    Agentic AI Application Developer

    KPI Partners • Bengaluru, Republic Of India, IN
    KPI Partners is seeking a talented and motivated.In this role, you will be responsible for designing, developing, and implementing advanced AI-driven solutions based on Google technologies.You will...Show more
    Last updated: 5 days ago • Promoted
    Generative AI Application Developer

    Generative AI Application Developer

    JobTwine • Bengaluru, Republic Of India, IN
    Reach out to : careers@jobtwine.Location : Bengaluru - 3 Days working from Office.Meet JayT : Your 24 / 7 AI Hiring Agent that powers Human Decisions for fastest and best hiring outcomes.It brings stat...Show more
    Last updated: 2 days ago • Promoted
    Generative AI Application Developer

    Generative AI Application Developer

    Impetus • Bengaluru, Republic Of India, IN
    High proficiency in Python and its relevant libraries.Strong hands-on experience with any cloud ecosystem.Exposure to Large Language Models (LLMs). Experience with Retrieval-Augmented Generation (RA...Show more
    Last updated: 30+ days ago • Promoted
    Generative AI Application Engineer

    Generative AI Application Engineer

    Recro • Bengaluru, Republic Of India, IN
    Hiring : AI Architect (7+ yrs) — Agentic & GenAI Systems.Full-time | AI Platforms & Architecture Team.You’ll own the design and scale of production-grade. Design production architectures on.AWS (EKS,...Show more
    Last updated: 16 days ago • Promoted
    Generative AI Engineer

    Generative AI Engineer

    Reyika • Bengaluru, Karnataka, India
    Job Description : Technical Lead – App Development (Generative AI).Location- Noida / Pune / Bangaluru / Hyderabad.We are seeking a talented and innovative. Technical Lead – App Development.In this role, yo...Show more
    Last updated: 10 days ago • Promoted
    Generative AI Application Developer

    Generative AI Application Developer

    Rooman Technologies • Bengaluru, Republic Of India, IN
    Are you passionate about building innovative AI products from the ground up?.AI venture and are looking for.Computer Science, Electronics & Communication (E&C), or Electrical & Electronics (E&E).Pa...Show more
    Last updated: 30+ days ago • Promoted
    Generative AI Application Developer

    Generative AI Application Developer

    Nbyula - Skillizens without Borders • Bengaluru, Republic Of India, IN
    We expect you to be hands-on working with various.This role is application-focused, requiring the ability to test, fine-tune, and deploy AI agents tailored for specific tasks.The ideal candidate sh...Show more
    Last updated: 23 hours ago • Promoted
    Generative AI Architect

    Generative AI Architect

    Leading Healthcare Industry • Bengaluru, Karnataka, India
    We are seeking a highly skilled and experienced.This role goes beyond building AI models—it involves architecting robust, scalable, and secure AI-powered applications and services across the full t...Show more
    Last updated: 18 days ago • Promoted
    Gen AI & Agentic AI Developer

    Gen AI & Agentic AI Developer

    Recmatrix Consulting • Bengaluru, Karnataka, India
    If you're passionate about developing innovative AI solutions, we'd love to have you on board.Develop and Optimize AI Models. AI models in production environments.Retrieval-Augmented Generation (RAG...Show more
    Last updated: 21 days ago • Promoted
    Generative AI Application Developer

    Generative AI Application Developer

    Capgemini • Bengaluru, Republic Of India, IN
    HANDS ON EXPERIENCE ON GENAI, MACHINE LEARNING, NLP, LLM, PYTHO.Show more
    Last updated: 8 days ago • Promoted
    AI Application Developer

    AI Application Developer

    HGS • Bengaluru, Republic Of India, IN
    Want to join a startup, but with the stability of a larger organization? Join our innovation team at HGS that's focused on building SaaS products. If you are highly driven & passionate person who'd ...Show more
    Last updated: 30+ days ago • Promoted
    Generative AI Application Engineer

    Generative AI Application Engineer

    Chubb • Bengaluru, Republic Of India, IN
    Chubb is a world leader in insurance.With operations in 54 countries and territories, Chubb provides commercial and personal property and casualty insurance, personal accident and supplemental heal...Show more
    Last updated: 2 days ago • Promoted