LLM Systems Architect

Location : Bangalore (Office)

Experience : 4–5 years

Role Type : Full-time

About TestZeus

TestZeus is pioneering the next generation of AI-powered software testing. We’re the team behind Hercules, the world’s first open-source testing agent. By combining large language models, multi-agent orchestration, and state-of-the-art retrieval pipelines, we deliver autonomous, zero-maintenance testing for web and API workloads. We are seeking an AI Engineer (4–5 years experience) who can design, build, and scale production-grade LLM systems. This is a hands-on product role where you’ll work closely with backend, frontend, and product teams to ship quickly, test with real users, and iterate in production. If you enjoy building RAG systems, prompt workflows, and agentic evaluation that solve real business problems, this role is for you.

Key Responsibilities

1. LLM Workflow & Prompt Engineering

• Design, build, and maintain LLM workflows that score freeform answers, generate contextual feedback, and assist users in real time.
• Develop and refine prompt templates, chains, and tools to improve relevance, reduce token usage, and mitigate hallucinations.
• Implement multi-step prompt workflows for tasks like mock interviews, code reviews, and automated test guidance.

2. Retrieval & RAG Pipelines

3. LLM Evaluation & Analytics

4. Agent-Based System Development

5. Cross-Functional Collaboration & Product Impact

6. Research, Experimentation & Innovation

Qualifications & Skills Education

Experience

Core Skills

LLM & Prompt Engineering

Retrieval & RAG

Evaluation Frameworks

Python Engineering

Collaboration & Product Mindset

Preferred

Why Join TestZeus?

Application Process

To apply, please share the following details with us :

📬 Send everything to : hiring@testzeus.com