Senior LLM Engineer (Prompting & Evaluation)

Job Summary (List Format) :

• Design, test, and deploy LLM (Large Language Model) workflows to power Klarion's feedback-intelligence engine.
• Own and improve prompt engineering for state-of-the-art models (OpenAI, Anthropic, etc.) using techniques like few-shot learning, chain-of-thought, and context-tuning.
• Develop and automate evaluation strategies, including precision, recall, cost, and downstream impact, for CI and production environments.
• Build and manage a versioned prompt library with automated rollout / rollback and context-injection patterns.
• Monitor model drift and performance, ensuring robustness against context or model shifts.
• Implement context injection using retrieval-augmented generation (RAG), vector search, and context management patterns (MCP).
• Stay updated on latest LLM research and best practices, translating learnings into code that improves key metrics.
• Collaborate closely with product, platform, and dashboard engineers to rapidly ship and measure new features.
• Utilize strong Python skills to build automated evaluation pipelines, data viewers, and model-selection logic.
• Communicate technical concepts and trade-offs clearly to both engineering and product teams.
• (Bonus) Apply NLP and data science skills for text analytics, pattern detection, and causal analysis.