Milestone Technologies - Senior AI Engineer - RAG / Python

Job Overview :

Were hiring an Senior AI Engineer to build production-grade components for an AI-first, data-centric platform.

You will implement agentic capabilities (intent, planner, router / composer), integrate knowledge-graph reasoning alongside a strong RAG baseline, and instrument robust evaluation and observability.

The ideal candidate writes clean, reliable code, understands LLM systems and data retrieval trade-offs, and can optimize for latency, quality, and cost.

Key Responsibilities :

• Agent Implementation : Build and harden Intent, Planner, and Router / Composer agents with typed JSON I / O, retries / timeouts, and idempotency; emit call-graph traces and correlation IDs.
• Knowledge-Graph Reasoning : Generate correct graph queries (SPARQL / Gremlin / PGQL) from planner outputs; perform subgraph extraction; encode rationale and references in responses.
• RAG Baseline & Retrieval : Implement document prep, chunking / embeddings, hybrid retrieval and (where available) reranking; maintain a high-quality baseline path for side-by-side comparisons.
• Prompt / Config Tuning : Version and tune prompts, routing policies (small?large model escalation), temperature / top-p settings, and caching; document routing outcomes and cost / latency budgets.
• Evaluation Hooks : Integrate test sets and scoring (faithfulness / correctness, precision / recall, multi-hop coverage, latency); enable automated re-evaluation on any change (model / agent / prompt / data).
• Observability & Cost Controls : Instrument traces / metrics / logs (token usage, latency P50 / P95, error codes); surface cost-per-answer dashboards; implement backpressure and graceful degradation.
• Security & Guardrails : Enforce policy-as-code and entitlement checks (role / row / column), PII / PHI handling, content moderation, and HITL approval prompts for state-changing actions.
• Quality & CI / CD : Write unit / integration / contract tests; participate in PR reviews; ship via CI / CD with feature flags and environment promotion; maintain API / connector schemas and docs.

Required Skills :

Good To Have Skills :

(ref : hirist.tech)