Lead Consultant LLMOps

Join Delphi - Where Innovation meets transformation

At Delphi, we believe in creating an environment where our people thrive. Our hybrid work model empowers you to choose where you work—whether it's from the office, your home, or a mix of both so you can prioritize what matters most. We are committed to supporting your personal goals, family, and overall well being while driving transformative results for our clients.

We welcome exceptional talent from anywhere across the globe. Interviews and onboarding are conducted virtually, reflecting our digital-first mindset.

Rooted in the region, we specialize in delivering tailored, impactful solutions in Data, Advanced Analytics and AI, Infrastructure, Cloud Security, and Application Modernization. Whether it’s enabling predictive analytics, transforming operations with automation, or driving customer engagement with intelligent platforms, we are the trusted partner for organizations ready to embrace a smarter, more efficient future.

About the Role: LLMOPS

Location: India Remote

Experience - 8+ years

Full time

The Azure LLMOps Engineer is responsible for operationalizing, securing, and governing Large Language Model (LLM)–based solutions on Microsoft Azure. The role ensures that LLM applications move reliably from development to production with strong controls around quality, security, cost, and compliance. This role sits at the intersection of DevOps, MLOps, and AI platform engineering, focusing on the lifecycle management of prompts, agents, models, and runtimes rather than model training. The LLMOps Engineer establishes CI/CD pipelines, evaluation frameworks, and runtime guardrails that enable development teams to safely consume Azure OpenAI and related AI services at enterprise scale. The role is critical in regulated and enterprise environments, where requirements such as signed artifacts, auditability, environment isolation, private networking, and controlled promotion are mandatory. By standardizing deployment, monitoring, and governance practices, the Azure LLMOps Engineer enables teams to innovate with LLMs while maintaining operational stability, security, and regulatory compliance.

What you will do

1. Design, build, and operate enterprise-grade LLM platforms on Microsoft Azure

2. Deploy and manage LLM runtimes using Azure OpenAI, AKS, App Service, or Azure ML

3. Implement and maintain CI/CD pipelines for LLM applications, prompts, and agents

4. Enforce digest-based artifact promotion across environments (DEV / NON-PROD / PROD)

5. Manage the lifecycle of prompts, agents, and configurations, including versioning and rollback

6. Implement LLM evaluation and quality gates (relevance, groundedness, hallucination checks)

7. Integrate RAG architectures using Azure AI Search and secure data sources

8. Implement supply-chain security controls including SBOM generation and image signing

9. Enforce runtime policy controls (signed images only, admission checks, environment isolation)

10. Secure LLM workloads using Managed Identity, Key Vault, and Private Endpoints

11. Enable observability and monitoring for prompts, responses, latency, errors, and token usage

12. Implement cost tracking and optimization for LLM consumption and inference workloads

13. Define and enforce guardrails for prompt injection, data leakage, and misuse scenarios

14. Support audit, compliance, and evidence collection for regulated environments

15. Collaborate with application, security, and cloud infrastructure teams on platform standards

16. Create and maintain runbooks, standards, and reference architectures for LLMOps

17. Support onboarding and enablement of development teams onto the LLM platform

18. Contribute to documentation, testing, and UAT processes to validate data solutions.

Skills Required

1. Azure & Cloud Platform (Must-Have)

Hands-on experience with Azure OpenAI and enterprise LLM deployments

Strong knowledge of Azure Kubernetes Service (AKS), App Service, or Azure ML for LLM

runtimes

Experience with Azure DevOps or GitHub Actions for CI/CD

Solid understanding of Azure networking and security, including:

Private Endpoints

VNet-integrated services

Network isolation across environments

Proficiency with Managed Identity and Azure Key Vault

2. LLMOps & AI Platform (Must-Have)

Experience operationalizing LLM-based applications (not just consuming APIs)

Prompt lifecycle management (versioning, validation, rollback)

Understanding of RAG architectures using Azure AI Search or equivalent

Experience implementing LLM evaluation and quality gates

Knowledge of agent-based AI systems and tool/function calling patterns

3. CI/CD & DevSecOps (Must-Have)

Strong CI/CD fundamentals with gated pipelines and approvals

Containerization using Docker

Experience with container image security:

SBOM generation

Image vulnerability scanning

Image signing and verification

Familiarity with Infrastructure as Code (Terraform or Bicep)

4. Security & Governance (Must-Have)

Understanding of secure AI deployment patterns in enterprise environments

Experience implementing:

Least-privilege access

Environment separation (DEV / NON-PROD / PROD)

Secure secret handling (no hard-coded secrets)

Awareness of AI-specific threats such as:

Prompt injection

Data leakage

Hallucination risks

Good To have Skills

1. Advanced Azure & Platform Engineering

Experience with Azure ML registries and endpoint management

Knowledge of service mesh / mTLS in AKS

Exposure to Azure Policy, Sentinel, and Defender for Cloud

Multi-region or DR patterns for AI workloads

2. Advanced LLMOps & AI Governance

Hands-on experience with: LLM observability frameworks Prompt/response telemetry and masking

Knowledge of content safety and moderation patterns

Experience with AI red-teaming or adversarial prompt testing

Familiarity with cost optimization for token-based workloads

Certifications: -

• Azure security or DevOps certifications

• AI or cloud architecture certifications

• Security-focused certifications relevant to cloud or DevSecOps.

What We Offer:

At Delphi, we are dedicated to creating an environment where you can thrive, both professionally and personally. Our competitive compensation package, performance-based incentives, and health benefits are designed to ensure you're well-supported. We believe in your continuous growth and offer company sponsored certifications, training programs, and skill-building opportunities to help you succeed. We foster a culture of inclusivity and support, with remote work options and a fully supported work-from home setup to ensure your comfort and productivity. Our positive and inclusive culture includes team activities, wellness and mental health programs to ensure you feel supported.