AI / ML Solution Architect

Role : Solution Architect-AI / ML

Experience : 10+ years

Location : Hyderabad & Pune

Key Responsibilities :

Architecture & Infrastructure

Design, implement, and optimize end-to-end ML training workflows including infrastructure setup, orchestration, fine-tuning, deployment, and monitoring.

Evaluate and integrate multi-cloud and single-cloud training options across AWS and other major platforms.

Lead cluster configuration, orchestration design, environment customization, and scaling strategies.

Compare and recommend hardware options (GPUs, TPUs, accelerators) based on performance, cost, and availability.

Performance & Optimization

Conduct performance benchmarking, hardware comparisons, and cost-performance trade-off analysis.

Implement real-time monitoring and control systems with metrics collection, observability, and custom performance tracking.

Optimize cost models, budget predictability, and resource utilization.

Data & Training Pipelines

Architect and validate data pipelines with storage, persistence, and throughput optimization.

Oversee data quality validation, pre-processing, and long-term experiment tracking.

Support framework flexibility for diverse training techniques (supervised, unsupervised, fine-tuning, reinforcement learning).

Integration & Deployment

Ensure seamless deployment across multi-cloud environments with security, compliance, and regional availability considerations.

Collaborate with DevOps and MLOps teams for automation, fault tolerance, job scheduling, and orchestration testing.

Provide technical guidance on integration with existing enterprise systems.

Analysis & Recommendations

Lead result analysis, insight generation, and actionable recommendations for training performance and user experience improvements.

Present performance claims, benchmarking reports, and speculative decoding insights to stakeholders.

Technical Expertise Requirements

10+ years in architecture roles with at least 5 years in AI / ML infrastructure and large-scale training environments.

Expert in AWS cloud services (EC2, S3, EKS, SageMaker, Batch, FSx, etc.) and familiar with Azure, GCP, and hybrid / multi-cloud setups.

Strong knowledge of AI / ML training frameworks (PyTorch, TensorFlow, Hugging Face, DeepSpeed, Megatron, Ray, etc.).

Proven experience with cluster orchestration tools (Kubernetes, Slurm, Ray, SageMaker, Kubeflow).

Deep understanding of hardware architectures for AI workloads (NVIDIA, AMD, Intel Habana, TPU).

Performance & Cost Management

Demonstrated expertise in performance benchmarking, reliability testing, and training speed optimization.

Skilled in cost modeling, budget forecasting, and cost-performance balancing.

Monitoring & Observability

Experience with real-time monitoring tools (Prometheus, Grafana, CloudWatch) and custom metric instrumentation.

Familiarity with network performance testing, regional load testing, and multi-region deployment strategies.

Soft Skills

Strong problem-solving skills with an analytical mindset.

Excellent communication skills to present technical trade-offs and strategic recommendations to executives and engineering teams.

Ability to lead cross-functional teams and drive innovation in AI infrastructure.

Other Required Skills :

LLM Inference Optimization

Expert knowledge of inference optimization techniques including speculative decoding, KV cache optimization (MQA / GQA / PagedAttention), and dynamic batching.

Deep understanding of prefill vs decode phases, memory-bound vs compute-bound operations.

Experience with quantization methods (INT4 / INT8, GPTQ, AWQ) and model parallelism strategies.

Inference Frameworks

Hands-on experience with production inference engines : vLLM, TensorRT-LLM, DeepSpeed-Inference, or TGI.

Proficiency with serving frameworks : Triton Inference Server, KServe, or Ray Serve.

Familiarity with kernel optimization libraries (FlashAttention, xFormers).

Performance Engineering

Proven ability to optimize inference metrics : TTFT (first token latency), ITL (inter-token latency), and throughput.

Experience profiling and resolving GPU memory bottlenecks and OOM issues.

Knowledge of hardware-specific optimizations for modern GPU architectures (A100 / H100).

System Architecture

Design scalable inference systems meeting strict latency SLAs and throughput requirements.

Implement production patterns for request routing, load balancing, and model versioning.

Balance trade-offs between latency, throughput, cost per token, and model accuracy.