Senior Engineer I (Inference Services)

What You’ll Be Doing :

• Design and implement an inference platform for serving large language models optimized for the various GPU platforms they will be run on.
• Develop and shepherd complex AI and cloud engineering projects through the entire product development lifecycle (PDLC) - ideation, product definition, experimentation, prototyping, development, testing, release, and operations.
• Optimize runtime and infrastructure layers of the inference stack for best model performance.
• Build native cross platform inference support across NVIDIA and AMD GPUs for a variety of model architectures.
• Contribute to open source inference engines to make them perform better on DigitalOcean cloud.
• Build tooling and observability to monitor system health, and build auto tuning capabilities.
• Build benchmarking frameworks to test model serving performance to guide system and infrastructure tuning efforts.
• Mentor engineers on inference systems, GPU infrastructure, and distributed systems best practices.

What You'll Add to DigitalOcean :

Why You’ll Like Working for DigitalOcean

This is a hybrid role based out of Hyderabad, India.

#LI-Hybrid