Python Systems Engineer – LLM EvaluationSupercoder • Nagpur, IN
No longer accepting applications
Python Systems Engineer – LLM Evaluation
Supercoder • Nagpur, IN
3 days ago
Job descriptionDesign complex, realistic engineering tasks to evaluate LLM reasoning, coding, debugging, and system understanding. Build Python- and Linux-based workflows, pipelines, and multi-step scenarios. Create reproducible environments using Python, Shell, and CLI tools. Develop tasks that measure code comprehension, debugging, refactoring, and optimization. Write clear technical documentation : problem statements, constraints, expected outputs, and detailed edge cases. Use LLM tools (ChatGPT, Claude, etc.) to validate tasks and analyze model performance. 5+ years of professional software development experience. Strong Python : modular code design, debugging complex programs, structured codebases. Proficiency with Linux, Shell scripting, Bash, and command-line tools. Solid technical English writing ability. Strong reasoning, analytical thinking, and problem-solving skills. Ability to design logical multi-step engineering scenarios. Experience creating benchmark datasets, online judge problems, coding tests, or technical challenges. Background with ICPC, Codeforces, Kaggle, or competitive programming. Familiarity with Docker, Git, and CI / CD pipelines. Experience with ML / AI or data-intensive engineering environments. Engineers who enjoy designing difficult problems rather than simple feature development. Developers who are strong at debugging, identifying subtle issues, and understanding complex system interactions. Engineers who work well independently and can define their own approach. Individuals interested in LLM evaluation, AI reliability, and technical task design.
Greetings and thank you for visiting our job post.
Supercoder is an AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.
- Type of work : 100% Remote
Overview :
The client is hiring Python / Linux Engineers to design complex system-level evaluation tasks for LLMs. Design advanced benchmark tasks that evaluate the capabilities of modern Large Language Models (LLMs) such as ChatGPT, Claude, and other AI systems.
This role focuses on building realistic, technically challenging engineering scenarios that test model reasoning, debugging, and problem-solving abilities.
What You Will Do
Must-Have Qualifications
Nice-to-Have Skills
Who Will Excel in This Role
Create a job alert for this search
Engineer Llm • Nagpur, IN
Related jobs
LLM Operations Engineer
Strategic Talent Partner • Republic Of India, IN
Design and deploy advanced multi-agent pipelines for credit analysis.Optimize inference and prompt chains using frameworks like DSPy, GEPA, and LangChain.
Implement reasoning techniques (CoT, ToT, G...Show more
Last updated: 12 days ago • Promoted
PLM & Systems Modeling Engineer
L&T Technology Services • Republic Of India, IN
MBSE - 3DEXPERIENCE Developers.Location : Bangalore / Mysore / Pune / Chennai / HYD / Mumbai.Should be well verse in MBSE, RFLP, CATIAMagic & CATIA NoMagic, 3DEXPERIENCE and ENOVIA.
Good in use of Systems Mode...Show more
Last updated: 7 days ago • Promoted
Genai Llm Engineer
The Judge Group • Chennai, Republic Of India, IN
Position Title : Gen AI LLM Engineer.Position Overview We are seeking a solution-oriented engineer who can identify where AI / LLM capabilities can transform business operations and architect practica...Show more
Last updated: 12 days ago • Promoted
Applied AI Engineer
Arcitech • Republic Of India, IN
Develop, train, and optimize ML models using PyTorch, TensorFlow, and Keras.Build end-to-end LLM and RAG pipelines using LangChain and LangGraph.
Work with LLM APIs (OpenAI, Anthropic Claude, Azure ...Show more
Last updated: 12 days ago • Promoted
LLM Solutions Architect
Vsynergize AI • Pune, Republic Of India, IN
We are looking for a talented and driven AI Engineer with a strong foundation in Python,.Natural Language Processing, and experience working with Large Language Models (LLMs).This is an exciting op...Show more
Last updated: 30+ days ago • Promoted
Machine Learning Pipeline Specialist
Arcitech • Republic Of India, IN
Develop, train, and optimize ML models using PyTorch, TensorFlow, and Keras.Build end-to-end LLM and RAG pipelines using LangChain and LangGraph.
Work with LLM APIs (OpenAI, Anthropic Claude, Azure ...Show more
Last updated: 12 days ago • Promoted
AI Evaluation - Python - LLM - MCP - CTC INR 50 L
CareerXperts Consulting • nagpur, India
We are seeking a highly skilled and motivated AI Engineer with expertise in large language models (LLMs), AI workflows, and machine learning.
This role combines deep technical knowledge in ML / AI wit...Show more
Last updated: 5 days ago • Promoted
Lead ML Systems Engineer
Ocrolus • Republic Of India, IN
Come build at the intersection of AI and fintech.At Ocrolus, we’re on a mission to help lenders automate workflows with confidence—streamlining how financial institutions evaluate borrowers and ena...Show more
Last updated: 12 days ago • Promoted
AI / ML Solutions Engineer
Arcitech • Republic Of India, IN
Develop, train, and optimize ML models using PyTorch, TensorFlow, and Keras.Build end-to-end LLM and RAG pipelines using LangChain and LangGraph.
Work with LLM APIs (OpenAI, Anthropic Claude, Azure ...Show more
Last updated: 12 days ago • Promoted
LLM Engineer
Smart Moves Consultants • Pune, Republic Of India, IN
We are looking for a skilled Python + Generative AI Engineer who is passionate about building intelligent systems using LLMs and modern AI frameworks.
The ideal candidate will have strong Python dev...Show more
Last updated: 5 days ago • Promoted
Ai Evaluation - Python - Llm - Mcp - Ctc Inr 50 L
CareerXperts Consulting • Pune, Republic Of India, IN
We are seeking a highly skilled and motivated AI Engineer with expertise in large language models (LLMs), AI workflows, and machine learning.
This role combines deep technical knowledge in ML / AI wit...Show more
Last updated: 6 days ago • Promoted
Cloud Based ML Systems Engineer
Quess IT Staffing • Pune, Republic Of India, IN
Pyspark, Air flow, Jenkins, cloud formation, Deep learning.If interested please share resume at.Show more
Last updated: 3 days ago • Promoted
LLM Engineer
Persistent Systems • Pune, Republic Of India, IN
We are hiring a GenAI Engineer who can code, build, and deploy production-ready AI solutions.The ideal candidate is hands-on with Python, LLM frameworks (LangChain, LlamaIndex), and has experience ...Show more
Last updated: 5 days ago • Promoted
Python_ AIML Engineer
Atyeti Inc • nagpur, maharashtra, in
We are seeking a highly skilled.This role involves integrating advanced machine learning models with reasoning, planning, and interaction capabilities to build scalable, intelligent applications.Th...Show more
Last updated: 12 days ago • Promoted
Python Systems Engineer – LLM Evaluation
Supercoder • nagpur, India
Greetings and thank you for visiting our job post.AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.Design advanced benchma...Show more
Last updated: 2 days ago • Promoted
Senior ML Systems Engineer
Zillwork • Chennai, Republic Of India, IN
Senior ML Engineer / Applied ML Lead.Own the ML architecture end-to-end.Handle NLP, embeddings, and ranking models.Guide and mentor junior ML engineers.
Strong Python (NumPy, Pandas, Scikit-learn).N...Show more
Last updated: 1 day ago • Promoted
Cloud-Based ML Pipeline Engineer
VSquare Systems Pvt. Ltd. • Pune, Republic Of India, IN
We are seeking a hands-on Machine Learning Engineer who will build, deploy and operationalise ML solutions using GCP and Vertex AI.
You will partner with data scientists, data engineers, product tea...Show more
Last updated: 27 days ago • Promoted
LLM Integration Engineer
Zenius • Republic Of India, IN
Seeking an AI Engineer to design, implement, and enhance AI-driven features using large language models and modern AI frameworks.
This role focuses on integrating pre-trained models with frameworks ...Show more
Last updated: 9 days ago • Promoted