Python Systems Engineer – LLM Evaluation
Supercoderindore, madhya pradesh, in1 day ago
Job descriptionDesign complex, realistic engineering tasks to evaluate LLM reasoning, coding, debugging, and system understanding. Build Python- and Linux-based workflows, pipelines, and multi-step scenarios. Create reproducible environments using Python, Shell, and CLI tools. Develop tasks that measure code comprehension, debugging, refactoring, and optimization. Write clear technical documentation : problem statements, constraints, expected outputs, and detailed edge cases. Use LLM tools (ChatGPT, Claude, etc.) to validate tasks and analyze model performance. 5+ years of professional software development experience. Strong Python : modular code design, debugging complex programs, structured codebases. Proficiency with Linux, Shell scripting, Bash, and command-line tools. Solid technical English writing ability. Strong reasoning, analytical thinking, and problem-solving skills. Ability to design logical multi-step engineering scenarios. Experience creating benchmark datasets, online judge problems, coding tests, or technical challenges. Background with ICPC, Codeforces, Kaggle, or competitive programming. Familiarity with Docker, Git, and CI / CD pipelines. Experience with ML / AI or data-intensive engineering environments. Engineers who enjoy designing difficult problems rather than simple feature development. Developers who are strong at debugging, identifying subtle issues, and understanding complex system interactions. Engineers who work well independently and can define their own approach. Individuals interested in LLM evaluation, AI reliability, and technical task design.
Greetings and thank you for visiting our job post.
Supercoder is an AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.
- Type of work : 100% Remote
Overview :
The client is hiring Python / Linux Engineers to design complex system-level evaluation tasks for LLMs. Design advanced benchmark tasks that evaluate the capabilities of modern Large Language Models (LLMs) such as ChatGPT, Claude, and other AI systems.
This role focuses on building realistic, technically challenging engineering scenarios that test model reasoning, debugging, and problem-solving abilities.
What You Will Do
Must-Have Qualifications
Nice-to-Have Skills
Who Will Excel in This Role
Create a job alert for this search
Engineer Llm • indore, madhya pradesh, in
Related jobs
Python Developer (AI / Backend Focus) – Technical Interview Panel Expert.Python and backend system expertise).Gachibowli, Hyderabad, Telangana / 100% Onsite.
Technical Interview Panel Expert.AI-inte...Show moreLast updated: 8 hours ago
- Promoted
Python_ AIML Engineer
Atyeti IncIndore, Madhya Pradesh, India Role Overview : We are seeking a highly skilled AI Engineer to design and implement intelligent systems using agentic AI capable of autonomous decision-making and task execution.This role involv...Show moreLast updated: 9 days ago
- Promoted
- New!
Cloud Engineer- AI / ML
Intuition IT – Intuitive Technology Recruitmentindore, madhya pradesh, in Design, deploy, and manage scalable ML and GenAI workloads using AWS services, including SageMaker Studio and Bedrock.Implement and maintain infrastructure using AWS Lambda, EKS, ECS on Fargate, an...Show moreLast updated: 8 hours ago
- Promoted
- New!
Team Lead - Automation & LLM Integration
Deep Cognitionindore, madhya pradesh, in We are looking for an experienced .Large Language Model (LLM) APIs.The ideal candidate is both a .Python and LLM integrations with .
LLM API integration and workflow automation.Collaborate with cros...Show moreLast updated: 4 hours ago
- Promoted
AI / ML Lead
Tata Technologiesindore, madhya pradesh, in Prompt Engineering and Model Tuning (LoRA).Embeddings & Vector Stores (FAISS / Weaviate).RAG pipelines & Knowledge Integration (LangChain / LightRAG).
Agent Orchestration (AutoGen / CrewAI / LangGra...Show moreLast updated: 30+ days ago
- Promoted
Senior Python / AI Engineer — Core Team @ Zumlo
Zumloindore, madhya pradesh, in Zumlo is an always-on well-being companion—one place for immediate help, gentle structure, and progress you can see.We unify mind, body, emotions, and relationships through timely support, a caring...Show moreLast updated: 17 days ago
- Promoted
AI / ML Engineer
Innodata Inc.indore, madhya pradesh, in Our AI-driven platforms and expert teams empower clients in healthcare, life insurance, and other industries to identify risks, improve efficiency, and make smarter decisions.By combining proprieta...Show moreLast updated: 8 days ago
- Promoted
- New!
AI / ML Engineer
TinyChequeindore, madhya pradesh, in We are looking for Python Data Science & AI / ML Engineers who can build simple data models, handle data preprocessing, train ML models, and work with FastAPI to deploy them.Strong Python skills, han...Show moreLast updated: 8 hours ago
- Promoted
AI Evaluation - Python - LLM - MCP - CTC INR 50 L
CareerXperts Consultingindore, madhya pradesh, in We are seeking a highly skilled and motivated AI Engineer with expertise in large language models (LLMs), AI workflows, and machine learning.
This role combines deep technical knowledge in ML / AI wit...Show moreLast updated: 3 days ago
- Promoted
ML Ops
EXLindore, madhya pradesh, in Deploy, monitor, and scale ML models on.GCP (Vertex AI, GKE, Cloud Functions).GitHub Actions / Jenkins / cloud-native tools.
Containerize and orchestrate workloads with.MLflow, Feast, Prometheus / Gra...Show moreLast updated: 30+ days ago
- Promoted
Data Integration & LLM Engineer
Chargebeeindore, madhya pradesh, in We are seeking a highly motivated.This role is ideal for engineers who enjoy working at the intersection of.APIs, SaaS connectors, and ETL / ELT pipelines to ensure reliable and scalable data flows.B...Show moreLast updated: 30+ days ago
- Promoted
AI / ML Lead Engineer
Innocitoindore, madhya pradesh, in We are hiring for AI / ML Lead to work with our Product based client in Hyderabad.Bachelor's or master's degree in computer science or equivalent degree or demonstrable experience.Proven experience a...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior LLM Engineer AWS Bedrock (6+ Years)
MindBrainindore, madhya pradesh, in Job Title : LLM Engineer – AWS Bedrock.We are seeking a highly skilled.The ideal candidate will be responsible for designing, developing, and deploying LLM-driven solutions while ensuring scalabilit...Show moreLast updated: 8 hours ago
- Promoted
Engineer : Senior LLM Optimization (LLMO) / GEO Expert – Google Vertex
Proso.aiIndore, Madhya Pradesh, India About the Role : Proso AI is seeking a Senior Expert with hands-on experience in Google Vertex AI to lead short-term projects focused on LLM Optimization (LLMO) and Generative Engine Optimiz...Show moreLast updated: 7 days ago
- Promoted
- New!
Python Developer (AI / Backend Focus) – Technical Interview Panel Expert(Remote)
Tek Supportindore, madhya pradesh, inRemote
- Promoted
Sr. AI / ML Engineer
Soulaxindore, madhya pradesh, in We’re looking for a Senior AI / ML Engineer with a strong foundation in geometric reasoning, hands-on Python and SQL expertise, and a flair for creative problem-solving.
This role is ideal for someone...Show moreLast updated: 3 days ago
- Promoted
Python + Generative AI Engineer
Smart Moves Consultantsindore, madhya pradesh, in We are looking for a skilled Python + Generative AI Engineer who is passionate about building intelligent systems using LLMs and modern AI frameworks.
The ideal candidate will have strong Python dev...Show moreLast updated: 3 days ago
- Promoted
Applied AI Engineer
Strategic Talent Partnerindore, madhya pradesh, in Design and deploy advanced multi-agent pipelines for credit analysis.Optimize inference and prompt chains using frameworks like DSPy, GEPA, and LangChain.
Implement reasoning techniques (CoT, ToT, G...Show moreLast updated: 9 days ago
- Promoted
- New!
L3 Engineer- Azure, Cloud & Infrastructure
RVS iGlobal - White Label Service Providerindore, madhya pradesh, in L3 Engineer – Azure, Cloud & Infrastructure.The role involves Azure cloud management, infrastructure administration, networking, desktop support, and direct client interaction.The ideal candidate w...Show moreLast updated: 8 hours ago