Python Systems Engineer – LLM EvaluationSupercoder • Visakhapatnam, IN

Python Systems Engineer – LLM Evaluation

Supercoder • Visakhapatnam, IN

1 day ago

Job description

Greetings and thank you for visiting our job post.

Supercoder is an AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.

Type of work : 100% Remote

Overview :

The client is hiring Python / Linux Engineers to design complex system-level evaluation tasks for LLMs. Design advanced benchmark tasks that evaluate the capabilities of modern Large Language Models (LLMs) such as ChatGPT, Claude, and other AI systems.

This role focuses on building realistic, technically challenging engineering scenarios that test model reasoning, debugging, and problem-solving abilities.

What You Will Do

Design complex, realistic engineering tasks to evaluate LLM reasoning, coding, debugging, and system understanding.

Build Python- and Linux-based workflows, pipelines, and multi-step scenarios.

Create reproducible environments using Python, Shell, and CLI tools.

Develop tasks that measure code comprehension, debugging, refactoring, and optimization.

Write clear technical documentation : problem statements, constraints, expected outputs, and detailed edge cases.

Use LLM tools (ChatGPT, Claude, etc.) to validate tasks and analyze model performance.

Must-Have Qualifications

5+ years of professional software development experience.

Strong Python : modular code design, debugging complex programs, structured codebases.

Proficiency with Linux, Shell scripting, Bash, and command-line tools.

Solid technical English writing ability.

Strong reasoning, analytical thinking, and problem-solving skills.

Ability to design logical multi-step engineering scenarios.

Nice-to-Have Skills

Experience creating benchmark datasets, online judge problems, coding tests, or technical challenges.

Background with ICPC, Codeforces, Kaggle, or competitive programming.

Familiarity with Docker, Git, and CI / CD pipelines.

Experience with ML / AI or data-intensive engineering environments.

Who Will Excel in This Role

Engineers who enjoy designing difficult problems rather than simple feature development.

Developers who are strong at debugging, identifying subtle issues, and understanding complex system interactions.

Engineers who work well independently and can define their own approach.

Individuals interested in LLM evaluation, AI reliability, and technical task design.

Create a job alert for this search

Engineer Llm • Visakhapatnam, IN

Related jobs

Prompt Engineer

Innodata Inc. • Visakhapatnam, IN

Demonstrated experience programmatically using LLMs to automate data labeling, classification, localization and annotation tasks. Strong expertise in Python for NLU, for data processing & transforma...Show more

Last updated: 9 days ago • Promoted

Senior AI / ML Engineer

Luxoft • Visakhapatnam, IN

Our client, one of the leading Agriculture Companies, is modernising their landscape and adopting AI and innovations in their process. We are seeking a highly skilled and innovative Python / AI Engine...Show more

Last updated: 6 days ago • Promoted

Cloud Engineer

People Prime Worldwide • Visakhapatnam, IN

Experience : 8 years to 12 year.Design, deploy and maintain automated invoice extraction pipelines using GCP Document AI. Develop custom model training workflows for documents with non-standard forma...Show more

Last updated: 30+ days ago • Promoted

Generative AI, Python

Tata Consultancy Services • Visakhapatnam, IN

LOCATION : Chennai / Hyderabad / Kolkata.Notice Period- • • • 30 DAYS • • •.Experience in automation preferably in DevOps or workflow automation using platforms like ServiceNow. Experience with AI / GenAI to...Show more

Last updated: 1 hour ago • Promoted • New!

Generative AI Engineer

TestZeus • Visakhapatnam, IN

TestZeus is pioneering the next generation of AI-powered software testing.We’re the team behind Hercules, the world’s first open-source testing agent. By combining large language models, multi-agent...Show more

Last updated: 1 hour ago • Promoted • New!

Applied AI Engineer

Strategic Talent Partner • Visakhapatnam, IN

Design and deploy advanced multi-agent pipelines for credit analysis.Optimize inference and prompt chains using frameworks like DSPy, GEPA, and LangChain. Implement reasoning techniques (CoT, ToT, G...Show more

Last updated: 10 days ago • Promoted

AI Evaluation - Python - LLM - MCP - CTC INR 50 L

CareerXperts Consulting • visakhapatnam, andhra pradesh, in

We are seeking a highly skilled and motivated AI Engineer with expertise in large language models (LLMs), AI workflows, and machine learning. This role combines deep technical knowledge in ML / AI wit...Show more

Last updated: 4 days ago • Promoted

Artificial Intelligence Engineer

Arcitech • Visakhapatnam, IN

Develop, train, and optimize ML models using PyTorch, TensorFlow, and Keras.Build end-to-end LLM and RAG pipelines using LangChain and LangGraph. Work with LLM APIs (OpenAI, Anthropic Claude, Azure ...Show more

Last updated: 1 hour ago • Promoted • New!

Search Engineer

YourTribe • Visakhapatnam, IN

Design & implement search solutions.Architect and develop advanced search features using.OpenSearch / Elasticsearch, including custom analysers, tokenisers, and scoring algorithms.Create and maintain...Show more

Last updated: 30+ days ago • Promoted

Founding AI / ML Engineer

Stealth Startup • Visakhapatnam, IN

Mind Solutions Nexus builds AI systems that control chemical manufacturing plants.Multi-agent automation, predictive maintenance, real-time optimization. We're solving actual industrial problems, no...Show more

Last updated: 1 hour ago • Promoted • New!

Cloud Engineer- AI / ML

Intuition IT – Intuitive Technology Recruitment • Visakhapatnam, IN

Design, deploy, and manage scalable ML and GenAI workloads using AWS services, including SageMaker Studio and Bedrock.Implement and maintain infrastructure using AWS Lambda, EKS, ECS on Fargate, an...Show more

Last updated: 1 day ago • Promoted

Lead Engineer

Hyqoo • Visakhapatnam, IN

Design, deploy, and manage AWS cloud infrastructure, including EC2 instances, S3 buckets, VPCs, RDS databases, and Lambda functions. Assist in the design, implementation, and maintenance of backup, ...Show more

Last updated: 5 days ago • Promoted

Site Reliability Engineer

Flipkart • Visakhapatnam, IN

Hiring Site Reliability Engineers.The engineer will work in the Reliability and Productivity Engineering team and is responsible for building industry standard large scale platforms to be utilised ...Show more

Last updated: 1 hour ago • Promoted • New!

ML Ops

EXL • Visakhapatnam, IN

Deploy, monitor, and scale ML models on.GCP (Vertex AI, GKE, Cloud Functions).GitHub Actions / Jenkins / cloud-native tools. Containerize and orchestrate workloads with.MLflow, Feast, Prometheus / Gra...Show more

Last updated: 30+ days ago • Promoted

Senior Machine Learning Engineer

Petals Careers Private Limited • Visakhapatnam, IN

Hiring : Senior Machine Learning Engineer | Onsite – Bengaluru.Senior ML Engineer to build cutting-edge.NLP, CV, and LLM-based AI solutions. Designing & deploying ML / LLM features.Building scalable ML...Show more

Last updated: 1 hour ago • Promoted • New!

Senior LLM Engineer AWS Bedrock (6+ Years)

MindBrain • vizianagaram, andhra pradesh, in

Job Title : LLM Engineer – AWS Bedrock.We are seeking a highly skilled.The ideal candidate will be responsible for designing, developing, and deploying LLM-driven solutions while ensuring scalabilit...Show more

Last updated: 1 day ago • Promoted

SENIOR MACHINE LEARNING ENGINEER / APPLIED ML LEAD

Zillwork • Visakhapatnam, IN

Senior ML Engineer / Applied ML Lead.Own the ML architecture end-to-end.Handle NLP, embeddings, and ranking models.Guide and mentor junior ML engineers. Strong Python (NumPy, Pandas, Scikit-learn).N...Show more

Last updated: 1 hour ago • Promoted • New!

Python Systems Engineer – Llm Evaluation

Supercoder • Visakhapatnam, Republic Of India, IN

Greetings and thank you for visiting our job post.AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.Design advanced benchma...Show more

Last updated: 1 day ago • Promoted