Python Systems Engineer – Llm EvaluationSupercoder • Visakhapatnam, Republic Of India, IN
No longer accepting applications
Python Systems Engineer – Llm Evaluation
Supercoder • Visakhapatnam, Republic Of India, IN
12 days ago
Job descriptionDesign complex, realistic engineering tasks to evaluate LLM reasoning, coding, debugging, and system understanding. Build Python- and Linux-based workflows, pipelines, and multi-step scenarios. Create reproducible environments using Python, Shell, and CLI tools. Develop tasks that measure code comprehension, debugging, refactoring, and optimization. Write clear technical documentation : problem statements, constraints, expected outputs, and detailed edge cases. Use LLM tools (ChatGPT, Claude, etc.) to validate tasks and analyze model performance. 5+ years of professional software development experience. Strong Python : modular code design, debugging complex programs, structured codebases. Proficiency with Linux, Shell scripting, Bash, and command-line tools. Solid technical English writing ability. Strong reasoning, analytical thinking, and problem-solving skills. Ability to design logical multi-step engineering scenarios. Experience creating benchmark datasets, online judge problems, coding tests, or technical challenges. Background with ICPC, Codeforces, Kaggle, or competitive programming. Familiarity with Docker, Git, and CI / CD pipelines. Experience with ML / AI or data-intensive engineering environments. Engineers who enjoy designing difficult problems rather than simple feature development. Developers who are strong at debugging, identifying subtle issues, and understanding complex system interactions. Engineers who work well independently and can define their own approach. Individuals interested in LLM evaluation, AI reliability, and technical task design.
Greetings and thank you for visiting our job post.
Supercoder is an AI-powered career development platform connecting developers worldwide to remote job opportunities with competitive payment.
- Type of work : 100% Remote
Overview :
The client is hiring Python / Linux Engineers to design complex system-level evaluation tasks for LLMs. Design advanced benchmark tasks that evaluate the capabilities of modern Large Language Models (LLMs) such as ChatGPT, Claude, and other AI systems.
This role focuses on building realistic, technically challenging engineering scenarios that test model reasoning, debugging, and problem-solving abilities.
What You Will Do
Must-Have Qualifications
Nice-to-Have Skills
Who Will Excel in This Role
Create a job alert for this search
Engineer Llm • Visakhapatnam, Republic Of India, IN
Related jobs
AI, we’re building the first AI + Data Fabric for the multifamily industry, transforming how clients manage, secure, and scale their marketing and operational data.
As the industry moves toward a co...Show more
Job Title : Openstack Support Engineer with OpenStack, Linux, Kubernetes, Terraform.Location : 100% REMOTE - Offshore India.
Employment Type : Long Term Contract.OpenStack, Linux systems, Kubernetes, T...Show more
Job Title : Openstack Support Engineer with OpenStack, Linux, Kubernetes, Terraform Location : 100% REMOTE - Offshore India Employment Type : Long Term Contract Job Overview : We are looking for a ski...Show more
Prompt Engineer
Innodata Inc. • Visakhapatnam, IN
Demonstrated experience programmatically using LLMs to automate data labeling, classification, localization and annotation tasks.
Strong expertise in Python for NLU, for data processing & transforma...Show more
Last updated: 21 days ago • Promoted
Software Engineer
Tilda Research • visakhapatnam, andhra pradesh, in
A passion for building scalable AI agents.Build scalable back-end services using.Optimize graph database queries and models in Neo4j.
Collaborate cross-functionally with Product, Engineering, and Cl...Show more
Last updated: 3 days ago • Promoted
Senior AI / ML Engineer
Luxoft • Visakhapatnam, IN
Our client, one of the leading Agriculture Companies, is modernising their landscape and adopting AI and innovations in their process.
We are seeking a highly skilled and innovative Python / AI Engine...Show more
Last updated: 18 days ago • Promoted
Platform Engineer – Python / Databricks / Notebooks / Kubernetes
Syntasa • Visakhapatnam, IN
Platform Engineer – Python / Databricks / Notebooks / Kubernetes.Syntasa is seeking a high-caliber and dedicated.Syntasa Technologies India Private Limited.
This position offers an exciting opportuni...Show more
Last updated: 30+ days ago • Promoted
Applied AI Engineer
Strategic Talent Partner • Visakhapatnam, IN
Design and deploy advanced multi-agent pipelines for credit analysis.Optimize inference and prompt chains using frameworks like DSPy, GEPA, and LangChain.
Implement reasoning techniques (CoT, ToT, G...Show more
Last updated: 21 days ago • Promoted
Senior Python Data Engineer – ETL & Pipeline Development (5 to 12 yrs)
AIMLEAP • Visakhapatnam, IN
Senior Python Data Engineer – ETL & Pipeline Development.Remote (WFH) / Bangalore / India.Tech / MCA / Computer Science / IT.
IT / Data / AI / LegalTech / Enterprise Solutions.ETL pipelines and data...Show more
Last updated: 2 days ago • Promoted
GenAI engineer
Confidential • Visakhapatnam
AI / ML and 1+ year in Generative AI.Hands-on experience with LangChain, LLM orchestration, and RAG patterns.Proficiency with search databases like Elasticsearch, OpenSearch, Vespa, or Weaviate.Exper...Show more
Last updated: 30+ days ago • Promoted
Senior Machine Learning Engineer - NLP
Observe.AI • vizianagaram, andhra pradesh, in
AI is the leading AI agent platform for customer experience.It enables enterprises to deploy AI agents that automate customer interactions, delivering natural conversations for customers with predi...Show more
Last updated: 21 days ago • Promoted
Llm Trainer
Avance Consulting • Visakhapatnam, Republic Of India, IN
Duration of contract : 1 month.Full Time Availability (8 Hrs) and 4 hours overlap with PST time zone.Any programming skills(Python OR JavaScript OR Java OR C++ OR C#).
We are developing a computer u...Show more
Last updated: 1 day ago • Promoted
Lead Engineer
Hyqoo • Visakhapatnam, IN
Design, deploy, and manage AWS cloud infrastructure, including EC2 instances, S3 buckets, VPCs, RDS databases, and Lambda functions.
Assist in the design, implementation, and maintenance of backup, ...Show more
Last updated: 16 days ago • Promoted
Data Engineer
Aceolution • Visakhapatnam, IN
Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show more
Last updated: 30+ days ago • Promoted
ML Ops
EXL • Visakhapatnam, IN
Deploy, monitor, and scale ML models on.GCP (Vertex AI, GKE, Cloud Functions).GitHub Actions / Jenkins / cloud-native tools.
Containerize and orchestrate workloads with.MLflow, Feast, Prometheus / Gra...Show more
Last updated: 30+ days ago • Promoted
Senior Site Reliability Engineer (C / Python)
Entech • Vizianagaram, Republic Of India, IN
Senior Software Site Reliability Engineer (C# / Python).You’ll ensure enterprise systems are reliable, scalable, and performant - driving improvements, leading SRE initiatives, and mentoring teams on...Show more
Last updated: 6 days ago • Promoted
Senior DevOps & Database Reliability Engineer – 100% Remote
Hyly.AI • Visakhapatnam, IN
Remote
Last updated: 13 days ago • Promoted
L2 Cloud Support Engineer with Openstack, Kubernetes, Linux, Python, Ansible / Terraform - 100% REMOTE - Contract Role (US Work Hours)
iShift • Visakhapatnam, IN
Remote
Last updated: 2 days ago • Promoted
MLOps Engineer
Capgemini • vizianagaram, andhra pradesh, in
Experience in developing MLOps framework cutting ML lifecycle : model development, training, evaluation, deployment, monitoring including Model Governance.
Expert in Azure Databricks, Azure ML, Unity...Show more
Last updated: 19 days ago • Promoted
L2 Cloud Support Engineer with Openstack, Kubernetes, Linux, Python, Ansible / Terraform - 100% REMOTE
iShift • Vizag, Andhra Pradesh, India
Remote
Last updated: 1 day ago • Promoted
Senior Site Reliability Engineer (C# / Python)
Entech • Visakhapatnam, IN
Senior Software Site Reliability Engineer (C# / Python).You’ll ensure enterprise systems are reliable, scalable, and performant - driving improvements, leading SRE initiatives, and mentoring teams on...Show more
Last updated: 6 days ago • Promoted