Talent.com
Senior Data Engineer

Senior Data Engineer

Incept Labspune, India
13 days ago
Job description

Software Engineer (Data)

Remote, India

About us

Incept Labs is an AI research lab based in San Francisco, California. Our small team of scientists, engineers, and builders is passionate about developing domain-specific, next-generation AI solutions to enhance education and research. Backed by marquee investors and scientific collaborators, we believe in a future where humans and AI work together side by side. Together, AI processes knowledge at scale while humans contribute imagination, values, and lived experience, creating new ways to discover, adapt, and grow.

About This Role

We're looking for a Software Engineer with deep expertise in large-scale data processing for LLM development. Data engineering is critical to successful model training and evaluation. You'll work directly with researchers to accelerate experiments, develop new datasets, improve infrastructure efficiency, and enable key insights across our data assets.

You’ll join a high-impact, compact team responsible for both architecture and scaling of Incept’s data and model development infrastructure, and work with highly complex, multi-modal data.

Responsibilities

  • Design, build, and operate scalable, fault-tolerant data infrastructure to support distributed computing and data orchestration for LLM Research
  • Develop and maintain high-throughput systems for data ingestion, processing, and transformation to support LLM model development
  • Develop synthetic datasets using state-of-the-art solutions
  • Collaborate with research teams to deliver critical data assets for model development and evaluation
  • Implement and maintain monitoring and alerting to support platform reliability and performance
  • Build systems for traceability, reproducibility, and robust quality control to ensure adherence to industry compliance standards

Required Qualifications

  • 5+ years of experience in data infrastructure, ideally supporting high-scale applications or research platforms.
  • Fluent in distributed computing frameworks.
  • Deeply familiar with cloud infrastructure, data storage architectures, and batch + streaming pipelines.
  • Experience with specialized hardware (GPUs, TPUs) computing and GPU cluster.
  • Strong knowledge of databases, storage systems, and how architecture choices impact performance at scale.
  • Familiar with microservices architectures, containerization and orchestration, and both synchronous and asynchronous processing.
  • Extensive experience with performance optimization and memory management in high-volume data systems.
  • Proactive about documentation, automation, testing, and empowering your teammates with good tooling.
  • This role is fully remote, India-based. Compensation and benefits will vary based on background, skills, and experience levels.

    Create a job alert for this search

    Senior Data Engineer • pune, India

    Related jobs
    • Promoted
    Big Data Engineer

    Big Data Engineer

    CoforgePune, Maharashtra, India
    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills : - AWS, Spark & Scala.Experience Required : 3 to 6 Years. Job Locations : Pune, Hyderabad, Greater Noida Only.For queri...Show moreLast updated: 18 days ago
    • Promoted
    Data Architect

    Data Architect

    Persistent SystemsPune, Maharashtra, India
    We are seeking a highly skilled and experienced Azure Databricks Architect to lead the design, development, and implementation of scalable data solutions using Azure Databricks.This role involves w...Show moreLast updated: 16 days ago
    • Promoted
    Big Data Engineer

    Big Data Engineer

    Information TechnologyPune
    Big Data Engineer Key Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets.C...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Big Data Engineer - PySpark / Scala

    Senior Big Data Engineer - PySpark / Scala

    Consulting PanditsPune
    Exciting Job Opportunity in Pune! Are you an experienced Senior Big Data Engineer with over 5 years of expertise? We have a fantastic opportunity waiting for you!<...Show moreLast updated: 30+ days ago
    • Promoted
    GCP Data Architect

    GCP Data Architect

    Tata Consultancy Servicespune, India
    Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and consumption.Develop Data Pipelines...Show moreLast updated: 13 days ago
    • Promoted
    Accion Labs - Senior Data Architect - Generative AI

    Accion Labs - Senior Data Architect - Generative AI

    ACCION TECHNOLOGIES PRIVATE LIMITEDPune
    Job Title : Senior Data Architect - GenAI, Snowflake & Azure Experience : 10+ Years Job Type : Full-time based &l...Show moreLast updated: 25 days ago
    • Promoted
    Data Architect - Google Cloud Platform

    Data Architect - Google Cloud Platform

    HyrEzy Talent SolutionsPune
    Position : GCP Data : 8-15 Years Budget : 20-35 LPA Show moreLast updated: 30+ days ago
    • Promoted
    Big Data Developer (Hyderabad, Pune, Chennai, Bangalore, Noida)

    Big Data Developer (Hyderabad, Pune, Chennai, Bangalore, Noida)

    Tata Consultancy ServicesPune, Maharashtra, India
    Relevant Experience (in Yrs) : .Hyderabad, Pune, Chennai, Bangalore, Noida.Technical / Functional Skills Must Have Experience in : . Hands on experience in Spark, Scala and Pyspark.Should be proficient i...Show moreLast updated: 30+ days ago
    • Promoted
    Big Data Engineer

    Big Data Engineer

    WorkiFicientPune
    To be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and...Show moreLast updated: 30+ days ago
    • Promoted
    Reveille Technologies - Big Data Engineer

    Reveille Technologies - Big Data Engineer

    Reveille TechnologiesPune
    Description : We are looking for highly skilled and motivated Big Data Engineers / Leads to join our team and drive the development of our next-generation data plat...Show moreLast updated: 27 days ago
    • Promoted
    Principal / Senior Data Architect

    Principal / Senior Data Architect

    AaysPune, IN
    Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 30+ days ago
    • Promoted
    Big Data Architect - Spark / Hadoop

    Big Data Architect - Spark / Hadoop

    Consulting PanditsPune
    Big Data Architect - Build the Future of Data Management Are you passionate about designing and implementing scalable and resilient data architectures? Do you have a ...Show moreLast updated: 30+ days ago
    • Promoted
    SAP Master data Specialist

    SAP Master data Specialist

    IT Services & ConsultingPune, Maharashtra, India
    SAP Expert / Master Data Specialist.SAP S / 4HANA Cloud MDG (Master Data Governance).Master Data across complex, global environments. The ideal candidate will have successfully delivered.Lead and supp...Show moreLast updated: 19 days ago
    • Promoted
    Softpath Technologies - Senior Big Data Engineer - Hive & Unix Scripting

    Softpath Technologies - Senior Big Data Engineer - Hive & Unix Scripting

    Softpath Technologies LLCPune
    Description : Job Title : Senior Big Data Engineer Hive & Unix Scripting Location : P...Show moreLast updated: 3 days ago
    • Promoted
    Big Data Engineer - Spark / Scala

    Big Data Engineer - Spark / Scala

    Leinex ConsultingPune
    Role & responsibilities : - Evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders < / p&...Show moreLast updated: 30+ days ago
    • Promoted
    Azure Data Architect

    Azure Data Architect

    Programmers.ioPune, Maharashtra, India
    Job Title : Azure Data Architect.Location : Hyderabad, pune (hybrid).Experience required : 12+ years.We are seeking a highly experienced S. The ideal candidate should bring strong expertise in SQL, ETL...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Big Data Engineer - Spark

    Lead Big Data Engineer - Spark

    QUALYS SECURITY TECHSERVICES PRIVATE LIMITEDPune
    Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!.Description : We are seeking a talented Lead Big ...Show moreLast updated: 30+ days ago
    • Promoted
    Data Architect

    Data Architect

    Tata Consultancy Servicespune, India
    Greetings from TCS Human Resources Team!!!.Years of experience : 8 to 10 Years.Airflow • : Data pipeline orchestration.Data Form & DBT • : Data architecture and modeling. Architecture Design • : Scalable d...Show moreLast updated: 13 days ago