Talent.com
Software Engineer-Data Engineering

Software Engineer-Data Engineering

ConfidentialChennai, India
14 days ago
Job description

Career Area

Technology, Digital and Data

Job Description

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

Purpose Of The Role

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

Build the Digital Backbone of Modern Manufacturing

We're assembling a dynamic team to develop and scale our Manufacturing & Supply Digital Platform—a next-generation software framework that transforms how manufacturing and supply operations connect, collaborate, and optimize.

This platform is not an ERP system. It's a purpose-built digital layer that integrates data, processes, and resources across the entire manufacturing lifecycle—from design and engineering to production and distribution.

This initiative is powered by NVIDIA technologies, including the Omniverse platform and AI computing capabilities, enabling immersive digital twins, accelerated simulation, and intelligent automation. You'll be part of a team that's not just building software—but shaping the future of how manufacturing works through AI-driven, collaborative, and scalable digital solutions.

As Part Of This Initiative, You'll Contribute To

Performs implementation, regular problem solving, maintenance and support for a agile software development.

Basic Qualifications

  • Bachelor's degree in computer science or a related field
  • At least 5+ years of experience in Software engineering
  • At least 3+ years of experience in Python development
  • Strong programming skills in Python
  • Should also have experience with Python frameworks, such as Pandas, NumPy, Django, Flask, and Pyramid.
  • Experience with relational databases such as Snowflake, MySQL or PostgreSQL
  • Experience with designing and developing software applications in Python for cloud computing platforms such as AWS or Equivalent.
  • A solid understanding of fundamental computer science principles, including data structures and algorithms, is a must.
  • Capable of thriving in high-pressure situations and delivering results within tight time constraints.
  • Demonstrated passion for technology coupled with an eagerness to contribute to a collaborative team environment.
  • This position requires working onsite five days a week.

Roles & Responsibilities

  • Responsible for designing, developing, and maintaining complex Python applications and services.
  • Mentoring and leading junior developers.
  • Working with other developers to design and implement new features
  • Troubleshooting and debugging code
  • Deploying applications to production.
  • Working with DevOps teams to ensure that applications are running smoothly.
  • Top Candidates Will Have Proven Experience In The Following

  • Designing, creating, deploying, and sustaining software solutions on a large scale.
  • Implementing application architectural patterns, including but not limited to MVC, Microservices, and Event-driven architectures.
  • Utilizing CI / CD tools such as Azure DevOps, Jenkins, GoCD, etc., for seamless software deployment.
  • Deploying and managing software on public cloud platforms like AWS.
  • Collaborating within an Agile framework, preferably following Scrum methodologies.
  • Proficiency in managing and deploying applications using container orchestration tools, specifically ECS (Elastic Container Service). Good understanding of Docker fundamentals.
  • Other AWS services like Lambda, CloudFormation, CloudWatch, S3.
  • Good working knowledge on Python modules like Pandas, NumPy, multithreading, requests, etc
  • Possess a solid understanding of version control systems, with a particular focus on GIT.
  • Proficiency with SQL.
  • Good to have

  • Proficient in managing diverse datastores, including Snowflake, Elasticsearch, MySQL, and Oracle.
  • Well-versed in developing Snowflake procedures, tasks, and other Snowflake components.
  • Expertise and experience in building Large Data Lakes and Data Warehouse (Snowflake Preferred).
  • Experienced in working with message brokers such as AWS SQS and AWS SNS. Lambda, API Gateway, DynamoDB, Aurora, AWS RDS PostgreSQL.
  • Proficient in utilizing batch or stream processing systems, including Apache Spark and AWS Glue.
  • Familiarity with scheduling tools like Apache Airflow and Perfect.
  • Skilled in developing and working with Restful APIs.
  • Hands-on experience with API tools like Swagger, Postman, and Assertible.
  • Advocate of Test-Driven Development (TDD) and Behaviour-Driven Development (BDD).
  • Extensive hands-on experience with testing tools like Selenium and Cucumber, with expertise in seamlessly integrating them into CI / CD pipelines.
  • Posting Dates

    November 3, 2025 - November 16, 2025

    Caterpillar is an Equal Opportunity Employer. Qualified applicants of any age are encouraged to apply

    Not ready to apply Join our Talent Community.

    Skills Required

    S3, Postgresql, AWS Glue, Restful Apis, Apache Airflow, Pyramid, Numpy, Lambda, Docker, snowflake , Mysql, Swagger, Flask, Selenium, Python, Aws, Azure Devops, Cloudformation, Apache Spark, Sql, Jenkins, Django, Git, Pandas, Cloudwatch, ECS, Postman, Cucumber

    Create a job alert for this search

    Software Engineering • Chennai, India

    Related jobs
    • Promoted
    Data Engineer

    Data Engineer

    DigitalzoneChennai, IN
    As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics. Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 18 days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Vista Applied Solutions Group IncChennai, IN
    Job Summary for AWS Data Engineer : .Job Qualification and Responsibilities for AWS Data Engineer : .Advanced SQL (performance tuning, joins, windows functions). Hands-on with AWS : Glue, Lambda, Redshif...Show moreLast updated: 4 days ago
    • Promoted
    Senior Software Engineer - Prompt Engineering

    Senior Software Engineer - Prompt Engineering

    SproutsAIChennai
    Description : What you'll do : - Design & build production serv...Show moreLast updated: 24 days ago
    • Promoted
    Data Engineer

    Data Engineer

    RecroChennai, IN
    Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake. AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    EXLChennai, Tamil Nadu, India
    Collaborate with project stakeholders (client) to identify product and technical requirements.Develop, implement, and tune large-scale distributed systems and pipelines that process large volume of...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer - Azure Cloud

    Senior Data Engineer - Azure Cloud

    ResourcetreeChennai
    Description : Job Description : We are looking for a skilled and experienced Senior Azure Developer to join our team! As part of the team, you will be involve...Show moreLast updated: 30+ days ago
    • Promoted
    AI Software Engineer

    AI Software Engineer

    Quik HireChennai, IN
    This role is ideal for professionals passionate about artificial intelligence, machine learning, and software engineering who want to make a tangible impact on real-world applications.As an AI Soft...Show moreLast updated: 3 days ago
    • Promoted
    Databricks Engineer

    Databricks Engineer

    TTC GroupChennai, IN
    We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform.The ideal candidate will...Show moreLast updated: 2 days ago
    • Promoted
    Developer - Data Engineering & Cloud Analytics

    Developer - Data Engineering & Cloud Analytics

    Geakminds Technologies Private LimitedChennai
    Developer - Data Engineering & Cloud Analytics.Role Overview : Responsible for building, maintaining, and optimizing large-scale data pipelines and analytics solutions lev...Show moreLast updated: 30+ days ago
    • Promoted
    Logitech - Lead Software Engineer - Algorithm & Data Structure

    Logitech - Lead Software Engineer - Algorithm & Data Structure

    LogitechChennai
    Description : Logitech is the Sweet Spot for people who want their actions to have a positive global impact while having the flexibility to do it in their own way.Log...Show moreLast updated: 13 days ago
    • Promoted
    Software Engineer- Data Engineering

    Software Engineer- Data Engineering

    ConfidentialChennai, India
    Your Work Shapes the World at Caterpillar Inc.When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem ...Show moreLast updated: 13 days ago
    • Promoted
    Data Engineer

    Data Engineer

    JMAN GroupChennai, Tamil Nadu, India
    JMAN Group is a fast-growing data engineering & data science consultancy.We work primarily with Private Equity Funds and their Portfolio Companies to create commercial value using Data & Artificial...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    MyData Insights Pvt LtdChennai, IN
    The ideal candidate has a strong background in building scalable data solutions on.CI / CD pipelines, infrastructure automation using Terraform, and cloud-native data engineering practices.Design, bu...Show moreLast updated: 2 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Insight GlobalChennai, IN
    GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts. Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago
    • Promoted
    Principal Data Engineer

    Principal Data Engineer

    CodeMyMobileChennai, IN
    Experience Required - 7 to 10 Years.Are you a Data Engineer who cares about clean engineering, autonomy, and solving real data challenges? If this sounds like you, we’d love to connect!.Email your ...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineering

    Data Engineering

    EXLChennai, India
    Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead the data validation...Show moreLast updated: 30+ days ago
    • Promoted
    AI Software Engineer

    AI Software Engineer

    Taskify AIChennai, IN
    This role is ideal for professionals passionate about artificial intelligence, machine learning, and software engineering who want to make a tangible impact on real-world applications.As an AI Soft...Show moreLast updated: 3 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    JMAN GroupChennai, Tamil Nadu, India
    JMAN Group is a fast-growing data engineering & data science consultancy.We work primarily with Private Equity Funds and their Portfolio Companies to create commercial value using Data & Artificial...Show moreLast updated: 24 days ago