Talent.com
AWS Cloud Python Data Engineer - ETL/ELT Pipelines
AWS Cloud Python Data Engineer - ETL/ELT Pipelines1HResource Solutions • Chennai
AWS Cloud Python Data Engineer - ETL / ELT Pipelines

AWS Cloud Python Data Engineer - ETL / ELT Pipelines

1HResource Solutions • Chennai
10 days ago
Job description

Data Engineer (L3) - Python & AWS

Job Overview :

We are seeking a highly skilled Data Engineer with strong expertise in AWS cloud services and Python programming. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines, ensuring data availability, quality, and performance across enterprise systems. You will collaborate closely with data analysts, data scientists, and business stakeholders to deliver reliable, high-quality data solutions.

Total experience : 5 to 8 years

Key Responsibilities

  • Design, develop, and maintain ETL / ELT data pipelines using Python and AWS native services (Glue, Lambda, EMR, Step Functions, etc.)
  • Build and manage data lakes and data warehouses using Amazon S3, Redshift, Athena, and Lake Formation
  • Implement data ingestion from diverse sources (RDBMS, APIs, streaming data, on-premise systems)
  • Optimize data workflows for performance, cost, and reliability using AWS tools like Glue Jobs, Athena, and Redshift Spectrum
  • Develop reusable, modular Python-based frameworks for data ingestion, transformation, and validation
  • Work with stakeholders to understand data requirements, model data structures, and ensure data consistency and governance
  • Deploy and manage data infrastructure using Infrastructure as Code (IaC) tools such as Terraform or AWS CloudFormation
  • Implement data quality, monitoring, and alerting using CloudWatch, Glue Data Catalog, or third-party tools
  • Support data security and compliance (IAM roles, encryption, data masking, GDPR policies, etc.)
  • Collaborate with DevOps and ML teams to integrate data pipelines into analytics and AI workflows

Required Skills & Qualifications :

  • Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
  • 5 to 8 years of experience as a Data Engineer or similar role.
  • Strong programming experience in Python (pandas, boto3, PySpark, SQLAlchemy, etc.)
  • Deep hands-on experience with AWS services, including :
  • AWS Glue, Lambda, EMR, Redshift, Athena, S3, Step Functions
  • IAM, CloudWatch, Kinesis (for streaming), and ECS / EKS (for containerized workloads)
  • Experience with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB, MongoDB)
  • Strong knowledge of data modeling, schema design, and ETL orchestration.
  • Familiarity with version control (Git) and CI / CD pipelines for data projects.
  • Understanding of data governance, lineage, and cataloging principles.
  • Excellent problem-solving, debugging, and performance-tuning skills.
  • Preferred / Nice-to-Have Skills :

  • Experience with Apache Spark or PySpark on AWS EMR.
  • Exposure to Airflow, dbt, or similar workflow orchestration tools.
  • Knowledge of containerization (Docker, Kubernetes) and DevOps practices.
  • Experience with machine learning data pipelines or real-time streaming (Kafka, Kinesis).
  • Familiarity with AWS Glue Studio, AWS DataBrew, or AWS Lake Formation.
  • Soft Skills :

  • Strong analytical and communication skills.
  • Ability to work independently and in cross-functional teams.
  • Passion for automation and continuous improvement.
  • Adaptability in fast-paced, evolving cloud environments.
  • (ref : hirist.tech)

    Create a job alert for this search

    Aws Cloud Engineer • Chennai

    Related jobs
    Data Engineer-Airflow, Python,Pyspark, Databricks

    Data Engineer-Airflow, Python,Pyspark, Databricks

    NielsenIQ • Chennai, Tamil Nadu, India
    Were seeking a highly motivated Data Engineer to join our agile cross-functional team and drive end-to-end data pipeline development in a cloud-native big data ecosystem. Youll leverage ETL / ELT best...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Cynosure Corporate Solutions • Chennai, Tamil Nadu, India
    We are seeking a highly skilled and experienced Senior / Lead Data Engineer to design develop and maintain scalable reliable and efficient data pipelines and ETL solutions.The role requires strong ...Show more
    Last updated: 23 days ago • Promoted
    GCP Data Engineer- Teradata

    GCP Data Engineer- Teradata

    Saaki Argus & Averil Consulting • Chennai, Tamil Nadu, India
    Quick Apply
    Extensive experience working with.Strong experience migrating data from.Familiarity with ETL / ELT processes and.Understanding of CI / CD tools like. Basic exposure to distributed processing frameworks ...Show more
    Last updated: 30+ days ago
    Python Data Engineer

    Python Data Engineer

    Dexian India • Chennai, Tamil Nadu, India
    Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure for efficient ETL processes from vari...Show more
    Last updated: 24 days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    Luxoft • Chennai, Tamil Nadu, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 15 hours ago • Promoted • New!
    AWS / Azure Data Engineer

    AWS / Azure Data Engineer

    Axiom Software Solutions Limited • Chennai, TN, IN
    Quick Apply
    Design, implement, and manage scalable ETL / ELT pipelines using AWS services and Databricks.Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data ...Show more
    Last updated: 30+ days ago
    Cloud - Snowflake, AWS

    Cloud - Snowflake, AWS

    Saaki Argus & Averil Consulting • Chennai, Tamil Nadu, India
    Quick Apply
    Cloud Data Engineer - Snowflake, AWS.Encourage applications from all qualified candidates, especially women who are returning to the workforce after a career break (up to 4 years).Develop and imple...Show more
    Last updated: 30+ days ago
    Data Engineer (Airflow, Azure)

    Data Engineer (Airflow, Azure)

    Luxoft • Chennai, Tamil Nadu, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 8 days ago • Promoted
    AWS Senior Data Engineer

    AWS Senior Data Engineer

    Tata Consultancy Services • Chennai, Tamil Nadu, India
    TCS is Hiring of AWS Senior Data Engineer Experience : 5 - 10 Yrs Location : Chennai Mode of Interview : Virtual Technical Skill Set 10 + years of Overall experience in AWS Cloud Technologies Profic...Show more
    Last updated: 18 days ago • Promoted
    Python Engineer (Airflow, Cloud)

    Python Engineer (Airflow, Cloud)

    Luxoft • Chennai, Tamil Nadu, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 2 days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Randstad India • Chennai, Tamil Nadu, India
    Collaborate with Product teams ensuring that raw data is cleansed and transformed.Assist and advise on the re-development and modernisation of end-to-end ETL. Working with AWS cloud infrastructure ...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    IntraEdge • Chennai, Tamil Nadu, India
    Job Title : Data Engineer Experience : 5–9 years Location : [Remote] Job Summary : We are looking for a skilled Data Engineer with strong expertise in Python, PySpark, AWS services (Glue, Lambda)...Show more
    Last updated: 30+ days ago • Promoted
    Cloud Data Engineer

    Cloud Data Engineer

    Luxoft • Chennai, Tamil Nadu, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 15 hours ago • Promoted • New!
    Senior Python Data Engineer

    Senior Python Data Engineer

    iVoyant • Chennai, IN
    Join a dynamic engineering team working on a high-impact tax reporting platform for the 2025 fiscal season.The core goal is to modernize and significantly accelerate the generation of Excel-based r...Show more
    Last updated: 17 days ago • Promoted
    Python Engineer ( Airflow / DBT Cloud )

    Python Engineer ( Airflow / DBT Cloud )

    Luxoft • Chennai, Tamil Nadu, India
    Project Description : This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.The goal is ...Show more
    Last updated: 9 days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Tata Consultancy Services • Chennai, Tamil Nadu, India
    Required Technical Skill Set AWS Redshift, Glue, PySpark.Desired Experience Range 4-10 Years.Location of Requirement Chennai / Pune. Desired Competencies (Technical / Behavioral Competency).Must-Have • ...Show more
    Last updated: 30+ days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    EXL • Chennai, Chennai (district), India
    Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead the data validation...Show more
    Last updated: 17 hours ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    Ironbook AI • Saint Thomas Mount, Tamil Nadu, India
    Role Summary We are looking for a Senior Data Engineer to design, build, and optimize high-performance data pipelines and data systems. The ideal candidate will have strong experience with cloud p...Show more
    Last updated: 1 hour ago • Promoted • New!