Talent.com
AWS Cloud Python Data Engineer - ETL/ELT Pipelines
AWS Cloud Python Data Engineer - ETL/ELT Pipelines1HResource Solutions • Chennai
AWS Cloud Python Data Engineer - ETL / ELT Pipelines

AWS Cloud Python Data Engineer - ETL / ELT Pipelines

1HResource Solutions • Chennai
5 days ago
Job description

Data Engineer (L3) - Python & AWS

Job Overview :

We are seeking a highly skilled Data Engineer with strong expertise in AWS cloud services and Python programming. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines, ensuring data availability, quality, and performance across enterprise systems. You will collaborate closely with data analysts, data scientists, and business stakeholders to deliver reliable, high-quality data solutions.

Total experience : 5 to 8 years

Key Responsibilities

  • Design, develop, and maintain ETL / ELT data pipelines using Python and AWS native services (Glue, Lambda, EMR, Step Functions, etc.)
  • Build and manage data lakes and data warehouses using Amazon S3, Redshift, Athena, and Lake Formation
  • Implement data ingestion from diverse sources (RDBMS, APIs, streaming data, on-premise systems)
  • Optimize data workflows for performance, cost, and reliability using AWS tools like Glue Jobs, Athena, and Redshift Spectrum
  • Develop reusable, modular Python-based frameworks for data ingestion, transformation, and validation
  • Work with stakeholders to understand data requirements, model data structures, and ensure data consistency and governance
  • Deploy and manage data infrastructure using Infrastructure as Code (IaC) tools such as Terraform or AWS CloudFormation
  • Implement data quality, monitoring, and alerting using CloudWatch, Glue Data Catalog, or third-party tools
  • Support data security and compliance (IAM roles, encryption, data masking, GDPR policies, etc.)
  • Collaborate with DevOps and ML teams to integrate data pipelines into analytics and AI workflows

Required Skills & Qualifications :

  • Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
  • 5 to 8 years of experience as a Data Engineer or similar role.
  • Strong programming experience in Python (pandas, boto3, PySpark, SQLAlchemy, etc.)
  • Deep hands-on experience with AWS services, including :
  • AWS Glue, Lambda, EMR, Redshift, Athena, S3, Step Functions
  • IAM, CloudWatch, Kinesis (for streaming), and ECS / EKS (for containerized workloads)
  • Experience with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB, MongoDB)
  • Strong knowledge of data modeling, schema design, and ETL orchestration.
  • Familiarity with version control (Git) and CI / CD pipelines for data projects.
  • Understanding of data governance, lineage, and cataloging principles.
  • Excellent problem-solving, debugging, and performance-tuning skills.
  • Preferred / Nice-to-Have Skills :

  • Experience with Apache Spark or PySpark on AWS EMR.
  • Exposure to Airflow, dbt, or similar workflow orchestration tools.
  • Knowledge of containerization (Docker, Kubernetes) and DevOps practices.
  • Experience with machine learning data pipelines or real-time streaming (Kafka, Kinesis).
  • Familiarity with AWS Glue Studio, AWS DataBrew, or AWS Lake Formation.
  • Soft Skills :

  • Strong analytical and communication skills.
  • Ability to work independently and in cross-functional teams.
  • Passion for automation and continuous improvement.
  • Adaptability in fast-paced, evolving cloud environments.
  • (ref : hirist.tech)

    Create a job alert for this search

    Aws Cloud Engineer • Chennai

    Related jobs
    AWS Data Engineer

    AWS Data Engineer

    Vista Applied Solutions Group Inc • Chennai, IN
    Job Summary for AWS Data Engineer : .Job Qualification and Responsibilities for AWS Data Engineer : .Advanced SQL (performance tuning, joins, windows functions). Hands-on with AWS : Glue, Lambda, Redshif...Show more
    Last updated: 11 days ago • Promoted
    Data Engineer-Airflow, Python,Pyspark, Databricks

    Data Engineer-Airflow, Python,Pyspark, Databricks

    NielsenIQ • Chennai, Tamil Nadu, India
    Were seeking a highly motivated Data Engineer to join our agile cross-functional team and drive end-to-end data pipeline development in a cloud-native big data ecosystem. Youll leverage ETL / ELT best...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Cynosure Corporate Solutions • Chennai, Tamil Nadu, India
    We are seeking a highly skilled and experienced Senior / Lead Data Engineer to design develop and maintain scalable reliable and efficient data pipelines and ETL solutions.The role requires strong ...Show more
    Last updated: 18 days ago • Promoted
    GCP Data Engineer- Teradata

    GCP Data Engineer- Teradata

    Saaki Argus & Averil Consulting • Chennai, Tamil Nadu, India
    Quick Apply
    Extensive experience working with.Strong experience migrating data from.Familiarity with ETL / ELT processes and.Understanding of CI / CD tools like. Basic exposure to distributed processing frameworks ...Show more
    Last updated: 30+ days ago
    Python (Airflow / DBT) Engineer

    Python (Airflow / DBT) Engineer

    Luxoft • Chennai, Tamil Nadu, India
    Project Description : This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.The goal is t...Show more
    Last updated: 11 days ago • Promoted
    AWS / Azure Data Engineer

    AWS / Azure Data Engineer

    Axiom Software Solutions Limited • Chennai, TN, IN
    Quick Apply
    Design, implement, and manage scalable ETL / ELT pipelines using AWS services and Databricks.Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data ...Show more
    Last updated: 30+ days ago
    Cloud - Snowflake, AWS

    Cloud - Snowflake, AWS

    Saaki Argus & Averil Consulting • Chennai, Tamil Nadu, India
    Quick Apply
    Cloud Data Engineer - Snowflake, AWS.Encourage applications from all qualified candidates, especially women who are returning to the workforce after a career break (up to 4 years).Develop and imple...Show more
    Last updated: 30+ days ago
    snowflake AWS Engineer

    snowflake AWS Engineer

    Virtusa • Chennai, Tamil Nadu, India
    Design develop and maintain data pipelines and ETL processes using AWS and data transformation workflows using DBT (Data Build Tool). Write efficient reusable and reliable code in and tune data so...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    Xakal - Hire Easy! • Chennai, Tamil Nadu, India
    We are seeking an experienced and certified AWS Data Engineer with hands-on experience in Amazon Redshift, S3, Lambda, Step Functions, and AWS DMS. The Data Engineer should use Python and SQL to bui...Show more
    Last updated: 14 hours ago • Promoted • New!
    AI Engineer (Data Pipelines & RAG)

    AI Engineer (Data Pipelines & RAG)

    BeGig • Chennai, IN
    Job Role- AI Engineer (Data Pipelines & RAG).Work Mode- Remote(6 days working).We are looking for a hands-on AI / Data Engineer (4–7 years) to build and scale data pipelines powering GenAI and agenti...Show more
    Last updated: 12 days ago • Promoted
    Python Data Engineer

    Python Data Engineer

    Dexian India • Chennai, Tamil Nadu, India
    Python Data Engineer : - Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure for efficie...Show more
    Last updated: 25 days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Randstad India • Chennai, Tamil Nadu, India
    Collaborate with Product teams ensuring that raw data is cleansed and transformed.Assist and advise on the re-development and modernisation of end-to-end ETL. Working with AWS cloud infrastructure ...Show more
    Last updated: 27 days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Epergne Solutions • Chennai, Tamil Nadu, India
    Quick Apply
    Job Roles & Responsibilities.Design, build, and optimize data pipelines and scalable data assets.Develop and maintain high-performance code using PySpark / Python with best practices.Perform code...Show more
    Last updated: 30+ days ago
    Python Engineer (Airflow, Cloud)

    Python Engineer (Airflow, Cloud)

    Luxoft • Chennai, Tamil Nadu, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 4 days ago • Promoted
    Python Engineer ( Airflow / DBT Cloud )

    Python Engineer ( Airflow / DBT Cloud )

    Luxoft • Chennai, Tamil Nadu, India
    Project Description : This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.The goal is ...Show more
    Last updated: 4 days ago • Promoted
    Sr. Manager, Python / Big Data Engineer

    Sr. Manager, Python / Big Data Engineer

    Standard Chartered Bank • Chennai, Tamil Nadu, India
    This job is with Standard Chartered Bank, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly....Show more
    Last updated: 16 days ago • Promoted
    Data Engineer

    Data Engineer

    IntraEdge • Chennai, IN
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show more
    Last updated: 30+ days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Tata Consultancy Services • Chennai, Tamil Nadu, India
    Required Technical Skill Set AWS Redshift, Glue, PySpark.Desired Experience Range 4-10 Years.Location of Requirement Chennai / Pune. Desired Competencies (Technical / Behavioral Competency).Must-Have • ...Show more
    Last updated: 30+ days ago • Promoted