Talent.com
Python Data Engineer
Python Data EngineerConfidential • Chennai, India
Python Data Engineer

Python Data Engineer

Confidential • Chennai, India
26 days ago
Job description

Python Data Engineer : -

  • Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
  • Constructing infrastructure for efficient ETL processes from various sources and storage systems.
  • Collaborating closely with Product Managers and Business Managers to design technical solutions aligned with business requirements.
  • Leading the implementation of algorithms and prototypes to transform raw data into useful information.
  • Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI / ML transformations.
  • Creating innovative data validation methods and data analysis tools.
  • Ensuring compliance with data governance and security policies.
  • Interpreting data trends and patterns to establish operational alerts.
  • Developing analytical tools, programs, and reporting mechanisms.
  • Conducting complex data analysis and presenting results effectively.
  • Preparing data for prescriptive and predictive modeling.
  • Continuously exploring opportunities to enhance data quality and reliability.
  • Applying strong programming and problem-solving skills to develop scalable solutions.
  • Passion for testing strategy, problem-solving, and continuous learning.
  • Willingness to acquire new skills and knowledge.
  • Possess a product / engineering mindset to drive impactful data solutions.
  • Experience working in distributed environments with global teams.

Technical Skills and Experience requirements :

  • Minimum 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
  • High proficiency in Python, Scala and Spark for applied large-scale data processing.
  • Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive.
  • Solid understanding of batch and streaming data processing techniques.
  • Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion.
  • Expert-level ability to write complex, optimized SQL queries across extensive data volumes.
  • Experience with RDBMS and OLAP databases like MySQL, Snowflake.
  • Familiarity with Agile methodologies.
  • Obsession for service observability, instrumentation, monitoring, and alerting.
  • Knowledge or experience in architectural best practices for building data lakes.
  • Qualification : Bachelor's / Master's Degree.

    Relevant Development Experience : 8+ Years

    Relevant Experience : 8+ Years (as Python and Data Engineer)

    Overall IT Experience : 8 to 12 Years

    Skills Required

    Agile Methodologies, Scala, Sql, Hive, Mysql, snowflake , Spark, Data Lake, Python

    Create a job alert for this search

    Data Engineer Python • Chennai, India

    Related jobs
    Data Engineer-Airflow, Python,Pyspark, Databricks

    Data Engineer-Airflow, Python,Pyspark, Databricks

    NielsenIQ • Chennai, Tamil Nadu, India
    Were seeking a highly motivated Data Engineer to join our agile cross-functional team and drive end-to-end data pipeline development in a cloud-native big data ecosystem. Youll leverage ETL / ELT best...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer - Python / Spark

    Senior Data Engineer - Python / Spark

    Getinz • Chennai
    Description : Role Overview : We are looking for a highly skilled and motivated Senior Data Engineer to join our clients tea...Show more
    Last updated: 28 days ago • Promoted
    Platform Engineer – Python / Databricks / Notebooks / Kubernetes

    Platform Engineer – Python / Databricks / Notebooks / Kubernetes

    Syntasa • Chennai, IN
    Platform Engineer – Python / Databricks / Notebooks / Kubernetes.Syntasa is seeking a high-caliber and dedicated.Syntasa Technologies India Private Limited. This position offers an exciting opportuni...Show more
    Last updated: 30+ days ago • Promoted
    Azure Data Engineer

    Azure Data Engineer

    Tata Consultancy Services • Greater Chennai Area, India
    TCS is hiring for AzureData Engineer.Extensive coding experience including Python / Pyspark.Experience developing in Azure with key data technologies (e. ADLS, ADF, Azure Databricks etc.Software devel...Show more
    Last updated: 5 days ago • Promoted
    Python (Airflow / DBT) Engineer

    Python (Airflow / DBT) Engineer

    Luxoft • Chennai, Tamil Nadu, India
    Project Description : This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.The goal is t...Show more
    Last updated: 8 days ago • Promoted
    Data Engineer

    Data Engineer

    Tata Consultancy Services • Chennai, Tamil Nadu, India
    TCS is hiring for Azure Data Engineer, please find the below JD.Experience range – 6 to 10 years.Skills Required - Azure Databricks, Azure data factory, Azure synapse, Pyspark.Strong design and dat...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    System Soft Technologies • Saint Thomas Mount, Tamil Nadu, India
    Title : Data Engineer Location : Remote (3–4-hour time zone overlaps with EST if off shore) Duration : 6 months Musts : Python NextFlow Plus : Computer Vision Image Processing Project Overview : Exper...Show more
    Last updated: 2 days ago • Promoted
    Senior Python Data Engineer – ETL & Pipeline Development (5 to 12 yrs)

    Senior Python Data Engineer – ETL & Pipeline Development (5 to 12 yrs)

    AIMLEAP • Chennai, IN
    Senior Python Data Engineer – ETL & Pipeline Development.Remote (WFH) / Bangalore / India.Tech / MCA / Computer Science / IT. IT / Data / AI / LegalTech / Enterprise Solutions.ETL pipelines and data...Show more
    Last updated: 3 days ago • Promoted
    Data Engineer

    Data Engineer

    EXL • Chennai, India
    Collaborate with project stakeholders (client) to identify product and technical requirements.Develop, implement, and tune large-scale distributed systems and pipelines that process large volume of...Show more
    Last updated: 30+ days ago • Promoted
    AI Engineer (Data Pipelines & RAG)

    AI Engineer (Data Pipelines & RAG)

    BeGig • Chennai, IN
    Job Role- AI Engineer (Data Pipelines & RAG).Work Mode- Remote(6 days working).We are looking for a hands-on AI / Data Engineer (4–7 years) to build and scale data pipelines powering GenAI and agenti...Show more
    Last updated: 10 days ago • Promoted
    Data Engineer

    Data Engineer

    Aceolution • Chennai, IN
    Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show more
    Last updated: 30+ days ago • Promoted
    Python Data Engineer

    Python Data Engineer

    Dexian India • Chennai, Tamil Nadu, India
    Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure for efficient ETL processes from vari...Show more
    Last updated: 22 days ago • Promoted
    Data engineer - Python Databricks - Pune / Chennai

    Data engineer - Python Databricks - Pune / Chennai

    NielsenIQ • Chennai, Tamil Nadu, India
    We strongly encourage applications from women individuals from underrepresented communities and candidates who will contribute to the diversity of our team. Write complex algorithms to get an optima...Show more
    Last updated: 15 days ago • Promoted
    Python Engineer (Airflow, Cloud)

    Python Engineer (Airflow, Cloud)

    Luxoft • Chennai, Tamil Nadu, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 1 day ago • Promoted
    Python Engineer ( Airflow / DBT Cloud )

    Python Engineer ( Airflow / DBT Cloud )

    Luxoft • Chennai, Tamil Nadu, India
    Project Description : This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.The goal is ...Show more
    Last updated: 1 day ago • Promoted
    Data Engineer

    Data Engineer

    IntraEdge • Chennai, IN
    Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer (Airflow, Azure)

    Data Engineer (Airflow, Azure)

    Luxoft • Greater Chennai Area, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 2 days ago • Promoted
    AWS Cloud Python Data Engineer - ETL / ELT Pipelines

    AWS Cloud Python Data Engineer - ETL / ELT Pipelines

    1HResource Solutions • Chennai
    Data Engineer (L3) - Python & AWS Job Overview : We are seeking a highly skilled Data Engineer with strong expertise in AWS cloud services and Python ...Show more
    Last updated: 3 days ago • Promoted