Talent.com
Big Data Pipeline Lead
Big Data Pipeline LeadHARP • Republic Of India, IN
Big Data Pipeline Lead

Big Data Pipeline Lead

HARP • Republic Of India, IN
14 hours ago
Job description

Job Summary :

We are looking for an experienced and motivated GCP Big Data Engineer to join our team in a leadership capacity . The ideal candidate will have 5 years of relevant experience in data engineering, with a strong focus on Google Cloud Platform (GCP) , SQL , PySpark , and ETL processes . This role requires strong technical expertise and leadership capabilities to guide and mentor a team of engineers while ensuring high-quality data solutions.

Key Responsibilities :

  • Design, develop, and maintain scalable and efficient data pipelines on Google Cloud Platform (GCP) .
  • Work with PySpark to process large-scale datasets and optimize performance.
  • Write complex and efficient SQL queries for data extraction, transformation, and analysis.
  • Lead the implementation of ETL workflows and ensure data accuracy, completeness, and integrity.
  • Collaborate with cross-functional teams including data analysts, architects, and product managers to define data needs.
  • Provide technical leadership, mentorship, and code reviews for junior engineers.
  • Drive best practices for data engineering and cloud-based data processing.

Required Skills & Qualifications :

  • 5 years of experience in Data Engineering roles.
  • Proven experience with Google Cloud Platform (GCP) and its data services (e.G., BigQuery, Dataflow, Cloud Storage).
  • Strong programming skills in PySpark and Python .
  • Advanced proficiency in SQL and working with large, complex datasets.
  • Deep understanding of ETL frameworks and data pipeline orchestration.
  • Experience leading or mentoring teams in a technical capacity.
  • Excellent communication and problem-solving skills.
  • Preferred Qualifications :

  • GCP Certification (e.G., Professional Data Engineer) is a plus.
  • Experience with CI / CD pipelines and data pipeline automation.
  • Familiarity with Agile / Scrum methodologies.
  • Create a job alert for this search

    Big Data Lead • Republic Of India, IN

    Related jobs
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    HNR Tech • Republic Of India, IN
    We are seeking an experienced Senior Data Engineer to design, build, and optimize our modern data ecosystem.The ideal candidate will have deep expertise in data architecture, pipeline orchestration...Show more
    Last updated: 6 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    ListenFirst • Jaipur, Republic Of India, IN
    We are looking for a skilled and motivated Senior Data Engineer with strong experience in data engineering and ETL processes. The ideal candidate should be comfortable working with any object-orient...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    EXL • Pune, Republic Of India, IN
    Develop high quality, secure and scalable data pipelines using spark, Scala / Python / Java on Hadoop or object storage like MinIO. Leverage technologies and solutions to innovate with increasingly larg...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    TECHY GEEKS • Chennai, Republic Of India, IN
    Qualifications - 5+ years of experience in Data Engineering or related roles.Strong proficiency in SQL, with hands-on experience in Snowflake or similar cloud data warehouses.Expertise in ETL dev...Show more
    Last updated: 14 hours ago • Promoted • New!
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    EXL • Chennai, Republic Of India, IN
    Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead the data validation...Show more
    Last updated: 3 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    PwC India • Republic Of India, IN
    As a Senior Associate, you will take ownership of designing and delivering complex data engineering solutions across cloud and on-premise environments. You’ll lead the technical delivery of ETL / ELT ...Show more
    Last updated: 8 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    Eucloid Data Solutions • Chennai, Republic Of India, IN
    Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure for our cli...Show more
    Last updated: 21 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    Persistent Systems • Pune, Republic Of India, IN
    We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services. The ideal candidate will have a strong backgro...Show more
    Last updated: 30+ days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    Luxoft • Chennai, Republic Of India, IN
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 2 days ago • Promoted
    Lead Data Pipeline Architect

    Lead Data Pipeline Architect

    Lear Corporation • Pune, Republic Of India, IN
    Lear, a global automotive technology leader in Seating and E-Systems, is Making every drive better by delivering intelligent in-vehicle experiences for customers around the world.With over 100 year...Show more
    Last updated: 20 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    Spocto X (A Yubi Company) • Chennai, Republic Of India, IN
    Yubi, formerly known as CredAvenue, is re-defining global debt markets by freeing the flow of finance between borrowers, lenders, and investors. We are the world's possibility platform for the disco...Show more
    Last updated: 3 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    MakeMyTrip • Republic Of India, IN
    At MakeMyTrip (MMT), technology is at the heart of everything we do.As a leading player in the travel industry, we leverage cutting-edge solutions like AI, machine learning, and cloud infrastructur...Show more
    Last updated: 23 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    SGS & Co • Chennai, Republic Of India, IN
    Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in. The ideal candidate will play a ke...Show more
    Last updated: 23 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    Matrimony.com Limited • Chennai, Republic Of India, IN
    Work Location : Chennai (Work From Office).The Decision Support Systems (DSS) function enables data-driven decision-making across Matrimony. Com by delivering high-quality data pipelines, analytical ...Show more
    Last updated: 16 days ago • Promoted
    Principal Data Pipeline Engineer

    Principal Data Pipeline Engineer

    EXL • Pune, Republic Of India, IN
    We are seeking a highly skilled.The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a fast-paced envir...Show more
    Last updated: 8 days ago • Promoted
    Data Pipeline Manager

    Data Pipeline Manager

    INDI Staffing Services • Republic Of India, IN
    At INDI, we're passionate about empowering individuals and businesses worldwide.Our cutting-edge recruiters connect leading companies with top talent, fostering a dynamic environment where innovati...Show more
    Last updated: 23 days ago • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    Saks Global India • Republic Of India, IN
    Data Engineer and Operations Lead.Saks Global is the largest multi-brand luxury retailer in the world, comprising Saks Fifth Avenue, Neiman Marcus, Bergdorf Goodman, Saks OFF 5TH, Last Call and Hor...Show more
    Last updated: 30+ days ago • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    Bloom AI • New Delhi, Republic Of India, IN
    Bloom AI is a modern intelligence layer that accelerates decision-making through AI-driven synthesized insights.We empower enterprises to unlock the value of data with human-like synthesis and deci...Show more
    Last updated: 3 days ago • Promoted