Talent.com
Senior Data Engineering Lead - Azure Platform
Senior Data Engineering Lead - Azure PlatformWorkassist • Gurgaon
Senior Data Engineering Lead - Azure Platform

Senior Data Engineering Lead - Azure Platform

Workassist • Gurgaon
30+ days ago
Job description

Description :

Position : Senior Data Engineering Team Leader (Azure Focus)

Work Level : Senior Leadership

Industry Type : IT Services & Consulting

Job Summary :

We are seeking a Disciplined and innovation-focused Senior Data Engineering Team Leader to lead a team in designing and implementing advanced data solutions on the Microsoft Azure cloud platform. This leadership role requires a minimum of three years of hands-on Azure Data Engineering experience, coupled with deep proficiency in Databricks, PySpark, and complex SQL. The leader will be responsible for driving technical innovation, mentoring a team of 5+ members, and delivering high-performance, scalable data pipelines for our IT Services and Consulting clients.

Job Description :

Technical Leadership and Innovation :

  • Provide Senior Leadership and technical oversight for a team of 5 or more Data Engineers, fostering a culture of technical excellence, accountability, and continuous improvement.
  • Drive Innovation within the data engineering function by evaluating and integrating cutting-edge Azure services, advanced Databricks features, and modern data processing methodologies to improve pipeline efficiency and capability.
  • Lead the technical design sessions and architectural reviews, ensuring all data solutions are scalable, resilient, and adhere to enterprise-level security and governance standards on Azure.
  • Instill Disciplined engineering practices, including robust code review processes, documentation standards, and continuous adherence to CI / CD pipelines for all data assets.

Azure Data Platform Engineering :

  • Possess 3+ years of mandatory, hands-on experience leading and delivering complex data engineering development projects within the Microsoft Azure cloud ecosystem.
  • Design and implement high-volume, low-latency ETL / ELT data pipelines using core Azure services, including Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), and Azure Key Vault.
  • Demonstrate 2+ years of deep, proven proficiency in developing, optimizing, and deploying complex data processing jobs utilizing Databricks notebooks and clusters.
  • Master the use of PySpark (Python with Spark) to perform advanced data transformations, aggregations, and feature engineering at scale within the Databricks environment.
  • Data Modeling and Team Management :

  • Develop and optimize complex SQL queries, stored procedures, and data models for consumption by reporting tools and analytical applications, managing data integration with services like Azure SQL Database or Synapse Analytics.
  • Provide expert-level guidance on performance tuning for Spark jobs, Databricks clusters, and SQL execution plans to ensure optimal query response times and resource utilization.
  • Demonstrate 2+ years of mandatory experience effectively leading and managing the day-to-day work, career development, and technical delivery of a data engineering team of 5 or more members.
  • Act as the primary technical point of contact, leveraging strong communication skills to articulate complex technical issues and architectural decisions to clients, product owners, and senior stakeholders.
  • Required Skills & Qualifications :

  • Experience : Mandatory 5+ years total experience in data engineering, including 3+ years of dedicated experience in Azure development projects.
  • Leadership : Mandatory 2+ years experience successfully leading and mentoring a data engineering team of 5 or more members.
  • Cloud & Platform : Deep technical expertise in the Microsoft Azure data ecosystem.
  • Big Data Tools : Mandatory proficiency (2+ years) with Databricks, PySpark for scalable data processing, and expert-level SQL for data manipulation and modeling.
  • Core Skills : Exceptional technical communication, stakeholder management, and problem-solving abilities within a consulting environment.
  • Education : Mandatory Graduate degree.
  • Preferred Skills :

  • Advanced Azure Services : Hands-on experience with Azure Synapse Analytics (SQL Pool, Spark Pool) for integrated data warehousing and big data workloads.
  • Data Visualization : Experience integrating data pipelines with leading business intelligence tools such as Tableau and Power BI for final data consumption.
  • Programming Languages : Proficiency in Scala or Java in addition to Python for high-performance data processing tasks.
  • DevOps for Data : Experience with specific CI / CD tools (e.g., Azure DevOps, GitHub Actions) for automating Databricks and Azure Data Factory deployments.
  • Data Governance : Familiarity with Azure Purview or other data governance tools for metadata management and data lineage tracking.
  • (ref : hirist.tech)

    Create a job alert for this search

    Data Engineering Lead • Gurgaon

    Related jobs
    Xebia - Senior Azure Data Engineer

    Xebia - Senior Azure Data Engineer

    Xebia IT Architects India Pvt Ltd • Gurugram
    Description : - Data Engineer (Azure) - Data Processing on Azure : Azure Data Factory, Streaming Analytics, Event Hubs, Azure Databricks, Data Mi...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Zyoin Group • Gurugram
    Description : We are seeking a Senior Data Engineer with 4+ years of experience in building scalable, distributed, and h...Show more
    Last updated: 30+ days ago • Promoted
    Azure Senior Data Engineer

    Azure Senior Data Engineer

    Tata Consultancy Services • gurugram, uttar pradesh, in
    TCS is hiring for AWS Data Engineer.Extensive coding experience including Python / Pyspark.Experience developing in Azure with key data technologies (e. ADLS, ADF, Azure Databricks etc.Software develo...Show more
    Last updated: 17 days ago • Promoted
    Auxo AI - Senior Data Engineer - Google Cloud Platform

    Auxo AI - Senior Data Engineer - Google Cloud Platform

    AuxoAI • Gurugram
    Description : AuxoAI is seeking a Senior Data Engineer to lead the design, development, and optimization of modern data ...Show more
    Last updated: 8 days ago • Promoted
    Publicis Sapient - Senior Manager - Data Engineering

    Publicis Sapient - Senior Manager - Data Engineering

    Publicis Sapient • Gurugram
    Job Description : We are looking for a technically strong, business friendly Data Engineering Leader who can drive initiatives across data lake infrastructure, large-...Show more
    Last updated: 21 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Donyati • gurgaon, India
    We are seeking a highly skilled Senior Data Engineer to join our team in building a modern data platform on AWS.You will play a key role in transitioning from legacy systems to a scalable, cloud-na...Show more
    Last updated: 7 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    VRIZE • Gurgaon, Haryana, India
    Responsibilities : Design, architect, administer, and manage large database systems, particularly on Microsoft platforms such as SQL Server and Azure Database. Develop and optimize complex SQL querie...Show more
    Last updated: 1 day ago • Promoted
    ProcDNA - Data Engineering Lead - ETL Tools

    ProcDNA - Data Engineering Lead - ETL Tools

    PROCDNA ANALYTICS PRIVATE LIMITED • Gurugram
    Description : ProcDNA is a global consulting firm.We fuse design thinking with cutting-edge technology to create game-changing Commercial Analytics and Technology sol...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Architect- Snowflake

    Senior Data Architect- Snowflake

    USEReady • gurgaon, haryana, in
    USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage. USEReady partners with cloud...Show more
    Last updated: 6 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Mastek • gurugram, India
    We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show more
    Last updated: 20 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    S&P Global • Gurgaon, Haryana, India
    This job is with S&P Global, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly.About the Rol...Show more
    Last updated: 27 days ago • Promoted
    Macquarie - Senior Associate - Data Engineering

    Macquarie - Senior Associate - Data Engineering

    Macquarie • Gurugram
    Description : Join our Commodities Data Analyst team, focusing on global oil, metal, and freight movement within the Energy Markets group.This is an exciting ...Show more
    Last updated: 2 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Talentmatics • Gurugram, Haryana, India
    ETL workflows, and big data processing on Google Cloud.This role includes technical leadership, hands-on development, and driving best practices. Design & develop ETL / ELT pipelines using.Lead data e...Show more
    Last updated: 1 day ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    KOGTA FINANCIAL (INDIA) LIMITED • Gurugram, Haryana, India
    ETL & Data Warehouse Developer.As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing ETL pipelines and scalable data warehouse solutions on...Show more
    Last updated: 4 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    RapidBrains • gurgaon, haryana, in
    Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show more
    Last updated: 13 days ago • Promoted
    Azure Data Engineering Lead - ETL / PySpark

    Azure Data Engineering Lead - ETL / PySpark

    Techno Wise • Gurugram
    Location : Gurugram (Work from Office 5 Days) Notice Period : Immediate to 45 Days About the Role : We are looking...Show more
    Last updated: 30+ days ago • Promoted
    Data Architect - Azure Platform

    Data Architect - Azure Platform

    CloudHire • Gurgaon
    Description : Position Overview : USEReady is seeking an experienced Data Architect with deep expertise in Sno...Show more
    Last updated: 22 hours ago • Promoted • New!
    Director - Data Engineering

    Director - Data Engineering

    Marsh McLennan • Gurgaon, Haryana, India
    This job is with Marsh McLennan, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly.Job title...Show more
    Last updated: 16 days ago • Promoted