Lead Data Engineer

Job Title :  Lead Data Engineer

Location :  Pune, India

Type :  Full-time

Overview :

We are seeking a skilled Lead Data Engineer with strong expertise in SQL, big data processing, and modern data platforms. The ideal candidate will have hands-on experience with modern unified data platforms (e.g. Databricks, Snowflake) and exposure to Microsoft Azure Fabric (nice to have). This role will support the design, development, and optimisation of scalable data projects.

Key Responsibilities :

Drive initiatives to develop and continuously improve the Data / AI Practice

Design, build, and optimise data pipelines for large-scale structured and unstructured datasets.

Implement ETL / ELT processes using SQL and modern big data frameworks.

Work with Modern Data Platforms to enable data integration, transformation, and analytics.

Collaborate with architects and governance leads to ensure data quality, lineage, and compliance.

Apply big data processing concepts (e.g., partitioning, distributed compute, streaming / batch data flows).

Support performance tuning, monitoring, and automation of data workflows.

Build data modelling processes

Contribute to assessments and adoption of new vendor / platform solutions (e.g., Azure Fabric).

Required Skills & Experience :

Ability to lead data engineering projects from inception to delivery, harnessing effective communication skills to a non-technical audience

Fluent understanding of data modelling to link data entities

Strong proficiency in SQL (queries, optimization, stored procedures).

Solid understanding of big data processing (distributed computing, Spark, Kafka, Delta Lake concepts, data partitioning, schema evolution).

Familiarity with cloud data ecosystems (Azure, AWS, or GCP).

Strong problem-solving skills and ability to work in a collaborative team environment.

Nice to Have :

Experience with Microsoft Azure Fabric.

Knowledge of data governance tools (e.g., Purview, Collibra) and data cataloguing concepts.

Exposure to DevOps / MLOps practices (CI / CD, Git, automation tools).