Data Engineering Lead

Data Engineer lead Role Overview

The Senior Data Engineer will design and implement scalable, secure, and high-performance data solutions using the Azure Data Stack . The role focuses on building reliable data pipelines, ensuring data quality, and enabling downstream analytics and AI workloads through well-structured, governed, and performant data models.

Key Responsibilities

Design, build, and maintain ETL / ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines .

Develop and manage data lakes and data warehouses on Azure Data Lake Storage (ADLS) and Azure Synapse Analytics .

Integrate data from multiple sources — including core banking, CRM, trading, and partner systems — into unified data models.

Build parameterized pipelines , implement data validation , and automate ingestion and transformation workflows.

Collaborate with Cloud Architects , Data Scientists , and BI Teams to enable high-quality, reusable data assets.

Establish data partitioning , indexing , and incremental load strategies for performance optimization.

Enable pipeline orchestration , logging , and error handling using ADF features and Azure Monitor.

Integrate CI / CD workflows via Azure DevOps for deployment automation and version control.

Ensure data security, encryption, masking, and retention policies aligned with RBI / SEBI guidelines.

Contribute to data governance, cataloging, and lineage tracking using Azure Purview or equivalent tools.

Technical Skills

Mandatory :

Azure Data Stack – ADF , Synapse , ADLS , Azure SQL , Event Hub / Service Bus .

Strong SQL skills and ETL design principles.

Experience with Azure DevOps CI / CD and ARM templates / Terraform for infrastructure automation.

Good understanding of data modelling , data quality , and incremental processing techniques.

Good to Have :

Experience with Python-based ETL scripting and API-driven data ingestion .

Knowledge of Power BI datasets , Azure Monitor , and Log Analytics .

Familiarity with DataOps and data governance frameworks .

Experience

8–12 years of experience in data engineering, with at least 4–5 years on Azure Cloud .

Proven experience in data lake / data warehouse implementation using Azure-native services.

Exposure to financial data models (Customer, Loans, Investments, Transactions) preferred.

Job Location : Bangalore (initially will have to work from our Kochi Office, as Bangalore office is yet to open)