Data Engineer lead
Role Overview
The Senior Data Engineer will design and implement scalable, secure, and high-performance data solutions using the Azure Data Stack . The role focuses on building reliable data pipelines, ensuring data quality, and enabling downstream analytics and AI workloads through well-structured, governed, and performant data models.
Key Responsibilities
- Design, build, and maintain ETL / ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines .
- Develop and manage data lakes and data warehouses on Azure Data Lake Storage (ADLS) and Azure Synapse Analytics .
- Integrate data from multiple sources — including core banking, CRM, trading, and partner systems — into unified data models.
- Build parameterized pipelines , implement data validation , and automate ingestion and transformation workflows.
- Collaborate with Cloud Architects , Data Scientists , and BI Teams to enable high-quality, reusable data assets.
- Establish data partitioning , indexing , and incremental load strategies for performance optimization.
- Enable pipeline orchestration , logging , and error handling using ADF features and Azure Monitor.
- Integrate CI / CD workflows via Azure DevOps for deployment automation and version control.
- Ensure data security, encryption, masking, and retention policies aligned with RBI / SEBI guidelines.
- Contribute to data governance, cataloging, and lineage tracking using Azure Purview or equivalent tools.
Technical Skills
Mandatory :Azure Data Stack – ADF , Synapse , ADLS , Azure SQL , Event Hub / Service Bus .Strong SQL skills and ETL design principles.Experience with Azure DevOps CI / CD and ARM templates / Terraform for infrastructure automation.Good understanding of data modelling , data quality , and incremental processing techniques.Good to Have :Experience with Python-based ETL scripting and API-driven data ingestion .Knowledge of Power BI datasets , Azure Monitor , and Log Analytics .Familiarity with DataOps and data governance frameworks .Experience
8–12 years of experience in data engineering, with at least 4–5 years on Azure Cloud .Proven experience in data lake / data warehouse implementation using Azure-native services.Exposure to financial data models (Customer, Loans, Investments, Transactions) preferred.Job Location : Bangalore (initially will have to work from our Kochi Office, as Bangalore office is yet to open)