Lead Engineer (MS Fabric Migration & Development)

Innover DigitalBengaluru, Karnataka, India

1 day ago

Job description

Architectural Blueprint : Define the target state architecture within Microsoft Fabric, specifically optimizing for OneLake and theLakehouse paradigm to handle 30+ TB of data.

Pipeline & SP Conversion Strategy : Develop the methodology for translating 20+ complex Azure Data Factory (ADF) pipelines and 30+ stored procedures into highly efficient Fabric Data Pipelines,Spark notebooks (PySpark), and Fabric Warehouse objects.

Performance Leadership : Establish performance benchmarks, select optimal Fabric features (like V-Order and Delta Lake), and oversee tuning efforts to ensure superior query performance post-migration.

Data Governance and Security (Fabric-First)

Unified Governance : Design and implement a centralized data access and security model across Fabric, ensuring consistency, compliance, and leveraging Microsoft Purview integration for unified data governance.

Quality & Lineage : Define standards for data quality checks and guarantee transparent data lineage for all migrated and new data flows.

Fabric DevOps and Automation (CI / CD)

DevOps Implementation : Lead the design and hands-on implementation of the end-to-end CI / CD pipeline for Fabric artifacts using Azure DevOps or GitHub Actions , automating deployment across Development, Test, and Production environments.

Version Control : Mandate and manage Git integration for all Fabric items, establishing branching, merging, and release management policies for the entire engineering team.

Required Qualifications :

Education : Bachelor or Masters degree in Computer Science, Engineering, or a related field.

Experience :

Expertise : 8+ years in data engineering roles, with at least 2 years in a technical leadership or architectural capacity.
Dual-Platform Mastery : Deep, demonstrable experience with Azure Data Factory, Azure Synapse Analytics, and Azure SQL DW / DB , paired with expert-level hands-on experience in Microsoft Fabric .
DevOps : Proven track record of designing and implementing CI / CD pipelines for data platforms, including experience with Git, YAML, and ARM / Bicep / PowerShell .
Data Languages : Expert proficiency in T-SQLand advanced capability in Python (PySpark) for large-scale data transformation.
Certifications (Highly Preferred) : Microsoft Certified : Fabric Analytics Engineer (DP-600) or equivalent Azure certification (DP-203)

Create a job alert for this search

Development Engineer • Bengaluru, Karnataka, India