Senior Azure Data Engineer Exp - Interviewing Today - Urgent Needs

Accion LabsPune (division)

1 day ago

Job description

Role Snapshot

Title : Senior Azure Data Engineer
Experience : 6–8+ years in Data Engineering ( minimum 4+ years on Azure , and ( Or ) 6 months to 1+ year with Microsoft Fabric )
Tech Focus : Microsoft Fabric, Azure Data Factory (ADF), Databricks (Python, PySpark, Spark SQL), Delta Lake, Power BI (DAX), Azure Storage, Lakehouse, Warehouse
Engagement : Client-facing, hands-on, design-to-delivery
Location : Any Accion Labs offices in India - Bangalore, Pune (Preferred), Mumbai, Hyderabad, Indore, Noida - THIS IS A HYBRID Work Model , NOT Remote.
Notice period : Preferred Immediate joiners or candidates who can join within 20 days are needed

Core Responsibilities

End-to-End Engineering

Design, implement, and deliver batch & streaming data pipelines into Fabric Lakehouse / Warehouse using ADF and Databricks with Delta Lake .

Data Architecture Understanding

Strong grasp of Bronze–Silver–Gold layering , incremental ingestion , watermarking , and best practices for scalable pipelines.

Medallion Architecture

Apply Bronze / Silver / Gold patterns , enforce schema evolution , handle late / dirty data , and implement SCD (Type 1 / 2) and late-arriving dimensions .

Fabric Platform & Security

Build solutions on Microsoft Fabric (OneLake, Lakehouse, Warehouse, Pipelines, Dataflows Gen2, Notebooks) .

Implement security layers : workspace & item permissions, RLS / OLS in Warehouse / Lakehouse SQL endpoints, credentialed connections / shortcuts to external storage, environments & capacities alignment.

ADF Orchestration & Reusability

Create parameterized, template-driven pipelines with reusable activities (ForEach, Lookup, Mapping Data Flows).

Ensure robust dependency management with retry / alert patterns.

Databricks Engineering Excellence

Author complex & nested notebooks (via %run / dbutils.notebook.run) in Python, PySpark, and Spark SQL for ETL / ELT.

Debug & troubleshoot jobs and clusters; resolve skew, shuffle spills, checkpoint failures, schema drift, streaming backlogs .

Apply performance optimizations : partitioning & clustering, Z-ORDER, OPTIMIZE / VACUUM, file size tuning, AQE, broadcast joins, caching, checkpoint & trigger strategies for Structured Streaming.

Data Quality, Observability & Reliability

Implement data quality checks (validations, expectations), idempotency, exactly-once / at-least-once semantics, and dead-letter flows .

Set up monitoring & logging (Azure Monitor / Log Analytics, Databricks system tables, Fabric monitoring), with alerting & dashboards.

SQL

Strong understanding of MS SQL concepts ; hands-on experience in writing functions and stored procedures , along with DDL / DML operations .

Design, Documentation & Governance

Contribute to data models (star / snowflake), semantic layers, dimensional design, and documentation (solution design docs, runbooks).

CI / CD & ADO Versioning Management

Implement branching strategy (Git / ADO) , perform PR reviews , manage environment promotion (Dev / Test / Prod), and support Fabric CI / CD process .

Leadership & Client Engagement

Mentor junior engineers; enforce reusable & scalable patterns .

Run client demos and brainstorming discussions .

Be self-driven and innovative in solution delivery.

Must-Have Skills (Strong, Hands-On)

Microsoft Fabric (2024+)

OneLake, Lakehouse, Warehouse, Pipelines, Dataflows Gen2, Notebooks, capacities, workspace & item security, RLS / OLS.

Azure Data Factory (ADF)

Reusable, parameterized pipelines; high-level orchestration; robust scheduling, logging, retries, and alerts.

Databricks (5+ years on Azure)

Python, PySpark, Spark SQL : complex transformations, joins, window functions, UDFs / UDAs.

Complex & nested notebooks; modular code with %run / dbutils.notebook.run.

Structured Streaming : watermarks, triggers, checkpointing, foreachBatch, schema evolution.

Delta Lake : Z-ORDER, OPTIMIZE / VACUUM, MERGE for SCD, Auto Optimize, compaction, time travel.

Performance tuning : partitioning, file sizing, broadcast hints, caching, Photon (where available), cluster sizing / pools.

Medallion Architecture

Bronze / Silver / Gold patterns, SCD (Type 1 / 2), handling late-arriving dimensions.

Azure Storage

ADLS Gen2 (hierarchical namespace), tiering (Hot / Cool / Archive), lifecycle & cost optimization, shortcuts into OneLake.

Data Warehousing

Dimensional modeling, fact / aggregate design, query performance tuning in Fabric Warehouse & Lakehouse SQL endpoint.

SQL

Excellent SQL development; advanced joins, windowing, CTEs, performance tuning / indexing where applicable.

Power BI (DAX)

Awareness of Power BI and DAX; RLS alignment with Warehouse / Lakehouse.

Security & Compliance

RBAC, item-level permissions, credentials for data sources, RLS / OLS, secret management (Key Vault), PII handling.

ETL / ELT Methodologies

Robust, testable pipelines; idempotency; error handling; data quality gates.

Ways of Working

Agile delivery, client-facing communication, crisp demos, documentation, and best-practice advocacy.

If interested or know anyone, Kindly write to me at shruti.saboo@accionlabs.com along with your latest CV.

Thank You,

Shruti Saboo

Create a job alert for this search

Azure Data Engineer • Pune (division)

Related jobs

Promoted

Senior Data Engineer( DWH)

MastercardPune (division)

Mastercard is a global technology company in the payments industry.Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, ...Show moreLast updated: 1 day ago

Promoted

Data Solution Engineer I II and III

OpenGov Inc.Pune (division)

OpenGov is the leader in AI and ERP solutions for local and state governments in the U.More than 2,000 cities, counties, state agencies, school districts, and special districts rely on the OpenGov ...Show moreLast updated: 1 day ago

Promoted

Azure Data Architect

Programmers.ioPune (division)

Job Title : Azure Data Architect.Location : Hyderabad, pune (hybrid).Experience required : 12+ years.We are seeking a highly experienced S. The ideal candidate should bring strong expertise in SQL, ETL...Show moreLast updated: 1 day ago

Promoted

Data Engineer

Persistent SystemsPune (division)

We are looking for Strong Data engineers to join the Data Integration team and become part of our successful group.Location : All Persistent Locations. Job Type : Full Time Employment.Design, code, te...Show moreLast updated: 10 days ago

Promoted

Senior Data Engineer- AWS

InfoCeptsPune (division)

Key Result Areas and Activities : .Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements. Should be able to design comple...Show moreLast updated: 1 day ago

Promoted

Cloud Data Engineer

Persistent SystemsPune (division)

We are seeking a highly analytical and detail-oriented Cloud Data Engineer.Location : All Persistent Locations.Job Type : Full Time Employment. Minimum of 5+ years of experience as a Cloud Data Engine...Show moreLast updated: 10 days ago

Promoted

Cloud Data Platform Engineer

Persistent SystemsPune (division)

We are seeking a highly skilled and experienced Data Engineer – SQL / SSIS to join the Optum Advisory Team.The ideal candidate will bring deep expertise in cloud-based data engineering, healthcare ...Show moreLast updated: 9 days ago

Promoted

Azure Data engineer

LTIMindtreePune (division)

Relevant into azure services : 5yrs.Notice period : imm to serving NP.Show moreLast updated: 10 days ago

Promoted

Data Engineer

thinkbridgePune (division)

We are a global digital product development firm that helps growth-stage companies gain the technology sophistication and maturity of leading modern digital businesses. We differentiate ourselves by...Show moreLast updated: 10 days ago

Promoted

Senior Data Platform Engineer

Persistent SystemsPune (division)

We are seeking a Senior Platform Engineer to architect, build, and manage enterprise-scale data platforms that power advanced analytics and AI initiatives. This role blends cloud engineering, DevOps...Show moreLast updated: 7 days ago

Promoted

5447 | Senior Data Engineer

EXLPune (division)

We are seeking a Senior Data Engineer to build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging.You will...Show moreLast updated: 1 day ago

Promoted

Data Engineer Lead

Calfus Inc.Pune (division)

As a Data Engineer – BI Analytics & DWH, you will play a pivotal role in designing and implementing comprehensive business intelligence solutions that empower our organization to make data-driven d...Show moreLast updated: 10 days ago

Promoted

Data Engineer

SynechronPune (division)

We have immediate opportunity for.Notice Period : Immediate to 30 Days.At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines cre...Show moreLast updated: 1 day ago

Promoted

Senior Data Engineer

Persistent SystemsPune (division)

We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services. The ideal candidate will have a strong backgro...Show moreLast updated: 10 days ago

Promoted

Data Engineer (PySpark)

RSquareSoft TechnologiesPune (division)

Design, develop, and maintain scalable.Work with large datasets using.S3, Glue, Lambda, and Redshift.Collaborate with cross-functional teams to define data requirements and optimize data flows.Ensu...Show moreLast updated: 1 day ago

Promoted

5676 - Data Engineer

EXLPune (division)

Location - Pune, Bangalore, Noida, Gurgaon, Hyderabad.The ideal candidate will have strong expertise in Snowflake, Hadoop ecosystem, PySpark, and SQL, and will play a key role in enabling data-driv...Show moreLast updated: 1 day ago

Promoted

Senior DevOps Enginer

EXLPune (division)

Responsible for building and managing our cloud infrastructure.Performance tune data pipelines using technologies such as. Write and maintain high-quality code and participate in design discussions ...Show moreLast updated: 10 days ago

Promoted

Senior Engineer

eGain CorporationPune (division)

Fortune 500 clients and government agencies trust eGain AI knowledge solution to improve customer experience and reduce cost of service. Top rated by Gartner, eGain AI Knowledge Hub orchestrates AI ...Show moreLast updated: 10 days ago