Key Responsibilities :
Design, develop, and optimize end-to-end data pipelines using Databricks, PySpark, Delta Lake, and Azure components.
Build new pipelines from scratch and enhance existing ones for scalability and performance.
Apply ETL design principles, data modeling, and governance standards.
Collaborate in an agile / scrum environment and deliver production-ready code.
Work with Azure services like ADLS, ADF, and Synapse.
Troubleshoot, debug, and optimize existing data pipelines.
Technical Expertise :
5–8 years of strong experience in Data Engineering (Databricks + Azure)
Deep understanding of Databricks Lakehouse Architecture, Apache Spark Architecture, and Data Warehousing
Knowledge of infra provisioning, compute cluster configuration, and workflow management in Databricks
Proficiency in PySpark, SQL, Delta Lake, and Unity Catalog
Familiarity with Git, CI / CD, and data governance practices
Senior Developer • guwahati, assam, in