Azure Databricks Engineer
Location : PAN India (Hybrid)
Experience : 6-9 Years
Must-Have :
- Experience with Azure Databricks , PySpark Notebooks , Spark SQL and Unity Catalog.
- Data Modelling experience with strong SQL knowledge.
Key Responsibilities :
Develop, maintain, and optimize PySpark notebooks on Azure Databricks for data processing and analytics pipelines.Implement and manage Spark Structured Streaming solutions for real-time data processing.Work extensively with Spark SQL and Unity Catalog to manage and govern enterprise datasets.Design and implement robust data models supporting efficient querying and data integrity.Write complex SQL queries and optimize them for performance and scalability.Collaborate with cross-functional teams including Data Scientists, Analysts, and Architects to understand data requirements.Must-Have Skills :
Hands-on experience with Azure Databricks, PySpark, and Spark SQLProficiency in Spark Structured StreamingExperience with Unity Catalog for data governanceStrong SQL skills for data extraction, transformation, and analysisSolid understanding of Data Modeling principlesGood to Have :
Experience in CI / CD pipelines for DatabricksKnowledge of Delta Lake, Data Lakehouse conceptsFamiliarity with Azure Data Factory or other orchestration tools(ref : hirist.tech)