Lead Software Engineer – Databricks

Mandatory Skills:

• Databricks
• PySpark
• Spark SQL
• Delta Lake
• Unity Catalog
• SQL Warehouse
• Lakehouse Architecture
• ETL Pipeline Development
• Azure/AWS Cloud
• Data Governance
• CI/CD
• Data Lineage & Access Governance

Good to Have:

• Databricks Certification
• MLflow
• Banking & Financial Services Domain
• Agile Methodology

Job Responsibilities:

• Design and implement scalable Lakehouse architecture.
• Develop ETL pipelines using PySpark and Spark SQL.
• Build solutions on Databricks with Delta Lake.
• Implement Unity Catalog, Data Governance, and Data Lineage.
• Integrate Databricks with Azure/AWS services.
• Ensure data quality, observability, and FinOps monitoring.

Skills Required
Databricks PySpark Spark SQL Delta Lake Unity Catalog SQL Warehouse Lakehouse Architecture ETL Pipeline Development Azure/AWS Cloud Data Governance CI/CD Data Lineage & Access Governance