Job Title : Databricks Lead
Location : Chennai / Hyderabad / Bangalore
Experience : 8-10 years of overall experience and 5+ years of hands-on leadership in data engineering projects
Budget : Open to discuss
Notice Period : Immediate joiner / Serving notice with less than 60 days / Notice is less than 60 days
About the Role
We are looking for a highly skilled Databricks Lead with 8–10 years of overall experience and 5+ years of hands-on leadership in data engineering projects. The ideal candidate will lead teams in designing, building, and optimizing scalable data pipelines and Lakehouse solutions on Databricks.
Key Responsibilities
- Lead the development and delivery of end-to-end data pipelines using Databricks , Spark, Delta Lake, and PySpark.
- Drive implementation of Lakehouse / Medallion architectures (Bronze / Silver / Gold).
- Guide and mentor data engineers, ensure best coding and architectural practices.
- Optimize Spark jobs, Databricks clusters, and workflows for performance and cost.
- Implement CI / CD pipelines, version control, and DevOps practices for Databricks (Repos, Workflows, Terraform).
- Collaborate with architects, data scientists, and business teams to deliver scalable data solutions.
- Ensure data quality, governance, security, and compliance standards are followed (Unity Catalog preferred).
Required Skills & Experience
8–10 years in Data Engineering or Cloud Data Platforms.5+ years leading data engineering teams or technical delivery.Strong hands-on experience with Databricks , Spark, PySpark, SQL, and Delta Lake.Experience with cloud platforms (AWS / Azure / GCP) and related data services.Solid understanding of data modeling, ingestion frameworks, orchestration, and performance tuning.Experience with CI / CD, Git, Terraform, Databricks Repos, and Workflows.Strong problem-solving, leadership, and communication skills.Preferred Qualifications
Databricks certifications (Associate or Professional).Experience with streaming technologies (Kafka, Kinesis, Event Hubs).Familiarity with MLflow and data science integration.What We Offer
Opportunity to lead cutting-edge data engineering initiatives.Collaborative, high-growth environment.Competitive compensation and professional development opportunities.