SME – Application Support (Databricks)

Job Title : SME – Application Support (Databricks)

Location : Ghansoli, Mumbai

Work Mode : Onsite

Experience : 6–8 Years

Budget : Up to 27 LPA

Shift : 16×5 Rotational (Morning / Afternoon / General)

Weekend & Night support on need basis

About the Role

We are hiring an experienced SME – Application Support (Databricks) to join our Big Data & Cloud Platform Support team. The ideal candidate will bring strong expertise in Databricks administration, Spark debugging, and Azure cloud support to ensure reliable and high-performing data platform operations.

Key Responsibilities

Act as First-Level Escalation for 24×7 monitoring of Databricks clusters, jobs, workflows, repos, and data pipelines.

Troubleshoot and analyze issues related to :

Cluster failures / auto-scaling problems

Job failures (PySpark, Scala, Spark SQL, DLT)

Workspace & availability issues

Work closely with application development teams to remediate pipeline failures.

Participate in Sev1 / Sev2 incident resolution and prepare RCA reports.

Implement workspace governance, RBAC, cluster policies, and data security best practices.

Ensure compliance with audit and platform governance requirements.

Build dashboards / logging for job performance, cluster utilization, and failure analytics.

Maintain SOPs, runbooks, and architecture documentation.

Identify recurring issues and collaborate with L3 / platform engineering teams.

Debug complex Spark issues (OOM, long GC cycles, driver / executor failures).

Required Skills

6–10 years of experience in Big Data / Cloud Data Platform Support .

Strong SME-level experience with Databricks (clusters, jobs, repos, MLflow, warehouse).

Expertise in UNIX, SQL, Shell Scripting , and Azure Cloud .

Strong knowledge of Apache Spark and Spark UI / JOB debugging .

Experience with CI / CD pipelines using Azure DevOps .

Educational Qualification

B.E / B.Tech / MCA