Lead Databricks Analyst

Location : Remote

Employment Type : Full-Time

Overview

Rialtes is seeking a Databricks Support Analyst to manage platform operations, ensure reliability of clusters and jobs, and support development of data workflows. Role requires hands-on experience with Databricks administration, SQL-based development, and basic Python.

Key Responsibilities

Primary : Platform Support

Monitor and maintain Databricks clusters, jobs, workflows, and schedules.

Troubleshoot issues related to compute resources, job failures, data ingestion completeness, schema changes, and data accuracy.

Manage user provisioning, access controls, workspace permissions, and catalogs using Unity Catalog aligned with governance standards.

Ensure cost-optimized cluster usage and enforce security, tagging, and compliance policies.

Collaborate with Cloud Ops, Data Engineering, and InfoSec teams to resolve platform-wide incidents.

Secondary : Design & Development

Assist in building Databricks workflows for ETL, analytics, and machine learning pipelines.

Support development and optimization of notebooks and scripts—primarily SQL, with some Python for transformations and validations.

Implement best practices for job cluster configuration, Photon acceleration, Delta Lake, and Unity Catalog integration.

Participate in unit testing, functional testing, UAT cycles, and production migration of Databricks jobs.

Contribute to documentation including runbooks, SOPs, job catalogs, and data lineage.

Qualifications

5 years of experience in Databricks, data engineering support, or cloud platform operations.

Strong SQL skills; working knowledge of Python preferred.

Hands-on experience with Databricks clusters, job orchestration, Delta Lake, and Unity Catalog.

Understanding of ETL workflows, CI / CD for data pipelines, and cloud environments (AWS / Azure / GCP).

Strong debugging skills for job failures, performance bottlenecks, and ingestion issues.

Ability to work in fast-paced environments with cross-functional teams.

Nice-to-Have

Databricks Certified Associate or equivalent.

Exposure to MLflow, Data Quality (DQ) frameworks, or Databricks Workflows orchestration.

Experience with Git, DevOps tools, or monitoring solutions (CloudWatch, Azure Monitor, etc.).