Data Administrator

Experience Required : 7+ Years

Work Location : Bangalore / Chennai / Hyderabad

Role Overview :

We are seeking an experienced Databricks Administrator to manage, optimize, and maintain our Databricks platform across multiple environments. The ideal candidate will have a strong background in data engineering infrastructure, cluster management, security, governance, and automation within Databricks and related Azure / AWS ecosystems.

Key Responsibilities :

Databricks Platform Administration :

Manage and maintain Databricks workspaces, clusters, jobs, pools, and libraries.

Monitor performance and optimize resource utilization across environments (dev, test, prod).

Implement autoscaling, cluster policies, and cost control measures.

Security & Access Management :

Configure and maintain user / group access via Azure AD or IAM integration.

Implement fine-grained access control using Unity Catalog and role-based permissions.

Ensure compliance with enterprise data security standards.

Automation & CI / CD :

Automate cluster provisioning, job deployment, and workspace management using Terraform, Databricks CLI, or REST APIs.

Integrate Databricks with CI / CD tools like Azure DevOps, Jenkins, or GitHub Actions.

Monitoring & Troubleshooting :

Proactively monitor cluster health, job performance, and data pipeline stability.

Diagnose and resolve system, configuration, and performance issues.

Implement alerting, logging, and audit trail monitoring.

Integration & Collaboration :

Work with Data Engineers, DevOps, and Cloud teams to ensure smooth data operations.

Support integration with Snowflake, Kafka, Azure Data Lake, or AWS S3.

Collaborate on optimizing Spark configurations and ETL performance.

Governance & Compliance :

Implement best practices for workspace and notebook governance.

Manage version control for notebooks and data pipelines.

Enforce tagging, cost allocation, and audit standards.

Required Skills & Qualifications :

Bachelor’s degree in Computer Science, Engineering, or related field.

7+ years of experience in Data Engineering or Platform Administration, with at least 3+ years in Databricks Administration .

Strong understanding of Apache Spark architecture and performance tuning.

Hands-on experience with Azure Databricks or AWS Databricks environments.

Proficiency with Terraform , Databricks CLI , REST APIs , and CI / CD pipelines .

Expertise in cluster configuration, job scheduling, and resource optimization .

Familiarity with Unity Catalog , Delta Lake , DBFS , and Lakehouse architecture .

Experience integrating Databricks with ADLS, S3, Snowflake, Synapse, or Redshift .

Strong problem-solving and scripting skills (Python, Shell, or PowerShell).