Description : Required : 7+ Years.
Work Location : Bangalore / Chennai / Hyderabad.
Role Overview :
We are seeking an experienced Databricks Administrator to manage, optimize, and maintain our Databricks platform across multiple environments.
The ideal candidate will have a strong background in data engineering infrastructure, cluster management, security, governance, and automation within Databricks and related Azure / AWS ecosystems.
Key Responsibilities :
Databricks Platform Administration :
- Manage and maintain Databricks workspaces, clusters, jobs, pools, and libraries.
- Monitor performance and optimize resource utilization across environments (dev, test, prod).
- Implement autoscaling, cluster policies, and cost control measures.
Security & Access Management :
Configure and maintain user / group access via Azure AD or IAM integration.Implement fine-grained access control using Unity Catalog and role-based permissions.Ensure compliance with enterprise data security standards.Automation & CI / CD :
Automate cluster provisioning, job deployment, and workspace management using Terraform, Databricks CLI, or REST APIs.Integrate Databricks with CI / CD tools like Azure DevOps, Jenkins, or GitHub Actions.Monitoring & Troubleshooting :
Proactively monitor cluster health, job performance, and data pipeline stability.Diagnose and resolve system, configuration, and performance issues.Implement alerting, logging, and audit trail monitoring.Integration & Collaboration :
Work with Data Engineers, DevOps, and Cloud teams to ensure smooth data operations.Support integration with Snowflake, Kafka, Azure Data Lake, or AWS S3.Collaborate on optimizing Spark configurations and ETL performance.Governance & Compliance :
Implement best practices for workspace and notebook governance.Manage version control for notebooks and data pipelines.Enforce tagging, cost allocation, and audit standards.Required Skills & Qualifications :
Bachelors degree in Computer Science, Engineering, or related field.7+ years of experience in Data Engineering or Platform Administration, with at least 3+ years in Databricks Administration.Strong understanding of Apache Spark architecture and performance tuning.Hands-on experience with Azure Databricks or AWS Databricks environments.Proficiency with Terraform, Databricks CLI, REST APIs, and CI / CD pipelines.Expertise in cluster configuration, job scheduling, and resource optimization.Familiarity with Unity Catalog, Delta Lake, DBFS, and Lakehouse architecture.Experience integrating Databricks with ADLS, S3, Snowflake, Synapse, or Redshift.Strong problem-solving and scripting skills (Python, Shell, or PowerShell).(ref : hirist.tech)