This job offer is not available in your country.

▷ (18 / 10 / 2025) Databricks Platform Engineer / Administrator

Digitrix Software LLPIndia

10 hours ago

Job description

Experience : 3 to 6 years

Notice Period : Immediate to 15 days

Location : Bangalore

Work from office

JOB DESCRIPTION

Job Description : Databricks Platform Engineer / Administrator (AWS)

Department : Data Engineering / Cloud Platform / DevOps

About the Role

We are looking for a skilled and proactive Databricks Platform Engineer / Administrator to manage, optimize, and govern our Databricks environment on AWS. The ideal candidate will be responsible for overseeing platform operations, optimizing performance and cost, and ensuring secure and efficient use of Databricks and integrated AWS services.

This role sits at the intersection of platform engineering, data infrastructure, and cost governance, requiring both technical depth and cross-functional collaboration.

Key Responsibilities

Platform Administration

Deploy, configure, and manage Databricks workspaces on AWS.
Set up and maintain AWS infrastructure (VPCs, IAM roles, S3, security groups) to support Databricks operations.
Manage user access, job permissions, cluster configurations, and workspace-level controls.

Cost Optimization

Implement and enforce cluster policies to control resource consumption and prevent cost overruns.

Promote efficient compute usage by enabling job clusters, auto-termination, and spot instance strategies.

Monitor and analyze cost using AWS Cost Explorer, Databricks REST APIs, and internal dashboards.

Optimize storage costs through S3 lifecycle policies, Delta Lake optimization, and table version management.

Performance Tuning

Analyze and tune Spark job performance, including partitioning, caching, and join strategies.

Provide guidance on using Photon runtime, Delta Live Tables, and SQL optimization best practices.

Collaborate with data engineers and analytics teams to improve job execution efficiency and pipeline throughput.

Monitoring & Observability

Set up and manage logging, monitoring, and alerting via CloudWatch, Databricks audit logs, and external tools.

Track metrics such as cluster utilization, job failures, job durations, and resource bottlenecks.

Develop dashboards for real-time visibility into platform usage, performance, and cost.

Security & Governance

Enforce access control through Unity Catalog, IAM roles, and cluster policies.

Ensure compliance with enterprise security policies, data governance standards, and audit requirements.

Prevent data duplication and misuse through strong access boundaries and usage monitoring.

Automation & Documentation

Automate provisioning and configuration using Terraform, Databricks CLI, and REST APIs.

Maintain detailed documentation of cluster policies, platform standards, optimization playbooks, and cost governance guidelines.

Drive platform standardization and advocate for best practices across teams.

Qualifications

Required

3–6+ years of experience managing Databricks on AWS or similar Spark-based platforms.

Strong hands-on experience with AWS services (S3, IAM, VPC, EC2, CloudWatch).

Proficiency in Spark performance tuning and Databricks architecture.

Solid understanding of cost optimization strategies in cloud-based data platforms.

Experience with Databricks cluster policies, job scheduling, and Delta Lake.

Familiarity with Python, SQL, Terraform, and Databricks REST APIs.

Preferred

Experience with Unity Catalog, Delta Live Tables, or MLflow.

Knowledge of AWS cost governance tools (Cost Explorer, Budgets, Trusted Advisor).

Prior work in FinOps, DataOps, or Cloud Center of Excellence (CCoE) roles.

Certifications such as :

Databricks Certified Data Engineer / Admin

AWS Certified Solutions Architect / DevOps Engineer

Create a job alert for this search

Platform Engineer • India

Related jobs

Promoted

GCP Data Engineer

EXLNagpur, IN

Must Have- GCP Data Engineer with Banking / Finance Institutions Experience.Google Cloud Platform (GCP) Engineers.This solution will integrate data from diverse sources including.SAS systems, Excel ...Show moreLast updated: 4 days ago

Promoted

Sr. Software Engineer

IntraEdgeNagpur, IN

Senior Data Engineer – Pyspark | AWS Lambda | AWS Glue | Snowflake.The ideal candidate will be responsible for building and maintaining scalable data pipelines, ensuring seamless data integration a...Show moreLast updated: 18 days ago

Promoted

Contract role -Databricks Engineer

KPG99 INCNagpur, IN

Duration : 12 months with extensions.REQUIRED SKILLS AND EXPERIENCE.Strong hands-on experience with Databricks for data processing and pipeline development. Python for data manipulation and automatio...Show moreLast updated: 1 day ago

Promoted

Database Administrator

Amicon Hub ServicesNagpur, IN

Role : Databricks Administrator (Client : Patch).Strong hands-on experience in Databricks workspace setup & cluster management. User & group administration via Azure AD / IAM / SCIM.Cluster configurat...Show moreLast updated: 2 days ago

Promoted
New!

Lead Architect Engineer (Data)

ConfidentialNagpur, IN

Lead Architect Engineer (Data).We are seeking a highly skilled and experienced Lead Architect Engineer (Data) to play a pivotal leadership role in the development, architecture, and engineering of ...Show moreLast updated: 20 hours ago

Promoted

Data Engineer - Scala

Idyllic ServicesNagpur, IN

Code Optimization , Shell Scripting, Data Engineering & Scala.Big Data tools like Spark, Hadoop & Hive.The Senior Data Engineer will be responsible for designing, developing, and maintaining scalab...Show moreLast updated: 2 days ago

Promoted

Data Engineer

TalogyNagpur, IN

This opportunity is ideal for a determined and proactive individual who has a wide range of skills in a variety of database administration, reporting and dashboarding disciplines.This role requires...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

KeasisNagpur, IN

We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS / Hive / Impala) into Apache Iceberg tables, with dow...Show moreLast updated: 4 days ago

Promoted

Snowflake Data Engineer

Radiant Systems IncNagpur, IN

Position : Snowflake Data Engineer.Need Strong Experience on Snowflake & SQL, ADF (Azure Data Factory), Microsoft MDS (Master Data Service). We are seeking a highly skilled and self-driven Data Engin...Show moreLast updated: 3 days ago

Promoted

PySpark Data Engineer

EXTRAGIGNagpur, IN

Contract Assistant – Data Engineer Support (Remote, EST Hours).PySpark Data Engineer with daily activities.This is a remote contract role. Execute creative software and data solutions, including des...Show moreLast updated: 4 days ago

Promoted

Principal Data Engineer

K&K Talents - IndiaNagpur, IN

This position is with one of our.Title : Principal Data Engineer.Location : Bangalore, Bhopal, Chennai, Gurugram, Hyderabad, Jaipur, Pune. Employment Type : Full-time Permanent.GCP or AWS, Spark, Pyspa...Show moreLast updated: 4 days ago

Promoted

Data Engineer

Manuh TechnologiesNagpur, IN

S3, Glue, Redshift, EMR, Lambda).Develop automation scripts and tools using.Collaborate with data analysts, data scientists, and business stakeholders to ensure data availability and reliability.Tr...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

Magma ConsultancyNagpur, IN

Part-time (20–25 hours per week).The ideal candidate has hands-on experience in building, optimizing, and maintaining data pipelines and architectures. You’ll work closely with analysts, developers,...Show moreLast updated: 2 days ago

Promoted

Databricks Engineer

Tata Consultancy Servicesnagpur, maharashtra, in

Skills : Azure Databricks, Pyspark, ADLS, ADF.Responsibility of / Expectations from the Role : .Designing and building Databricks data pipelines is required. Experience in transforming the data using A...Show moreLast updated: 12 days ago

Promoted

Data Platform Engineer

Intuitive.CloudNagpur, IN

With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization. Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 24 days ago

Promoted

Databricks Engineer

KPG99 INCNagpur, IN

REQUIRED SKILLS AND EXPERIENCE.Strong hands-on experience with Databricks for data processing and pipeline development.Proficiency in SQL for data querying, transformation, and troubleshooting.Soli...Show moreLast updated: 1 day ago

Promoted

Data Engineer

V2Softnagpur, maharashtra, in

Experience to deal with large stream volumes.Implementation experience ( atleast couple of projects ) in MSK / Flink / Spark / Scala. Very good knowledge on atleast 3 out of the 4 technologies.Desirable e...Show moreLast updated: 4 days ago

Promoted

Databricks Data Engineer

Insight GlobalNagpur, IN

Insight Global is seeking a Databricks Data Engineer in India with with 3–5 years of experience to support data engineering initiatives in the pharmaceutical domain. The ideal candidate will have ha...Show moreLast updated: 2 days ago

Promoted

Senior Data Platform Engineer

Black Dog LabsIndia, India

Remote (collaboration across time zones), India or LATAM preferred.Proficient English communication.Data Engineering / Backend Engineering / DevOps. We’re looking for a hands-on Senior Data Platform...Show moreLast updated: 30+ days ago

Promoted
New!

Databricks Engineer

Sapphire Software Solutions IncNagpur, IN

Duration : 12 months with extensions.REQUIRED SKILLS AND EXPERIENCE.Strong hands-on experience with Databricks for data processing and pipeline development. Proficiency in SQL for data querying, tran...Show moreLast updated: 20 hours ago