No longer accepting applications
Cloud Data Platform Engineer
Digitrix Software LLPBengaluru, Republic Of India, IN30+ days ago
Job descriptionImplement and enforce cluster policies to control resource consumption and prevent cost overruns. Promote efficient compute usage by enabling job clusters , auto-termination , and spot instance strategies. Monitor and analyze cost using AWS Cost Explorer , Databricks REST APIs , and internal dashboards. Optimize storage costs through S3 lifecycle policies , Delta Lake optimization , and table version management . Analyze and tune Spark job performance , including partitioning, caching, and join strategies. Provide guidance on using Photon runtime , Delta Live Tables , and SQL optimization best practices . Collaborate with data engineers and analytics teams to improve job execution efficiency and pipeline throughput. Set up and manage logging, monitoring, and alerting via CloudWatch, Databricks audit logs, and external tools. Track metrics such as cluster utilization, job failures, job durations, and resource bottlenecks. Develop dashboards for real-time visibility into platform usage, performance, and cost. Enforce access control through Unity Catalog , IAM roles , and cluster policies . Ensure compliance with enterprise security policies, data governance standards, and audit requirements. Prevent data duplication and misuse through strong access boundaries and usage monitoring. Automate provisioning and configuration using Terraform , Databricks CLI , and REST APIs . Maintain detailed documentation of cluster policies, platform standards, optimization playbooks, and cost governance guidelines. Drive platform standardization and advocate for best practices across teams. 3–6+ years of experience managing Databricks on AWS or similar Spark-based platforms. Strong hands-on experience with AWS services (S3, IAM, VPC, EC2, CloudWatch). Proficiency in Spark performance tuning and Databricks architecture . Solid understanding of cost optimization strategies in cloud-based data platforms. Experience with Databricks cluster policies , job scheduling , and Delta Lake . Familiarity with Python, SQL, Terraform , and Databricks REST APIs . Experience with Unity Catalog , Delta Live Tables , or MLflow . Knowledge of AWS cost governance tools (Cost Explorer, Budgets, Trusted Advisor). Prior work in FinOps, DataOps, or Cloud Center of Excellence (CCoE) roles. Certifications such as : Databricks Certified Data Engineer / Admin AWS Certified Solutions Architect / DevOps Engineer
Experience : 3 to 6 years
Notice Period : Immediate to 15 days
Location : Bangalore
Work from office
JOB DESCRIPTION
Job Description : Databricks Platform Engineer / Administrator (AWS)
Department : Data Engineering / Cloud Platform / DevOps
About the Role
We are looking for a skilled and proactive Databricks Platform Engineer / Administrator to manage, optimize, and govern our Databricks environment on AWS. The ideal candidate will be responsible for overseeing platform operations, optimizing performance and cost, and ensuring secure and efficient use of Databricks and integrated AWS services.
This role sits at the intersection of platform engineering, data infrastructure, and cost governance, requiring both technical depth and cross-functional collaboration.
Key Responsibilities
Platform Administration
- Deploy, configure, and manage Databricks workspaces on AWS.
- Set up and maintain AWS infrastructure (VPCs, IAM roles, S3, security groups) to support Databricks operations.
- Manage user access, job permissions, cluster configurations, and workspace-level controls.
Cost Optimization
Performance Tuning
Monitoring & Observability
Security & Governance
Automation & Documentation
Qualifications
Required
Preferred
Create a job alert for this search
Data Engineer Platform • Bengaluru, Republic Of India, IN
Related jobs
- Promoted
Senior AWS Data Engineer
CYAN360hosur, tamil nadu, in Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 30+ days ago
- Promoted
Data Platform Engineer
Grizmo LabsBengaluru, Karnataka, India Design, build, and maintain scalable data platforms and pipelines to support the needs of the business.Ensure data quality and accuracy by implementing data validation and testing processes.Partici...Show moreLast updated: 30+ days ago
- Promoted
AWS Cloud Engineer
Proglitehosur, tamil nadu, in Infrastructure & System Administration : .Deploy, manage, and optimize EC2 instances across dev, test, and production environments.
Perform system administration and troubleshooting for Linux and Wind...Show moreLast updated: 30+ days ago
- Promoted
Cloud Engineer
Landmark GroupBengaluru, Karnataka, India Position Title - Data Operations Engineer.Bahrain and has grown into one of the largest retail and hospitality conglomerates in the.
Currently the Group operates over.In this role, you will streamli...Show moreLast updated: 7 days ago
- Promoted
Senior Data Platform Engineer
Black Dog Labshosur, tamil nadu, in Remote (collaboration across time zones), India or LATAM preferred.Proficient English communication.Data Engineering / Backend Engineering / DevOps.
We’re looking for a hands-on Senior Data Platform...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Insight Globalhosur, tamil nadu, in The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-sca...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Recrohosur, tamil nadu, in Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Cloud Engineer
SysTechCorp Inchosur, tamil nadu, in Experience in AWS cloud – Lambda based microservices.GitHub CICD experience (GitHub Actions is a plus).Go & Python programming language.
Experience in building & managing CICD pipelines for Kubernet...Show moreLast updated: 9 days ago
- Promoted
Data Engineer
Digitalzonehosur, tamil nadu, in As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 9 days ago
- Promoted
Cloud Engineer
Tata Consultancy ServicesBengaluru, Karnataka, India Good to have Docker and Kubernetes knowledge for troubleshooting clusters and managing container images.Good to have CI / CD deployment knowledge, GitHub Workflows, YAML Pipelines.Good Knowledge of A...Show moreLast updated: 30+ days ago
- Promoted
Cloud Data Engineer
Lemongrasshosur, tamil nadu, in Lemongrass is a software-enabled services provider, synonymous with SAP on Cloud, focused on delivering superior, highly automated Managed Services to Enterprise customers.Our customers span multip...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
Tata Consultancy Serviceshosur, tamil nadu, in Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 17 days ago
- Promoted
Lead - Cloud Reliability Engineer
Searce Inchosur, tamil nadu, in The ‘process-first’ AI-native modern tech consultancy that's rewriting the rules.As an engineering-led consultancy, we are dedicated to relentlessly improving the real business outcomes.Our solvers...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Insight Globalhosur, tamil nadu, in GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago
- Promoted
- New!
Cloud Engineer(AWS)
Sutantra Systems Private Limitedhosur, tamil nadu, in We are looking for an experienced AWS Cloud Engineer with 5+ years of hands-on experience in designing, developing, and maintaining scalable cloud-based solutions on Amazon Web Services (AWS).You w...Show moreLast updated: 20 hours ago
- Promoted
Senior Data Engineer
Mastekhosur, tamil nadu, in We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Amicon Hub Serviceshosur, tamil nadu, in Required Skills & Qualifications.Delta Lake, Spark, PySpark, SQL).SQL Server, MongoDB, InfluxDB).Kafka, Azure Event Hubs, or similar).
Excellent problem-solving skills and the ability to work in a f...Show moreLast updated: 29 days ago
- Promoted
Data Engineer
Dexian Indiahosur, tamil nadu, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 9 days ago