Databricks Engineer

TTC Groupbangalore district, karnataka, in

2 days ago

Job description

Role : Databricks Engineer

Position : Remote

Type : Contract

Duration : 3 - 6 months (extendable)

Role Overview

We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform. The ideal candidate will be comfortable working with distributed data processing, designing Delta Lake architectures, implementing CI / CD for data workloads, and optimizing Spark jobs for performance and reliability.

Required Skills

Strong hands-on experience in building data pipeline using scripting language includes Python, PySpark and SQL
Strong working experience with Databricks (Unity Catalog is plus) and Azure Data Factory (ADF)
Hands-on working experience with Azure DevOps Repo and CI / CD
Strong understanding of modern Data Architecture, Data Warehousing concepts, Data Modelling and ETL / ELT processes
Understanding of Agile ways of working
Any experience working in DBT is a plus
Working with Retail Domain is a plus

Platform Engineering & Operations

Configure and optimize Databricks clusters, including autoscaling, cluster sizing, and runtime selection.

Manage databases, schemas, and Delta Lake tables, including vacuuming, Z-ordering, partitioning, and optimization.

Work with catalogs, Unity Catalog (UC), and access controls for secure data governance.

Monitor and troubleshoot performance issues across jobs, clusters, and workflows.

CI / CD & DevOps

Implement CI / CD pipelines for Databricks workflows using GitHub Actions / Azure DevOps / GitLab.

Work with Databricks Repos and Git workflows (branching, PRs, code reviews).

Version and promote data pipelines across environments.

Cloud & Integration

Integrate Databricks with cloud services such as :

Azure : ADLS Gen2, Key Vault, ADF, Event Hub, Synapse

AWS : S3, Glue, IAM, Lambda, Kinesis (depending on cloud environment)

Work with data ingestion frameworks, messaging systems, and event-driven architectures.

Quality, Testing & Documentation

Implement unit tests and integration tests using pytest, SQL tests, or dbx testing frameworks.

Maintain pipeline documentation, data dictionaries, and technical design artifacts.

Ensure high data quality, reliability, and observability.

Create a job alert for this search

Engineer • bangalore district, karnataka, in

Related jobs

Promoted

DataOps Engineer

Landmark GroupBengaluru, India

Responsible for the governance ensuring data availability, accuracy, and performance across environments as an expert Azure Data Ops with 6-8 years of experience. Orchestrate, monitor, and automate ...Show moreLast updated: 28 days ago

Promoted

Data Engineer

DigitalzoneBangalore, IN

As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics. Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 18 days ago

Promoted

Data Engineer

Insight GlobalBangalore, IN

GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts. Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

Tata Consultancy ServicesBangalore, IN

Locations : Hyderabad, Kolkata, Noida, Chennai, Bangalore.Designing and building Databricks data pipelines is required.Experience in transforming the data using Azure Databricks & Delta lake.Hands-o...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

MareanaBengaluru, Karnataka, India

Mareana is looking for experienced Data Engineer with a strong development background using, RDMS, graph database and NoSQL platforms. This role is responsible for developing and enhancing Mareana’s...Show moreLast updated: 30+ days ago

Promoted

AWS Data Engineer

Atyeti IncBengaluru, Karnataka, India

Looking for Data Engineer who will be responsible for design, build and maintenance of data pipelines running on Airflow, Spark on the AWS Cloud platform. Roles and Responsibility : - Build and main...Show moreLast updated: 4 days ago

Promoted
New!

Data Bricks Engineer

Relevance LabGreater Bengaluru Area, India

SQL skills and hands-on experience using.AI-assisted development tools).The ideal candidate will be responsible for designing, developing, and optimizing data pipelines, supporting analytics worklo...Show moreLast updated: 17 hours ago

Promoted

Data Engineer

SynechronBengaluru, Karnataka, India

We have immediate opportunity for AWS data engineer.At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innova...Show moreLast updated: 30+ days ago

Promoted
New!

Data Engineer I

LiciousGreater Bengaluru Area, India

SDE 1 – Data Platform Engineer.We are looking for a passionate and motivated.Software Development Engineer (SDE 1) – Data Platform. Data Platform Engineering team at.In this role, you’ll design, bui...Show moreLast updated: 17 hours ago

Promoted

Databricks Engineer

TTC GroupBengaluru, IN

Promoted

Databricks Engineer with D365

GyanSys Inc.Bengaluru, Karnataka, India

Opportunity for Databricks Engineer with D365.Notice Period : Immediate - 30 days.The resource will need to have databricks experience as well as experience with handling D365 data.They will need st...Show moreLast updated: 18 days ago

Promoted

GCP Data Engineer

Xsell Resourceshosur, tamil nadu, in

Seeking a GCP Certified Data Engineer to work remotely from India for our Fortune 5 healthcare client in the US.No notice periods more than 15 days. GCP Google Cloud Professional Data Engineer Certi...Show moreLast updated: 17 days ago

Promoted
New!

Databricks Pyspark

Tata Consultancy ServicesGreater Bengaluru Area, India

Greetings from Tata Consulting Services.TCS is Hiring for Azure Databricks & Pyspark.Location : Bangalore / Chennai / Mumbai / Delhi / Hyderabad. Required Technical Skill - Azure Databricks & Pyspark.Show moreLast updated: 17 hours ago

Promoted

Senior Azure Data Engineer

Tata Consultancy ServicesGreater Bengaluru Area, India

Job Title : Senior Azure Data Engineer.Required Skillset : Azure data factory, data bricks, data lake, automation and performance optimization of ETL. Good Knowledge of Data Brick lakehouse and Azure ...Show moreLast updated: 30+ days ago

Promoted
New!

AWS Data Engineer

Tata Consultancy ServicesGreater Bengaluru Area, India

TCS has always been in the spotlight for being adept in the next big technologies.What we can offer you is a space to explore varied technologies and quench your techie soul.Required Skill Set : AWS...Show moreLast updated: 17 hours ago

Promoted
New!

Platform Engineer

Alp Consulting Ltd.Greater Bengaluru Area, India

Mode of working : Hybrid(3days to office).Job Description : Data Platform Engineer-AWS.The Data Platform team in Avis is a level 3 technology specialist team to roll out new capabilities in to cloud ...Show moreLast updated: 17 hours ago

Promoted
New!

Senior Data Engineer

ArcanaGreater Bengaluru Area, India

Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer.Embark on a journey where your deep-rooted expertise in computer science fundamentals, a...Show moreLast updated: 17 hours ago

Promoted

Databricks Data Engineer Lead – Sustainability Project

Blue Cloud Softech Solutions LimitedBangalore, IN

BCSS is seeking a Databricks Data Engineer to support its enterprise-wide Sustainability initiative.The engineer will be responsible for building data pipelines and models to support product-level ...Show moreLast updated: 4 days ago