Databricks Engineer

TTC GroupAurangabad, IN

1 day ago

Job description

Role : Databricks Engineer

Position : Remote

Type : Contract

Duration : 3 - 6 months (extendable)

Role Overview

We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform. The ideal candidate will be comfortable working with distributed data processing, designing Delta Lake architectures, implementing CI / CD for data workloads, and optimizing Spark jobs for performance and reliability.

Required Skills

Strong hands-on experience in building data pipeline using scripting language includes Python, PySpark and SQL
Strong working experience with Databricks (Unity Catalog is plus) and Azure Data Factory (ADF)
Hands-on working experience with Azure DevOps Repo and CI / CD
Strong understanding of modern Data Architecture, Data Warehousing concepts, Data Modelling and ETL / ELT processes
Understanding of Agile ways of working
Any experience working in DBT is a plus
Working with Retail Domain is a plus

Platform Engineering & Operations

Configure and optimize Databricks clusters, including autoscaling, cluster sizing, and runtime selection.

Manage databases, schemas, and Delta Lake tables, including vacuuming, Z-ordering, partitioning, and optimization.

Work with catalogs, Unity Catalog (UC), and access controls for secure data governance.

Monitor and troubleshoot performance issues across jobs, clusters, and workflows.

CI / CD & DevOps

Implement CI / CD pipelines for Databricks workflows using GitHub Actions / Azure DevOps / GitLab.

Work with Databricks Repos and Git workflows (branching, PRs, code reviews).

Version and promote data pipelines across environments.

Cloud & Integration

Integrate Databricks with cloud services such as :

Azure : ADLS Gen2, Key Vault, ADF, Event Hub, Synapse

AWS : S3, Glue, IAM, Lambda, Kinesis (depending on cloud environment)

Work with data ingestion frameworks, messaging systems, and event-driven architectures.

Quality, Testing & Documentation

Implement unit tests and integration tests using pytest, SQL tests, or dbx testing frameworks.

Maintain pipeline documentation, data dictionaries, and technical design artifacts.

Ensure high data quality, reliability, and observability.

Create a job alert for this search

Engineer • Aurangabad, IN

Related jobs

Promoted

Data Engineer

Insight GlobalAurangabad, IN

GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts. Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago

Promoted

Senior Azure Data Engineer

Tata Consultancy ServicesAurangabad, IN

Job Title : Senior Azure Data Engineer.Required Skillset : Azure data factory, data bricks, data lake, automation and performance optimization of ETL. Good Knowledge of Data Brick lakehouse and Azure ...Show moreLast updated: 1 day ago

Promoted

Data Engineer

K&K Talents - IndiaAurangabad, IN

This position is with one of our clients in.Location : Chennai ,Bangalore , Hyderabad , Gurgaon , Bhopal, Pune,Jaipur.Experience in data engineering, with a focus on data architecture and pipeline d...Show moreLast updated: 30+ days ago

Promoted

Azure Databricks

Kumaran SystemsAurangabad, IN

We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault. The candidate should be proficient in Dat...Show moreLast updated: 3 days ago

Promoted

Data Engineer

NexionPro ServicesAurangabad, IN

PySpark-based data processing workflows.Collaborate with data architects, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.Optimi...Show moreLast updated: 1 day ago

Promoted

Senior DataOps Engineer (AWS)

MSBC GroupAurangabad, IN

Join us as a Senior DataOps Engineer (AWS)—Drive High-Performance Data Systems for Financial Services.Lead the E-Comms data pipeline within Compass’s Application Simplification workstream : design, ...Show moreLast updated: 1 day ago

Promoted

Data Engineer

DigitalzoneAurangabad, IN

As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics. Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 17 days ago

Promoted

Data Engineer | Bangalore | Leading Big 4 Consulting Firm

Acme ServicesAurangabad, IN

Design, develop, and maintain scalable and robust data pipelines (ETL / ELT) using PySpark on the Databricks platform.Implement data ingestion strategies to load data from various sources (e.APIs, da...Show moreLast updated: 1 day ago

Promoted

Data Engineer

RecroAurangabad, IN

Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake. AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

MyData Insights Pvt LtdAurangabad, IN

The ideal candidate has a strong background in building scalable data solutions on.CI / CD pipelines, infrastructure automation using Terraform, and cloud-native data engineering practices.Design, bu...Show moreLast updated: 1 day ago

Promoted

AWS data engineer

Tata Consultancy ServicesAurangabad, IN

TCS is looking for AWS data engineer.Location : Kolkata, Hyderabad, Bangalore, Chennai, Pune, Gurgaon.Strong hands-on experience with AWS Data Services : . Amazon S3, Glue, Redshift, Athena, Kinesis, E...Show moreLast updated: 1 day ago

Promoted

Databricks Gen AI Engineer

SyrenAurangabad, IN

Model Serving, Vector Search, and embedding workflows.Clustering, Unity Catalog, Delta Lake).OpenAI / Azure OpenAI), and GenAI app patterns (RAG / Agents). Proficiency in SQL, Spark performance tuning...Show moreLast updated: 1 day ago

Promoted

Data Engineer

Ubique SystemsAurangabad, IN

Primary skills : Python, SQL, data lakes, azure.Pipeline Development & Automation.Design, build, and maintain CI / CD pipelines to automate deployment of DQ rules and data services across environments...Show moreLast updated: 30+ days ago

Promoted

(Senior) Azure Data Engineer - Full remote - contractor in USD

All European CareersAurangabad, IN

Remote

For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databric...Show moreLast updated: 3 days ago

Promoted

Principal Data Engineer

KPI PartnersAurangabad, IN

We are seeking a highly skilled Principal Data Engineer to join our dynamic team at KPI Partners.Lead data engineering initiatives and projects from conception to delivery.Design and implement scal...Show moreLast updated: 1 day ago

Promoted

Azure Databricks+Pyspark

Tata Consultancy ServicesAurangabad, IN

TCS is conducting Virtual Drive for Azure Data Engineer + Pyspark.Years of Experience : 6 to 10yrs.Location : Bangalore & Hyderabad. Kindly apply if there is a skill match with 6+yrs of experience.Ove...Show moreLast updated: 17 days ago

Promoted

Data Engineer

JIT Global Infosystems Pvt. Ltd.Aurangabad, IN

Applications, Analytics, and Consulting services to clients worldwide.By combining technological expertise with deep domain knowledge, we ensure tangible improvements in business performance.Our ex...Show moreLast updated: 1 day ago

Promoted

GCP Data Engineer

AdastraAurangabad, IN

We are looking for a proactive and solution-oriented GCP Data Engineer to join our team.This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, ...Show moreLast updated: 4 days ago