Databricks Engineer

TTC GroupPrayagraj(Allahabad), IN

21 hours ago

Job description

Role : Databricks Engineer

Position : Remote

Type : Contract

Duration : 3 - 6 months (extendable)

Role Overview

We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform. The ideal candidate will be comfortable working with distributed data processing, designing Delta Lake architectures, implementing CI / CD for data workloads, and optimizing Spark jobs for performance and reliability.

Required Skills

Strong hands-on experience in building data pipeline using scripting language includes Python, PySpark and SQL
Strong working experience with Databricks (Unity Catalog is plus) and Azure Data Factory (ADF)
Hands-on working experience with Azure DevOps Repo and CI / CD
Strong understanding of modern Data Architecture, Data Warehousing concepts, Data Modelling and ETL / ELT processes
Understanding of Agile ways of working
Any experience working in DBT is a plus
Working with Retail Domain is a plus

Platform Engineering & Operations

Configure and optimize Databricks clusters, including autoscaling, cluster sizing, and runtime selection.

Manage databases, schemas, and Delta Lake tables, including vacuuming, Z-ordering, partitioning, and optimization.

Work with catalogs, Unity Catalog (UC), and access controls for secure data governance.

Monitor and troubleshoot performance issues across jobs, clusters, and workflows.

CI / CD & DevOps

Implement CI / CD pipelines for Databricks workflows using GitHub Actions / Azure DevOps / GitLab.

Work with Databricks Repos and Git workflows (branching, PRs, code reviews).

Version and promote data pipelines across environments.

Cloud & Integration

Integrate Databricks with cloud services such as :

Azure : ADLS Gen2, Key Vault, ADF, Event Hub, Synapse

AWS : S3, Glue, IAM, Lambda, Kinesis (depending on cloud environment)

Work with data ingestion frameworks, messaging systems, and event-driven architectures.

Quality, Testing & Documentation

Implement unit tests and integration tests using pytest, SQL tests, or dbx testing frameworks.

Maintain pipeline documentation, data dictionaries, and technical design artifacts.

Ensure high data quality, reliability, and observability.

Create a job alert for this search

Engineer • Prayagraj(Allahabad), IN

Related jobs

Promoted
New!

Senior DataOps Engineer (AWS)

MSBC GroupPrayagraj(Allahabad), IN

Join us as a Senior DataOps Engineer (AWS)—Drive High-Performance Data Systems for Financial Services.Lead the E-Comms data pipeline within Compass’s Application Simplification workstream : design, ...Show moreLast updated: 21 hours ago

Promoted
New!

Azure Databricks Specialist

KPI PartnersPrayagraj(Allahabad), IN

KPI Partners are seeking highly skilled and experienced Senior Data Engineers to join our dynamic team at KPI, working on challenging and multi-year data transformation projects for our clients.Thi...Show moreLast updated: 16 hours ago

Promoted

Data Engineer

RecroPrayagraj(Allahabad), IN

Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake. AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago

Senior Data Engineer

Professional Search GroupAU

PSG are proud to be partnering with a large-scale mining services organisation that plays a critical role in keeping goods and services moving across Australia. With a rapidly growing customer base ...Show moreLast updated: 30+ days ago

Data Engineer

BethanieAU

Digital Enablement team to support the delivery of our data and systems integration requirements.Reporting to the Manager – Digital Enablement this is a permanent role available full-time or part-t...Show moreLast updated: 7 days ago

Databricks Engineer

AccentureAU

Databricks Data Engineer Consultant.You’ll work on cutting-edge cloud and hybrid data projects, developing scalable pipelines and transforming raw data into meaningful insights.You’ll be hands-on f...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

IntraEdgePrayagraj(Allahabad), IN

We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show moreLast updated: 30+ days ago

Promoted

Databricks Data Engineer Lead – Sustainability Project

Blue Cloud Softech Solutions LimitedPrayagraj(Allahabad), IN

BCSS is seeking a Databricks Data Engineer to support its enterprise-wide Sustainability initiative.The engineer will be responsible for building data pipelines and models to support product-level ...Show moreLast updated: 2 days ago

Promoted

Data Engineer

Insight GlobalPrayagraj(Allahabad), IN

GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts. Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago

Promoted
New!

Data Engineer | Bangalore | Leading Big 4 Consulting Firm

Acme ServicesPrayagraj(Allahabad), IN

Design, develop, and maintain scalable and robust data pipelines (ETL / ELT) using PySpark on the Databricks platform.Implement data ingestion strategies to load data from various sources (e.APIs, da...Show moreLast updated: 21 hours ago

Promoted

Databricks SQL Engineer with Commercial Pharma

KeasisPrayagraj(Allahabad), IN

Job Title : Databricks SQL Engineer with Commercial Pharma.We are looking for a Databricks SQL Engineer to design, develop, and optimize data solutions using Databricks SQL.The ideal candidate will ...Show moreLast updated: 30+ days ago

Promoted
New!

Senior Data Engineer

MindlancePrayagraj(Allahabad), IN

We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show moreLast updated: 21 hours ago

Promoted

Data Engineer

DigitalzonePrayagraj(Allahabad), IN

As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics. Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 17 days ago

Promoted

Google Looker Data Engineer

Insight GlobalPrayagraj(Allahabad), IN

We’re seeking a skilled Data Engineer to support a major reporting migration from Tableau to Google Looker.This role focuses on designing and building scalable data models, data marts, and ETL / ELT ...Show moreLast updated: 8 days ago

Promoted

Azure Databricks

Kumaran SystemsPrayagraj(Allahabad), IN

We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault. The candidate should be proficient in Dat...Show moreLast updated: 2 days ago

Promoted
New!

Data Engineer

MyData Insights Pvt LtdPrayagraj(Allahabad), IN

The ideal candidate has a strong background in building scalable data solutions on.CI / CD pipelines, infrastructure automation using Terraform, and cloud-native data engineering practices.Design, bu...Show moreLast updated: 21 hours ago

Promoted
New!

Databricks Gen AI Engineer

SyrenPrayagraj(Allahabad), IN

Model Serving, Vector Search, and embedding workflows.Clustering, Unity Catalog, Delta Lake).OpenAI / Azure OpenAI), and GenAI app patterns (RAG / Agents). Proficiency in SQL, Spark performance tuning...Show moreLast updated: 21 hours ago

Promoted

Immediate Opening for Data Engineer (Databricks & D365)

GyanSys Inc.Prayagraj(Allahabad), IN

Need to have databricks experience as well as experience with handling D365 data.Need strong SQL skills, strong analytical skills.Show moreLast updated: 7 days ago