Databricks Engineer

TTC Groupvapi, gujarat, in

13 hours ago

Job description

Role : Databricks Engineer

Position : Remote

Type : Contract

Duration : 3 - 6 months (extendable)

Role Overview

We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform. The ideal candidate will be comfortable working with distributed data processing, designing Delta Lake architectures, implementing CI / CD for data workloads, and optimizing Spark jobs for performance and reliability.

Required Skills

Strong hands-on experience in building data pipeline using scripting language includes Python, PySpark and SQL
Strong working experience with Databricks (Unity Catalog is plus) and Azure Data Factory (ADF)
Hands-on working experience with Azure DevOps Repo and CI / CD
Strong understanding of modern Data Architecture, Data Warehousing concepts, Data Modelling and ETL / ELT processes
Understanding of Agile ways of working
Any experience working in DBT is a plus
Working with Retail Domain is a plus

Platform Engineering & Operations

Configure and optimize Databricks clusters, including autoscaling, cluster sizing, and runtime selection.

Manage databases, schemas, and Delta Lake tables, including vacuuming, Z-ordering, partitioning, and optimization.

Work with catalogs, Unity Catalog (UC), and access controls for secure data governance.

Monitor and troubleshoot performance issues across jobs, clusters, and workflows.

CI / CD & DevOps

Implement CI / CD pipelines for Databricks workflows using GitHub Actions / Azure DevOps / GitLab.

Work with Databricks Repos and Git workflows (branching, PRs, code reviews).

Version and promote data pipelines across environments.

Cloud & Integration

Integrate Databricks with cloud services such as :

Azure : ADLS Gen2, Key Vault, ADF, Event Hub, Synapse

AWS : S3, Glue, IAM, Lambda, Kinesis (depending on cloud environment)

Work with data ingestion frameworks, messaging systems, and event-driven architectures.

Quality, Testing & Documentation

Implement unit tests and integration tests using pytest, SQL tests, or dbx testing frameworks.

Maintain pipeline documentation, data dictionaries, and technical design artifacts.

Ensure high data quality, reliability, and observability.

Create a job alert for this search

Engineer • vapi, gujarat, in

Related jobs

Promoted
New!

Senior Data Engineer

Mindlancevapi, gujarat, in

We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show moreLast updated: 13 hours ago

Promoted
New!

Snowflake + DBT Engineer

MindBrainvapi, gujarat, in

Location : Pune — Remote / Hybrid.We are looking for a highly experienced Snowflake + DBT Engineer to architect, build, and optimize modern cloud data platforms. The ideal candidate should have deep ex...Show moreLast updated: 13 hours ago

Promoted

Senior AWS Data Engineer_Exp : 5+ Years

Atyeti Incvapi, gujarat, in

Required Skills & Qualifications.Bachelor’s or Master’s degree in Computer Science or equivalent experience.Application Developer or in similar software engineering roles.Python; strong SQL and clo...Show moreLast updated: 16 days ago

Promoted
New!

Data Engineer

JIT Global Infosystems Pvt. Ltd.vapi, gujarat, in

Applications, Analytics, and Consulting services to clients worldwide.By combining technological expertise with deep domain knowledge, we ensure tangible improvements in business performance.Our ex...Show moreLast updated: 13 hours ago

Promoted
New!

Senior DataOps Engineer (AWS)

MSBC Groupvapi, gujarat, in

Join us as a Senior DataOps Engineer (AWS)—Drive High-Performance Data Systems for Financial Services.Lead the E-Comms data pipeline within Compass’s Application Simplification workstream : design, ...Show moreLast updated: 13 hours ago

Promoted

AWS Data Engineer

Tata Consultancy Servicesvapi, gujarat, in

Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 25 days ago

Promoted
New!

Data Engineer (Remote)

QAAgility Technologiesvapi, gujarat, in

Remote

The ideal candidate will be responsible for designing, developing, and maintaining robust data pipelines that support key business functions across the supply chain domain.This role requires strong...Show moreLast updated: 13 hours ago

Promoted

Data Engineer

K&K Talents - Indiavapi, gujarat, in

This position is with one of our clients in.Location : Chennai ,Bangalore , Hyderabad , Gurgaon , Bhopal, Pune,Jaipur.Experience in data engineering, with a focus on data architecture and pipeline d...Show moreLast updated: 30+ days ago

Promoted

Azure Databricks

Kumaran Systemsvapi, gujarat, in

We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault. The candidate should be proficient in Dat...Show moreLast updated: 2 days ago

Promoted
New!

Data Engineer

NexionPro Servicesvapi, gujarat, in

PySpark-based data processing workflows.Collaborate with data architects, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.Optimi...Show moreLast updated: 13 hours ago

Promoted

Data Engineer

MerilVapi, Gujarat, India

Experience Required : Minimum 1–2 Years.We are seeking a skilled and motivated Data Engineer to join our growing technology team. You’ll work with diverse data types and modern data platforms to desi...Show moreLast updated: 14 days ago

Promoted
New!

ETL & Tableau Data Engineer

Magma Consultancyvapi, gujarat, in

ETL & Tableau Data Engineer (Remote | Full-Time | Join in 2 Weeks).We are a forward-thinking technology and data consulting firm dedicated to helping organizations unlock the full potential of thei...Show moreLast updated: 13 hours ago

Promoted

Databricks Data Engineer Lead – Sustainability Project

Blue Cloud Softech Solutions Limitedvapi, gujarat, in

BCSS is seeking a Databricks Data Engineer to support its enterprise-wide Sustainability initiative.The engineer will be responsible for building data pipelines and models to support product-level ...Show moreLast updated: 2 days ago

Promoted
New!

Azure Databricks Specialist

KPI Partnersvapi, gujarat, in

KPI Partners are seeking highly skilled and experienced Senior Data Engineers to join our dynamic team at KPI, working on challenging and multi-year data transformation projects for our clients.Thi...Show moreLast updated: 13 hours ago

Promoted

Data Engineer

Sourcebaevapi, gujarat, in

Join Our Data Engineering Team in Mumbai! (5-8 Yrs Exp.We're looking for an experienced and driven.If you are passionate about building scalable, high-performance data platforms and mentoring the n...Show moreLast updated: 16 days ago

Promoted

(Senior) Azure Data Engineer - Full remote - contractor in USD

All European Careersvapi, gujarat, in

Remote

For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databric...Show moreLast updated: 2 days ago

Promoted
New!

Databricks Gen AI Engineer

Syrenvapi, gujarat, in

Model Serving, Vector Search, and embedding workflows.Clustering, Unity Catalog, Delta Lake).OpenAI / Azure OpenAI), and GenAI app patterns (RAG / Agents). Proficiency in SQL, Spark performance tuning...Show moreLast updated: 13 hours ago

Promoted
New!

Senior Data Engineer / Architect - Snowflake

Aezion, Incvapi, gujarat, in

We are seeking a highly skilled Snowflake Data Engineer with expertise in DBT, Airbyte, and Airflow, combined with strong experience in Kimball dimensional modeling. The ideal candidate will design ...Show moreLast updated: 13 hours ago