Databricks Engineer

TTC Groupludhiana, punjab, in

1 day ago

Job description

Role : Databricks Engineer

Position : Remote

Type : Contract

Duration : 3 - 6 months (extendable)

Role Overview

We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform. The ideal candidate will be comfortable working with distributed data processing, designing Delta Lake architectures, implementing CI / CD for data workloads, and optimizing Spark jobs for performance and reliability.

Required Skills

Strong hands-on experience in building data pipeline using scripting language includes Python, PySpark and SQL
Strong working experience with Databricks (Unity Catalog is plus) and Azure Data Factory (ADF)
Hands-on working experience with Azure DevOps Repo and CI / CD
Strong understanding of modern Data Architecture, Data Warehousing concepts, Data Modelling and ETL / ELT processes
Understanding of Agile ways of working
Any experience working in DBT is a plus
Working with Retail Domain is a plus

Platform Engineering & Operations

Configure and optimize Databricks clusters, including autoscaling, cluster sizing, and runtime selection.

Manage databases, schemas, and Delta Lake tables, including vacuuming, Z-ordering, partitioning, and optimization.

Work with catalogs, Unity Catalog (UC), and access controls for secure data governance.

Monitor and troubleshoot performance issues across jobs, clusters, and workflows.

CI / CD & DevOps

Implement CI / CD pipelines for Databricks workflows using GitHub Actions / Azure DevOps / GitLab.

Work with Databricks Repos and Git workflows (branching, PRs, code reviews).

Version and promote data pipelines across environments.

Cloud & Integration

Integrate Databricks with cloud services such as :

Azure : ADLS Gen2, Key Vault, ADF, Event Hub, Synapse

AWS : S3, Glue, IAM, Lambda, Kinesis (depending on cloud environment)

Work with data ingestion frameworks, messaging systems, and event-driven architectures.

Quality, Testing & Documentation

Implement unit tests and integration tests using pytest, SQL tests, or dbx testing frameworks.

Maintain pipeline documentation, data dictionaries, and technical design artifacts.

Ensure high data quality, reliability, and observability.

Create a job alert for this search

Engineer • ludhiana, punjab, in

Related jobs

Promoted

GCP Data Engineer

Xsell Resourcesludhiana, punjab, in

Seeking a GCP Certified Data Engineer to work remotely from India for our Fortune 5 healthcare client in the US.No notice periods more than 15 days. GCP Google Cloud Professional Data Engineer Certi...Show moreLast updated: 16 days ago

Promoted

Senior AWS Data Engineer_Exp : 5+ Years

Atyeti Includhiana, punjab, in

Required Skills & Qualifications.Bachelor’s or Master’s degree in Computer Science or equivalent experience.Application Developer or in similar software engineering roles.Python; strong SQL and clo...Show moreLast updated: 17 days ago

Promoted
New!

AWS Data Engineer

COVET IT INCludhiana, punjab, in

Please go through the below requirements and let me know your interest and forward your resume along with your contact information to raja@covetitinc. We are seeking a highly skilled AWS Data Engine...Show moreLast updated: 12 hours ago

Promoted

Data Engineer

BayOne Solutionsludhiana, punjab, in

We are seeking a highly experienced Data Engineer to join our MarTech team and play a pivotal role in driving innovation within our microservices architecture, with a strong emphasis on data engine...Show moreLast updated: 3 days ago

Promoted

Senior Data Engineer

Intelliasludhiana, punjab, in

Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines. Kafka + Flink / Spark Streaming).Python for data engin...Show moreLast updated: 18 days ago

Promoted

AWS Engineer

Spryc Systemsludhiana, punjab, in

We are seeking an experienced AWS Engineer to design, implement, and maintain AWS infrastructure and services in a managed service environment. The ideal candidate will possess deep expertise in AWS...Show moreLast updated: 18 days ago

Promoted

AWS Data Engineer

Tata Consultancy Servicesludhiana, punjab, in

Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 26 days ago

Promoted

Data Engineer

MyData Insights Pvt Ltdludhiana, punjab, in

The ideal candidate has a strong background in building scalable data solutions on.CI / CD pipelines, infrastructure automation using Terraform, and cloud-native data engineering practices.Design, bu...Show moreLast updated: 1 day ago

Promoted

Principal Data Engineer

CodeMyMobileludhiana, punjab, in

Experience Required - 7 to 10 Years.Are you a Data Engineer who cares about clean engineering, autonomy, and solving real data challenges? If this sounds like you, we’d love to connect!.Email your ...Show moreLast updated: 17 days ago

Promoted

Freelance Data Quality Engineer

Leading MNCludhiana, punjab, in

Freelance Data Quality Engineer.The candidate should have a minimum of 8+ yrs.If you're looking for freelance / part time opportunity (along with your day job) & a chance to work with the top 0.You ...Show moreLast updated: 17 days ago

Promoted

DevOps Engineer

DBiz.ailudhiana, punjab, in

We are seeking a skilled Analyst - DevOps to join our team at DBiz AI Solutions Pvt.The ideal candidate will be responsible for implementing and managing DevOps practices, ensuring seamless integra...Show moreLast updated: 1 day ago

Promoted

Azure Data Engineer

Tata Consultancy Servicesludhiana, punjab, in

Interview Date : Weekend Virtual Drive 15-Nov-25.Extensive coding experience (3+ years) including Python / Pyspark (1 years +). Experience developing in Azure with key data technologies (e.ADLS, ADF, A...Show moreLast updated: 30+ days ago

Promoted

(Senior) Azure Data Engineer - Full remote - contractor in USD

All European Careersludhiana, punjab, in

Remote

For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databric...Show moreLast updated: 3 days ago

Promoted

Azure Databricks

Kumaran Systemsludhiana, punjab, in

We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault. The candidate should be proficient in Dat...Show moreLast updated: 3 days ago

Promoted

Databricks Data Engineer Lead – Sustainability Project

Blue Cloud Softech Solutions Limitedludhiana, punjab, in

BCSS is seeking a Databricks Data Engineer to support its enterprise-wide Sustainability initiative.The engineer will be responsible for building data pipelines and models to support product-level ...Show moreLast updated: 3 days ago

Promoted

Snowflake + DBT Engineer

MindBrainludhiana, punjab, in

Location : Pune — Remote / Hybrid.We are looking for a highly experienced Snowflake + DBT Engineer to architect, build, and optimize modern cloud data platforms. The ideal candidate should have deep ex...Show moreLast updated: 1 day ago

Promoted

Data Engineer

Tata Consultancy Servicesludhiana, punjab, in

Required Technical Skill Set -.Create and maintain optimal data pipeline architecture,.Assemble large, complex data sets that meet functional / non-functional business requirements.Identify, design...Show moreLast updated: 30+ days ago

Promoted

ETL & Tableau Data Engineer

Magma Consultancyludhiana, punjab, in

ETL & Tableau Data Engineer (Remote | Full-Time | Join in 2 Weeks).We are a forward-thinking technology and data consulting firm dedicated to helping organizations unlock the full potential of thei...Show moreLast updated: 1 day ago