Databricks Engineer
TTC Groupchennai, tamil nadu, in1 day ago
Job descriptionConfigure and optimize Databricks clusters, including autoscaling, cluster sizing, and runtime selection. Manage databases, schemas, and Delta Lake tables, including vacuuming, Z-ordering, partitioning, and optimization. Work with catalogs, Unity Catalog (UC), and access controls for secure data governance. Monitor and troubleshoot performance issues across jobs, clusters, and workflows. Implement CI / CD pipelines for Databricks workflows using GitHub Actions / Azure DevOps / GitLab. Work with Databricks Repos and Git workflows (branching, PRs, code reviews). Version and promote data pipelines across environments. Integrate Databricks with cloud services such as : Azure : ADLS Gen2, Key Vault, ADF, Event Hub, Synapse AWS : S3, Glue, IAM, Lambda, Kinesis (depending on cloud environment) Work with data ingestion frameworks, messaging systems, and event-driven architectures. Implement unit tests and integration tests using pytest, SQL tests, or dbx testing frameworks. Maintain pipeline documentation, data dictionaries, and technical design artifacts. Ensure high data quality, reliability, and observability.
Role : Databricks Engineer
Position : Remote
Type : Contract
Duration : 3 - 6 months (extendable)
Role Overview
We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform. The ideal candidate will be comfortable working with distributed data processing, designing Delta Lake architectures, implementing CI / CD for data workloads, and optimizing Spark jobs for performance and reliability.
Required Skills
- Strong hands-on experience in building data pipeline using scripting language includes Python, PySpark and SQL
- Strong working experience with Databricks (Unity Catalog is plus) and Azure Data Factory (ADF)
- Hands-on working experience with Azure DevOps Repo and CI / CD
- Strong understanding of modern Data Architecture, Data Warehousing concepts, Data Modelling and ETL / ELT processes
- Understanding of Agile ways of working
- Any experience working in DBT is a plus
- Working with Retail Domain is a plus
Platform Engineering & Operations
CI / CD & DevOps
Cloud & Integration
Quality, Testing & Documentation
Create a job alert for this search
Engineer • chennai, tamil nadu, in
Related jobs
- Promoted
Data Engineer
DigitalzoneChennai, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 17 days ago
- Promoted
Data Engineer
NexionPro ServicesChennai, IN PySpark-based data processing workflows.Collaborate with data architects, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.Optimi...Show moreLast updated: 1 day ago
- Promoted
Senior DataOps Engineer (AWS)
MSBC GroupChennai, IN Join us as a Senior DataOps Engineer (AWS)—Drive High-Performance Data Systems for Financial Services.Lead the E-Comms data pipeline within Compass’s Application Simplification workstream : design, ...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
RecroChennai, IN Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Senior Azure Data Engineer
Tata Consultancy ServicesChennai, IN Job Title : Senior Azure Data Engineer.Required Skillset : Azure data factory, data bricks, data lake, automation and performance optimization of ETL.
Good Knowledge of Data Brick lakehouse and Azure ...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
EXLChennai, Tamil Nadu, India Collaborate with project stakeholders (client) to identify product and technical requirements.Develop, implement, and tune large-scale distributed systems and pipelines that process large volume of...Show moreLast updated: 30+ days ago
- Promoted
Databricks Engineer
TTC GroupChennai, IN We are seeking a Mid-Level Databricks Engineer with strong data engineering fundamentals and hands-on experience building scalable data pipelines on the Databricks platform.The ideal candidate will...Show moreLast updated: 1 day ago
- Promoted
Databricks Gen AI Engineer
SyrenChennai, IN Model Serving, Vector Search, and embedding workflows.Clustering, Unity Catalog, Delta Lake).OpenAI / Azure OpenAI), and GenAI app patterns (RAG / Agents).
Proficiency in SQL, Spark performance tuning...Show moreLast updated: 1 day ago
- Promoted
Azure Databricks
Kumaran SystemsChennai, IN We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault.
The candidate should be proficient in Dat...Show moreLast updated: 3 days ago
- Promoted
Senior Data Engineer
SGS & CoChennai, Tamil Nadu, India Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show moreLast updated: 17 days ago
- Promoted
Data Engineer
Tata Consultancy ServicesChennai, Tamil Nadu, India TCS present an excellent opportunity for Data Engineer Job Location : Delhi, Hyderabad, Bangalore, Kochi, Chennai, Kolkata, Pune Experience required : 4- 12 yrs Skills : Data Engineer - GCPSQL + Py...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
MindlanceChennai, IN We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show moreLast updated: 1 day ago
- Promoted
Azure Datafactory Engineer
Tata Consultancy ServicesChennai, Tamil Nadu, India Azure Synapse, Databricks, ADLS, Pyspark.Follow me for more Job updates!.Show moreLast updated: 7 days ago
- Promoted
AWS Data Engineer
Tata Consultancy Serviceschennai, tamil nadu, in Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Insight GlobalChennai, IN GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago
- Promoted
Azure Databricks Specialist
KPI PartnersChennai, IN KPI Partners are seeking highly skilled and experienced Senior Data Engineers to join our dynamic team at KPI, working on challenging and multi-year data transformation projects for our clients.Thi...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
MyData Insights Pvt LtdChennai, IN The ideal candidate has a strong background in building scalable data solutions on.CI / CD pipelines, infrastructure automation using Terraform, and cloud-native data engineering practices.Design, bu...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
loanDNAChennai, Tamil Nadu, India We are seeking an experienced Snowflake Developer with a minimum of 5 years of hands-on experience in Snowflake development, integration, migration, and deployment.
The ideal candidate will be respo...Show moreLast updated: 14 days ago