Databricks Data Engineer

About the Role

We are looking for a highly skilled Senior Data Engineer with strong expertise in Apache

Spark and Databricks to join our growing data engineering team. You will be responsible for

designing, developing, and optimizing scalable data pipelines and applications using modern

cloud data technologies. This is a hands-on role requiring deep technical knowledge, strong

problem-solving skills, and a passion for building efficient, high-performance data solutions

that drive business value.

Responsibilities :

Design, develop, and implement scalable data pipelines and applications using

Apache Spark and Databricks, adhering to industry best practices.

Perform in-depth performance tuning and optimization of Spark applications within

the Databricks environment.

Troubleshoot complex issues related to data ingestion, transformation, and pipeline

execution.

Collaborate with cross-functional teams including data scientists, analysts, and

architects to deliver end-to-end data solutions.

Continuously evaluate and adopt new technologies and tools in the Databricks and

cloud ecosystem.

Optimize Databricks cluster configurations for cost-effectiveness and performance.

Apply data engineering principles to enable high-quality data ingestion,

transformation, and delivery processes.

Document technical designs, development processes, and operational procedures.

Qualifications :

Bachelor’s degree in Computer Science, Engineering, or a related field.

10+ years of experience in data engineering or big data development.

5+ years of hands-on experience with Apache Spark and Databricks.

Deep understanding of Spark internals, Spark Streaming, and Delta Lake.

Experience in developing solutions using Azure Data Services including : Azure

Databricks, Azure Data Factory, Azure DevOps, Azure Functions, Azure SQL Database,

Azure Event Grid, Cosmos DB.

Familiarity with Striim or similar real-time data integration platforms is a plus.

Proficient in PySpark or Scala.

Strong experience in performance tuning, cost optimization, and cluster

management in Databricks.

Solid understanding of data warehousing, ETL / ELT pipelines, and data modelling.

Experience working with cloud platforms (Azure preferred; AWS / GCP is a plus).

Familiarity with Agile / Scrum methodologies.

Preferred Qualifications

Databricks Certified Professional Data Engineer certification is a strong plus.

Strong communication skills—both written and verbal—with the ability to convey

technical concepts to non-technical stakeholders.