Data Engineering Manager (AWS Databricks)

We are looking for an experienced

Data Engineering

Manager

with a strong foundation in

Python, SQL, and Spark , and hands-on expertise in

AWS, Databricks . In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You’ll work closely with cross-functional teams to drive data reliability, quality, and performance.

Responsibilities :

Design, develop, and optimize scalable data pipelines using Databricks in

AWS

such as

Glue, S3, Lambda, EMR, Databricks notebooks,

workflows and jobs.

Building data lake in WS Databricks.

Build and maintain robust

ETL / ELT workflows

using

Python

and

SQL

to handle structured and semi-structured data.

Develop distributed data processing solutions using

Apache Spark

or

PySpark .

Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.

Ensure data quality, governance, security, and compliance across pipelines and data stores.

Monitor, troubleshoot, and improve the performance of data systems and pipelines.

Participate in code reviews and help establish engineering best practices.

Mentor junior data engineers and support their technical development.

Requirements :

Bachelor's or master's degree in computer science, Engineering, or a related field.

8+

years of hands-on experience in

data engineering , with at least 3 years working with

AWS

Databricks .

Strong programming skills in

Python

for data processing and automation.

Advanced proficiency in

SQL

for querying and transforming large datasets.

Deep experience with

Apache Spark / PySpark

in a distributed computing environment.

Solid understanding of data modelling, warehousing, and performance optimization techniques.

Proficiency with AWS services such as

Glue ,

S3 ,

Lambda

and

EMR .

Experience with version control Git or Code commit

Experience in any workflow orchestration like Airflow, AWS Step functions is a plus.

Excellent communication and stakeholder management skills with a strategic mindset.