Data Engineering Manager (AWS Databricks)

We are looking for an experienced Data Engineering Manager with a strong foundation in Python, SQL, and Spark , and hands-on expertise in AWS, Databricks . In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You'll work closely with cross-functional teams to drive data reliability, quality, and performance.

Responsibilities :

• Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs.
• Building data lake in WS Databricks.
• Build and maintain robust ETL / ELT workflows using Python and SQL to handle structured and semi-structured data.
• Develop distributed data processing solutions using Apache Spark or PySpark .
• Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.
• Ensure data quality, governance, security, and compliance across pipelines and data stores.
• Monitor, troubleshoot, and improve the performance of data systems and pipelines.
• Participate in code reviews and help establish engineering best practices.
• Mentor junior data engineers and support their technical development.

Requirements :

Skills Required

S3, Airflow, Pyspark, Emr, Sql, Lambda, Git, glue , Spark, Databricks, Python, Aws