Talent.com
Data Engineering Manager (AWS Databricks)

Data Engineering Manager (AWS Databricks)

BlendHyderabad, Telangana, India
23 hours ago
Job description

We are looking for an experienced Data Engineering Manager with a strong foundation in Python, SQL, and Spark , and hands-on expertise in AWS, Databricks . In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You’ll work closely with cross-functional teams to drive data reliability, quality, and performance.

Responsibilities :

  • Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs.
  • Building data lake in WS Databricks.
  • Build and maintain robust ETL / ELT workflows using Python and SQL to handle structured and semi-structured data.
  • Develop distributed data processing solutions using Apache Spark or PySpark .
  • Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.
  • Ensure data quality, governance, security, and compliance across pipelines and data stores.
  • Monitor, troubleshoot, and improve the performance of data systems and pipelines.
  • Participate in code reviews and help establish engineering best practices.
  • Mentor junior data engineers and support their technical development.

Requirements :

  • Bachelor's or master's degree in computer science, Engineering, or a related field.
  • 8+ years of hands-on experience in data engineering , with at least 3 years working with AWS Databricks .
  • Strong programming skills in Python for data processing and automation.
  • Advanced proficiency in SQL for querying and transforming large datasets.
  • Deep experience with Apache Spark / PySpark in a distributed computing environment.
  • Solid understanding of data modelling, warehousing, and performance optimization techniques.
  • Proficiency with AWS services such as Glue , S3 , Lambda and EMR .
  • Experience with version control Git or Code commit
  • Experience in any workflow orchestration like Airflow, AWS Step functions is a plus.
  • Excellent communication and stakeholder management skills with a strategic mindset.
    • Create a job alert for this search

    Manager Data Engineering • Hyderabad, Telangana, India