Talent.com
EverestDX - Senior Data Engineer - Python / PySpark

EverestDX - Senior Data Engineer - Python / PySpark

EverestDXHyderabad
1 day ago
Job description

Description : Job Summary :

  • We are seeking an experienced Azure Data Engineer / Lead with strong expertise in Azure services, distributed data processing, and team leadership.
  • The ideal candidate will have hands-on experience with Azure Databricks, Azure Data Lake, Azure Data Factory, PySpark / Spark, and SQL, along with the ability to guide a small team on technical design and delivery.

Key Responsibilities :

  • Design, develop, and optimize large-scale data pipelines using Azure Databricks, ADLS, and ADF.
  • Implement and manage distributed data processing solutions using Spark / PySpark.
  • Write and optimize SQL queries for data extraction, transformation, and analysis.
  • Lead and mentor a small team of data engineers, providing technical guidance and reviewing design solutions.
  • Collaborate with stakeholders to translate business requirements into scalable and efficient data solutions.
  • Ensure data quality, governance, and compliance across data platforms.
  • Support production deployments, troubleshooting, and performance optimization.

    • Required Skills :

  • Strong expertise with Azure Databricks, Azure Data Lake, and Azure Data Factory.
  • Hands-on experience in Spark / PySpark for distributed data processing.
  • Proficiency in SQL for complex queries and optimization.
  • Solid understanding of Azure cloud platform and its ecosystem.
  • Proven experience in leading and mentoring teams.
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration abilities

    • (ref : hirist.tech)

    Create a job alert for this search

    Senior Data Engineer • Hyderabad