Azure Data Engineer - Python / PySpark

Role Description :

We are seeking a highly skilled Azure Data Engineer to join our team. The ideal candidate will be responsible for designing, building, and maintaining robust and scalable data pipelines on the Microsoft Azure platform. This role involves integrating, transforming, and consolidating data from various sources to enable business intelligence and analytics. You will work closely with stakeholders to understand their data needs and deliver efficient, high-quality data solutions.

Key Responsibilities :

• Data Pipeline Development : Design and implement data pipelines using Azure Data Factory and Databricks to ingest, transform, and load data from various sources.
• Big Data Processing : Utilize PySpark and Databricks notebooks to process large datasets and implement complex data transformations.
• Data Integration & Consolidation : Integrate and consolidate data from structured and unstructured sources, loading the processed data into Azure Data Lake Storage (ADLS).
• ETL / ELT Processes : Develop and maintain Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes, including the use of SSIS.
• DevOps & CI / CD : Implement Azure DevOps and CI / CD pipelines for automated deployment of data solutions.
• Solution Design : Collaborate with business stakeholders to understand customer needs and translate them into effective technical data solutions.

Technical Skills :

Education :

Required Experience :

Certifications (Preferred) :

(ref : hirist.tech)