Azure Data Engineer - PySpark

Description : LanceSoft is a global leader in providing technology and talent solutions, and we are looking for a highly skilled and experienced Azure Data Engineer to join our dynamic team.

If you thrive on building scalable, high-performance data pipelines in the cloud, we want to hear from you!

Role Details

Job Title : Azure Data Engineer

Experience : 58 years

Location : SEEPZ, Andheri, Mumbai, Maharashtra

Key Responsibilities :

As an Azure Data Engineer, you will be crucial in designing and maintaining our cutting-edge data platform.

Your primary responsibilities will include :

• Data Pipeline Development : Design, build, and maintain robust and scalable data pipelines utilizing Azure Databricks, PySpark, and Azure Data Factory (ADF).
• Data Transformation (ETL / ELT) : Perform complex Extract, Transform, and Load (ETL) processes from various disparate sources into Azure Data Lake Storage (ADLS) or equivalent.
• SQL Expertise : Write, optimize, and tune advanced SQL queries, stored procedures, and complex data transformations for optimal performance.
• Collaboration : Work closely with data scientists, business analysts, and fellow engineers to translate business requirements into efficient, high-impact data solutions.
• Performance & Reliability : Monitor data pipeline performance, troubleshoot technical issues, and ensure solutions are reliable, scalable, and meet service level objectives (SLOs).
• Documentation : Maintain comprehensive and clear documentation of data flows, schemas, and transformations to ensure system maintainability.

Required Technologies & Skills :

Candidates must have strong, hands-on experience with the following :

(ref : hirist.tech)