Key Responsibilities :
- Design and develop scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks .
- Implement and manage data storage solutions using Azure Data Lake Storage (ADLS) .
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
- Optimize data workflows for performance, reliability, and cost-efficiency.
- Develop and maintain CI / CD pipelines for data solutions using tools like Azure DevOps or GitHub Actions.
- Ensure data security, governance, and compliance standards are met.
- Troubleshoot and resolve issues related to data quality, performance, and system integration.
Required Skills :
6–8 years of overall experience in data engineering or related roles.Strong hands-on experience with Azure Data Factory (ADF) , Azure Databricks , and ADLS .Proficient in PySpark , SQL , and data modeling techniques.Experience in implementing CI / CD pipelines in Azure environments.Solid understanding of ETL / ELT processes and best practices.Familiarity with Azure DevOps , Git , and monitoring tools like Azure Monitor or Log Analytics.Good problem-solving skills and attention to detail.Excellent communication and collaboration abilities.Preferred Qualifications :
Azure certifications such as DP-203 : Azure Data Engineer Associate .Skills Required
Azure Databricks, Pyspark, Sql, Azure Devops