Azure Data Engineer
Location : Chennai / Bangalore (Work from Office only)
Experience : 3-5 Years
Job Type : Full-time
Job Overview :
Greetings from Colan Infotech! We are urgently hiring a skilled Azure Data Engineer to join our team. With 3-5 years of experience, you'll be instrumental in designing, developing, and maintaining scalable and efficient data pipelines within the Azure ecosystem. Your expertise in Python, PySpark / Scala Spark, Azure Data Factory, and Azure Databricks will be crucial in building robust data solutions that support our analytical and business needs.
Responsibilities :
- Design, develop, and maintain scalable data pipelines using PySpark / Scala Spark, Azure Databricks, Python, and SQL.
- Demonstrate strong programming skills, with the ability to write optimized, reusable, and high-quality code for data processing and transformation.
- Optimize SQL queries for efficient data extraction, manipulation, and loading, particularly within Azure SQL Database and Azure Synapse Analytics.
- Implement and manage ETL processes using Azure Data Factory, ensuring reliable data ingestion and transformation.
- Exhibit expertise in Azure Databricks, including efficient workflow management, data exploration, and leveraging its capabilities for large-scale data processing.
- Collaborate effectively with cross-functional teams to deeply understand business requirements and translate them into technical data solutions.
- Implement best practices for data pipeline optimization, ETL processes, and query performance tuning.
- Create and maintain comprehensive documentation for all designed data pipelines and workflows.
- Troubleshoot and resolve complex issues in data pipelines promptly, ensuring minimal downtime and data integrity.
Required Skills :
3-5 years of experience as an Azure Data Engineer.Strong proficiency in PySpark / Scala Spark and Python programming.Hands-on expertise with Azure Data Factory (ADF) for ETL / ELT pipeline orchestration.Extensive experience with Azure Databricks for data processing and analytics.Proficiency in SQL, with experience working with Azure SQL Database and Azure Synapse Analytics.Solid understanding and practical experience with ETL concepts and methodologies.Ability to write optimized, reusable, and high-quality code.Strong problem-solving and debugging skills for data pipelines.Excellent collaboration and communication skills to work with cross-functional teams.ref : hirist.tech)