Colan Infotech - Azure Data Engineer - Python / PySpark

Azure Data Engineer

Location : Chennai / Bangalore (Work from Office only)

Experience : 3-5 Years

Job Type : Full-time

Job Overview :

Greetings from Colan Infotech! We are urgently hiring a skilled Azure Data Engineer to join our team. With 3-5 years of experience, you'll be instrumental in designing, developing, and maintaining scalable and efficient data pipelines within the Azure ecosystem. Your expertise in Python, PySpark / Scala Spark, Azure Data Factory, and Azure Databricks will be crucial in building robust data solutions that support our analytical and business needs.

Responsibilities :

• Design, develop, and maintain scalable data pipelines using PySpark / Scala Spark, Azure Databricks, Python, and SQL.
• Demonstrate strong programming skills, with the ability to write optimized, reusable, and high-quality code for data processing and transformation.
• Optimize SQL queries for efficient data extraction, manipulation, and loading, particularly within Azure SQL Database and Azure Synapse Analytics.
• Implement and manage ETL processes using Azure Data Factory, ensuring reliable data ingestion and transformation.
• Exhibit expertise in Azure Databricks, including efficient workflow management, data exploration, and leveraging its capabilities for large-scale data processing.
• Collaborate effectively with cross-functional teams to deeply understand business requirements and translate them into technical data solutions.
• Implement best practices for data pipeline optimization, ETL processes, and query performance tuning.
• Create and maintain comprehensive documentation for all designed data pipelines and workflows.
• Troubleshoot and resolve complex issues in data pipelines promptly, ensuring minimal downtime and data integrity.

Required Skills :

ref : hirist.tech)