Data Engineer - Azure

Role Overview :

The Data Engineer - Azure will be responsible for managing, maintaining, and optimizing data pipelines within the Azure ecosystem. You will work closely with various internal teams, including developers, stakeholders, and business units, to ensure smooth and efficient data operations. The role involves troubleshooting, debugging, and resolving data issues, ensuring the timely and accurate delivery of data solutions.

Key Responsibilities :

• Oversee the daily execution and monitoring of data pipelines, ensuring they are running as expected.
• Troubleshoot and resolve issues in Azure Data Factory and other Azure services, ensuring pipeline reliability and data consistency.
• Work with developers to proactively identify and address potential data issues before they impact operations.
• Establish strong relationships with internal teams, including developers, stakeholders, and business units, to understand data needs and priorities.
• Act as a liaison between stakeholders and developers to ensure timely updates and transparency regarding data issues, delays, and resolutions.
• Prioritize and manage tasks based on business needs, ensuring that all parties are aligned on project timelines and expectations.
• Respond promptly to both internal and external inquiries related to data issues, ensuring urgent issues are resolved as a priority.
• Work on debugging logs to identify data issues, applying appropriate solutions to ensure seamless data flow.
• Assist with user access management, including reviewing permissions, managing security credentials, and overseeing key vault management.
• Perform password / secret rotation in compliance with security protocols.
• Create detailed requirement documents and markups to guide developers in building future automated data pipelines and data tables.
• Develop and maintain comprehensive documentation on data processes, pipelines, and operational procedures.
• Provide accurate and timely administrative, data, and operational support to stakeholders in the Insurance, Finance, Business Services, and Core Systems domains.
• Gain a deep understanding of the entire data lifecycle, from ingestion through processing, transformation, and storage, ensuring high-quality data management practices.
• Continuously evaluate and optimize processes to enhance data quality, reduce bottlenecks, and improve operational efficiency.
• Manage assigned projects effectively, ensuring they are completed within set timelines and meet business requirements.
• Track project progress, manage dependencies, and communicate updates to stakeholders to ensure alignment with business objectives.
• Implement best practices for data management and encourage the team to adhere to high-quality standards while developing.
• Experience in Power BI or similar BI tools to create reports and dashboards that monitor the status of the Azure stack, including resources, pipelines, and key metrics.
• Provide insights and status updates on pipeline health and resource utilization to business stakeholders through automated reports and dashboards.

Required Skills & Qualifications :

Education : Bachelors degree in Computer Science, Information Technology, or a related : 4- 6 years of experience in Data Engineering with a strong focus on Azure technologies and cloud-based data operations.

ref : hirist.tech)