Azure Data Engineer

Data Engineer (Property and Casualty Insurance - Commercial Insurance) - SQL, Azure, ADF with 7+ years experience

A Data Engineer with 7+ years of experience in Commercial Insurance and expertise in SQL, Azure, and Azure Data Factory (ADF) is responsible for designing, building, and maintaining the data infrastructure that supports complex analytical and reporting needs. They must ensure data from diverse sources is reliably ingested, transformed, and made accessible for business insights within a heavily regulated industry.

Core roles and responsibilities

• Data pipeline development : Design, build, and maintain robust and scalable ETL (Extract, Transform, Load) and ELT processes using Azure Data Factory (ADF). This includes orchestrating data movement and transformation from various sources into centralized data storage.
• Commercial insurance data expertise : Apply a deep understanding of commercial insurance data, including policies, claims, premiums, and reinsurance, to design data models that meet business requirements.
• SQL development and optimization : Write and optimize complex SQL queries, stored procedures, and scripts for data transformation, aggregation, and analysis within Azure SQL databases.
• Azure cloud architecture : Implement and manage scalable data solutions on the Azure platform, leveraging services such as Azure Data Lake Storage (ADLS) for efficient storage of large datasets.
• Data modeling and warehousing : Create and manage logical and physical data models for data warehouses and data lakes, ensuring the design supports efficient reporting and analytics.
• Collaboration and communication : Work closely with data analysts, data scientists, and business stakeholders to understand data requirements and assist with technical issues.
• Data quality and governance : Implement data validation, quality checks, and monitoring to ensure data integrity, accuracy, and adherence to security and compliance standards (e.g., GDPR).
• Performance tuning and monitoring : Identify and resolve performance bottlenecks in data pipelines and optimize database and query performance. Use monitoring tools to ensure high availability and minimal downtime.

Essential skills and qualifications :

Technical skills

Industry and domain knowledge :

Soft skills :