Principal Data Integration Engineer

About the role

We are seeking Senior Data Engineers to join our data engineering team, who are passionate about data & analytics. The ideal fit for the role will have a strong background in handling large volumes of data with Apache Spark to build, enhance bespoke systems to harness data .The key focus of the position is to build or maintain systems to capture and store data on behalf of the business.

Key Responsibilities

• Design, develop, and maintain ETL processes and data pipelines using AWS Glue with Pyspark.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize and tune data pipelines for performance and scalability.
• Ensure data quality and integrity through robust testing and validation processes.
• Implement data governance and security best practices.
• Monitor and troubleshoot data pipelines to ensure continuous data flow and address any issues promptly.
• Stay up-to-date with the latest trends and technologies in data engineering.

Required Qualification