Senior Manager - Data Governance Engineer

Overview

Apply best practices for data quality & triaging - ( with goal to reduce data downtimes) on existing and new pipelines for our business, customer and Data Science team

Responsibilities

• Own data quality from existing data pipelines, development end-to-end quality framework spanning outages, freshness, accuracy and issue reporting
• Define best practices for quality development, engineering, and coding as part of a world class engineering team in PepsiCo.
• Work with the product team leveraging our core technology platforms, which include Direct Commerce, Supply Chain, Marketing Automation, Mobile, and Data. Collaborate in architecture discussions and architectural decision making that is part of continually improving and expanding these platforms.
• Lead framework development in collaboration with other engineers validate requirements / stories, assess current capabilities, and decompose manual quality related tasks , engineering tasks
• Focus on delivering high quality data pipelines and tools through careful analysis of system capabilities and feature requests, peer reviews, test automation, and collaboration with QA engineers
• Develop software and partner with software teams like MonteCarlo / Great Expectations / Sentry in short iterations to quickly add business value
• Introduce new tools / practices to improve data and code quality this includes researching / sourcing 3rd party tools and libraries, as well as developing tools in-house to improve workflow and quality for all data engineers and operations teams members

Qualifications

Skills Required

Airflow, Sql, Aws, Python, Azure