Cloud Data Pipeline Architect

Key Responsibilities

• Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and consumption.
• Develop Data Pipelines : Design and develop ETL / ELT data pipelines using tools like Apache Airflow (Cloud Composer) and programming languages such as Python and SQL for batch and real-time processing.
• Create Data Models : Build logical and physical data models, including dimensional modelling and schema design, to support data warehousing, data lakes, and analytics.
• Ensure Data Quality and Governance : Establish and enforce data governance, security, and quality standards, implementing data validation and testing procedures.
• Collaborate with Stakeholders : Work with data engineers, business analysts, data scientists, and product owners to translate business requirements into technical data solutions.
• Optimize GCP Services : Optimize the performance and cost-effectiveness of GCP services, particularly Big Query, for analytics and data storage.
• Provide Technical Guidance : Lead architectural reviews, provide technical guidance on cloud-native data strategies, and mentor engineering teams on GCP best practices.

Required Skills and Knowledge