Senior Data Engineer

Summary

The Senior Data Engineer is responsible for building and optimizing ETL / ELT pipelines that process terabytes of data daily across 186 data assets, implementing BigQuery datasets with enterprise-scale performance optimization, and creating the data quality monitoring and transparency dashboards that enable data owner self-service.

Required Qualifications

Google Cloud Platform Data Engineering

• 5+ years of data engineering experience with at least 2+ years focused on Google Cloud Platform
• Strong proficiency with BigQuery including :
• Advanced SQL for analytical queries (window functions, CTEs, complex joins)
• Partitioning and clustering strategies for performance optimization
• Materialized views, authorized views, and query optimization techniques
• Cost optimization through efficient query design and storage management
• Understanding of BigQuery architecture (slot allocation, shuffle operations, distributed execution)
• Hands-on experience with Google Cloud Dataflow and Apache Beam :
• Pipeline development in Python or Java
• Batch and streaming data processing patterns
• Performance tuning and resource optimization
• Error handling and pipeline monitoring
• Proficiency with Cloud Composer (Apache Airflow) :
• DAG development and dependency management
• Airflow operators (BigQueryOperator, DataflowOperator, custom operators)
• Workflow orchestration for complex multi-step processes
• Monitoring and troubleshooting failed workflows

ETL / ELT & Data Integration

SQL & Database Technologies

Programming & Scripting

Data Security & Compliance

Preferred Qualifications