Lead Gcp Data Engineer

Job Title : Lead Data Engineer – GCP (BigQuery

• Composer
• Python
• PySpark)

Location : Bengaluru

Experience : 8+ years (data engineering / analytics engineering), with previous lead responsibilities

About the Role :

You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform. You will manage a team of data engineers, collaborate with analytics & data science stakeholders, define best-practices, and ensure robust, production-ready pipelines that support business insights and data products.

Key Responsibilities :

Architect and implement end-to-end data pipelines using BigQuery, PySpark / Spark, Python.

Use Cloud Composer (Airflow) to orchestrate workflows, manage DAGs, scheduling, dependencies and monitoring.

Design and optimise BigQuery usage (partitioning, clustering, cost-control, performance tuning).

Develop and manage large-scale Spark / PySpark jobs for batch and potentially streaming data.

Ensure cloud-native architecture on GCP : storage (Cloud Storage), compute (Dataproc / Dataflow), orchestration (Composer), security / iam.

Lead and mentor a team of data engineers : code reviews, best practices, architecture decisions.

Required Skills & Qualifications :

Minimum ~8 years of experience in data engineering or similar, ideally with lead responsibilities.

Hands-on experience in GCP services—especially BigQuery (data warehousing, optimisation), Cloud Composer (Airflow) for orchestration.

Strong experience with PySpark / Spark for large-scale data processing.

Excellent proficiency in Python (scripting, automation, building frameworks).

Strong SQL skills (in BigQuery or similar cloud data warehouse), including complex queries, optimisation.

Experience designing data models, data warehouses / lakes, ingestion pipelines, transformations.

Experience building / operating CI / CD pipelines, version control (Git), test / monitoring practices.

Excellent communication and leadership skills—able to mentor, lead architecture discussions and engage stakeholders.

Why Join Us :

Lead a talented team and influence data engineering architecture & tooling.

Work on high-impact data platforms that enable business growth.

Use cutting-edge GCP services and modern data engineering practices.