Lead Data Engineer - Python & GCP

Job Title :

Lead Data Engineer – Python & GCP

Location : Hyderabad

Exp : 10+ Years

Overview :

We are seeking an experienced

Lead Data Engineer

with strong expertise in

Python

and

Google Cloud Platform (GCP)

• . You will design, develop, and manage scalable

ETL / ELT data pipelines

, work closely with clients to understand requirements, and build data solutions using modern cloud and open-source technologies.

Key Responsibilities :

Design, build, and maintain scalable

data pipelines

using

Python

and

GCP services

Work with

GCP tools

such as

Dataflow, BigQuery, Cloud Composer, Cloud Functions, Cloud Run, GCS,

and

IAM

Implement

data ingestion, transformation, and quality checks

to ensure accurate, reliable data.

Architect enterprise data solutions using tools like

Kafka, Spark, Airflow, DBT, GKE, APIGEE,

and

DLP

Use

FastAPI

for API development and

GitHub

for version control.

Deploy solutions using

CI / CD pipelines

and manage data workflows in

Airflow / Cloud Composer

Write and optimize complex

SQL queries

(SQL Server, Oracle, PostgreSQL, etc.).

Collaborate with data scientists and analysts to deliver efficient data solutions.

Document data flows, pipelines, and operational procedures.

Required Skills & Experience :

10+ years

of experience in data engineering or backend development.

Strong programming skills in

Python

Hands-on experience with

GCP

(Dataflow, BigQuery, Cloud Functions, Cloud Composer).

Experience with

Kafka, Spark, Redis, Airflow, GKE, Cloud Run,

and

FastAPI

Excellent knowledge of

SQL

and relational databases.

Experience with

CI / CD

and

GitHub

Experience migrating data from on-premise systems to the cloud.

Nice to Have : Experience with

Snowflake

or

Databricks

Knowledge of

Azure Data Factory

or other Azure data tools.

Exposure to

machine learning

or

LLMs

in data solutions.