Lead GCP Data Engineer

Exp : 10+ Years

Location : Hyderabad

Work Mode : Hybrid

Notice Period : Immediate

Mandatory Skills :

Bigquery, Google Cloud Storage, GCP, Python

Experience Level :

10 to 12 years of relevant IT experience

Key Responsibilities :

• Design, develop, test, and maintain scalable ETL data pipelines using Python.
• Architect the enterprise solutions with various technologies like Kafka, multi-cloud services, auto-scaling using GKE, Load balancers, APIGEE proxy API management, DBT, using LLMs as needed in the solution, redaction of sensitive information, DLP (Data Loss Prevention) etc.
• Work extensively on Google Cloud Platform (GCP) services such as :

○ Dataflow for real-time and batch data processing

○ Cloud Functions for lightweight serverless compute

○ BigQuery for data warehousing and analytics

○ Cloud Composer for orchestration of data workflows (on Apache Airflow)

○ Google Cloud Storage (GCS) for managing data at scale

○ IAM for access control and security

○ Cloud Run for containerized applications

Should have experience in the following areas :

○ API framework : Python FastAPI

○ Processing engine : Apache Spark

○ Messaging and streaming data processing : Kafka

○ Storage : MongoDB, Redis / Bigtable

○ Orchestration : Airflow

○ Experience in deployments in GKE, Cloud Run.

Good to Have (Optional Skills) :

Additional Details :

Education :

Regards,

Sandeep Kumar

sandeep.vinaganti@quesscorp.com