Talent.com
No longer accepting applications
Lead Data Engineer – Python & GCP

Lead Data Engineer – Python & GCP

People Prime Worldwidecoimbatore, tamil nadu, in
17 hours ago
Job description

Job Title : Lead Data Engineer – Python & GCP

Location : Hyderabad

Exp : 10+ Years

Overview :

We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP) . You will design, develop, and manage scalable ETL / ELT data pipelines , work closely with clients to understand requirements, and build data solutions using modern cloud and open-source technologies.

Key Responsibilities :

  • Design, build, and maintain scalable data pipelines using Python and GCP services .
  • Work with GCP tools such as Dataflow, BigQuery, Cloud Composer, Cloud Functions, Cloud Run, GCS, and IAM .
  • Implement data ingestion, transformation, and quality checks to ensure accurate, reliable data.
  • Architect enterprise data solutions using tools like Kafka, Spark, Airflow, DBT, GKE, APIGEE, and DLP .
  • Use FastAPI for API development and GitHub for version control.
  • Deploy solutions using CI / CD pipelines and manage data workflows in Airflow / Cloud Composer .
  • Write and optimize complex SQL queries (SQL Server, Oracle, PostgreSQL, etc.).
  • Collaborate with data scientists and analysts to deliver efficient data solutions.
  • Document data flows, pipelines, and operational procedures.

Required Skills & Experience :

  • 10+ years of experience in data engineering or backend development.
  • Strong programming skills in Python .
  • Hands-on experience with GCP (Dataflow, BigQuery, Cloud Functions, Cloud Composer).
  • Experience with Kafka, Spark, Redis, Airflow, GKE, Cloud Run, and FastAPI .
  • Excellent knowledge of SQL and relational databases.
  • Experience with CI / CD and GitHub .
  • Experience migrating data from on-premise systems to the cloud.

    • Nice to Have :

  • Experience with Snowflake or Databricks .
  • Knowledge of Azure Data Factory or other Azure data tools.
  • Exposure to machine learning or LLMs in data solutions.
    • Create a job alert for this search

    Data Engineer Python • coimbatore, tamil nadu, in