Important Note (Please Read Before Applying)
🚫 Do NOT apply if :
- You have less than 10 years of relevant IT experience
- You lack hands-on Python + GCP project experience
- You have no real-time experience with Dataflow, BigQuery, Airflow, or Spark
- You are not experienced in architecting or designing data pipelines
- You cannot work in the specified work mode / location
- You cannot join within a reasonable timeframe
✅ Apply ONLY if you meet ALL mandatory criteria above. Random or irrelevant applications will not be processed.
Job Title : Lead Data Engineer – Python & GCP
Location : Hyderabad
Experience : 10+ Years (STRICTLY)
Employment Type : Permanent
Notice Period : Immediate
About the Company :
Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.
Job Description :
Design, build, and maintain scalable ETL / ELT data pipelines using PythonArchitect enterprise-grade solutions leveraging modern data technologiesWork directly with customers for requirement gathering, discovery, and designImplement data ingestion, transformation, validation, and quality checksDevelop solutions using key GCP services (Dataflow, BigQuery, Cloud Functions, Cloud Composer, GCS, IAM, Cloud Run)Build and manage pipelines using Airflow and write advanced SQL queriesWork with Kafka, Spark, FastAPI, Redis / MongoDB / BigtableParticipate in CI / CD deployments using GitHubEnsure performance optimization, monitoring, logging, and data consistencyCreate architecture diagrams, design documents, and support post-delivery activitiesMandatory Skills :
10+ years of hands-on experience in Python (backend or data engineering)Strong experience with Google Cloud Platform (Dataflow, BigQuery, Cloud Composer, Cloud Functions, GCS, IAM, Cloud Run)Experience designing scalable ETL / ELT pipelinesStrong SQL skills (SQL Server / Oracle / PostgreSQL)Hands-on experience with Apache Spark & KafkaExperience with Airflow / GCP Composer DAGsExperience with FastAPIProficiency in GitHub & CI / CD workflowsExperience with NoSQL technologies (MongoDB / Redis / Bigtable)Experience deploying solutions on GKE & Cloud RunGood to Have (Optional) :
Snowflake experienceDatabricks (PySpark)Azure Data Factory or Azure data engineering tools