Lead Data Engineer – Python & GCP
People Prime Worldwidegurgaon, haryana, in8 hours ago
Job description10+ years of experience in data engineering or backend development. Strong programming skills in Python . Hands-on experience with GCP (Dataflow, BigQuery, Cloud Functions, Cloud Composer). Experience with Kafka, Spark, Redis, Airflow, GKE, Cloud Run, and FastAPI . Excellent knowledge of SQL and relational databases. Experience with CI / CD and GitHub . Experience migrating data from on-premise systems to the cloud. Experience with Snowflake or Databricks . Knowledge of Azure Data Factory or other Azure data tools. Exposure to machine learning or LLMs in data solutions.
Job Title : Lead Data Engineer – Python & GCP
Location : Hyderabad
Exp : 10+ Years
Overview :
We are seeking an experienced Lead Data Engineer with strong expertise in Python and Google Cloud Platform (GCP) . You will design, develop, and manage scalable ETL / ELT data pipelines , work closely with clients to understand requirements, and build data solutions using modern cloud and open-source technologies.
Key Responsibilities :
- Design, build, and maintain scalable data pipelines using Python and GCP services .
- Work with GCP tools such as Dataflow, BigQuery, Cloud Composer, Cloud Functions, Cloud Run, GCS, and IAM .
- Implement data ingestion, transformation, and quality checks to ensure accurate, reliable data.
- Architect enterprise data solutions using tools like Kafka, Spark, Airflow, DBT, GKE, APIGEE, and DLP .
- Use FastAPI for API development and GitHub for version control.
- Deploy solutions using CI / CD pipelines and manage data workflows in Airflow / Cloud Composer .
- Write and optimize complex SQL queries (SQL Server, Oracle, PostgreSQL, etc.).
- Collaborate with data scientists and analysts to deliver efficient data solutions.
- Document data flows, pipelines, and operational procedures.
Required Skills & Experience :
Nice to Have :
Create a job alert for this search
Data Engineer Python • gurgaon, haryana, in
Related jobs
- Promoted
Data Engineer
Canopus Infosystems - A CMMI Level 3 CompanyDelhi, IN Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization.The ideal candidate should be capable of building data pipelines, performing web scra...Show moreLast updated: 30+ days ago
- Promoted
Python Data Engineer
iVoyantGhaziabad, IN One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs.
we use Fast API).Data processing and reporting usin...Show moreLast updated: 13 days ago
- Promoted
AWS Data Engineer
TerraGiGDelhi, IN Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 13 days ago
- Promoted
Senior Data Engineer
SAIVA AIDelhi, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
IntelliasDelhi, IN Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines.
Kafka + Flink / Spark Streaming).Python for data engin...Show moreLast updated: 2 days ago
- Promoted
Platform Engineer – Python / Databricks / Notebooks / Kubernetes
SyntasaDelhi, IN Platform Engineer – Python / Databricks / Notebooks / Kubernetes.Syntasa is seeking a high-caliber and dedicated.Syntasa Technologies India Private Limited.
This position offers an exciting opportuni...Show moreLast updated: 12 days ago
- Promoted
- New!
Data Engineer - GCP
People Prime Worldwidenew delhi, delhi, in Senior Data Engineer - GCP + Python.Our client is a trusted global innovator of IT and business services, present in 50+ countries.
They specialize in digital & IT modernization, consulting, managed...Show moreLast updated: 8 hours ago
- Promoted
- New!
Lead Data Engineer
Fornaxfaridabad, haryana, in If solving business challenges drives you.Fornax is a team of cross-functional individuals who solve critial business challenges using core concept of analytics, critical thinking.We are seeking a ...Show moreLast updated: 8 hours ago
- Promoted
Data Engineer
IntraEdgeDelhi, IN Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show moreLast updated: 30+ days ago
- Promoted
- New!
AWS Lead Data engineer
Tata Consultancy Servicesfaridabad, haryana, in In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new La...Show moreLast updated: 8 hours ago
- Promoted
- New!
SnapLogic ETL & Python Lead
Swift Strategic Solutions Incnew delhi, delhi, in Our technology services client is seeking multiple.These positions offer a strong potential for conversion to full-time employment upon completion of the initial contract period.Below are further d...Show moreLast updated: 8 hours ago
- Promoted
- New!
Senior Data Engineer
SII Group Indiaghaziabad, uttar pradesh, in Databricks, GCP, and Data Pipeline Development.The ideal candidate will have a deep understanding of.Design, develop, and optimize scalable.
Cloud Build, BigQuery, Firestore.Collaborate with cross-f...Show moreLast updated: 8 hours ago
- Promoted
Data Engineer
Tata Consultancy Servicesgurgaon, haryana, in Required Technical Skill Set -.Create and maintain optimal data pipeline architecture,.Assemble large, complex data sets that meet functional / non-functional business requirements.Identify, design...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
VeraxionDelhi, IN Python, Spark, DBT, and AWS-native services.Agile environment to deliver scalable, secure, and high-performance data solutions.
Python, DBT, and AWS services (Data Ops Live).Deliver end-to-end data ...Show moreLast updated: 13 days ago
- Promoted
- New!
Python Lead Developer
People Prime WorldwideDelhi, IN Our client is one of the world's fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems.
They helps customers in two ways : Working with the world’s leading ...Show moreLast updated: 12 hours ago
- Promoted
Data Engineer
Dexian Indiadelhi, delhi, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 2 days ago
- Promoted
Backend and Data Pipeline Engineer
JRD SystemsDelhi, IN Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticsgurgaon, haryana, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago