Lead Data Engineer
People Prime Worldwidesecunderabad, telangana, in6 hours ago
Job descriptionPerform data ingestion from various sources and apply transformation and cleansing logic to ensure high-quality data delivery. Implement and enforce data quality checks, validation rules, and monitoring. Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver efficient data solutions. Manage version control using GitHub and participate in CI / CD pipeline deployments for data projects. Write complex SQL queries for data extraction and validation from relational databases such as SQL Server, Oracle, or PostgreSQL. Document pipeline designs, data flow diagrams, and operational support procedures. Required Skills : 10 to 12 years of hands-on experience in Python for backend or data engineering projects. Strong understanding and working experience with GCP cloud services (especially Dataflow, BigQuery, Cloud Functions, Cloud Composer, etc.). Solid understanding of data pipeline architecture, data integration, and transformation techniques. Experience in working with version control systems like GitHub and knowledge of CI / CD practices. Experience in Apache Spark, Kafka, Redis, Fast APIs, Airflow, GCP Composer DAGs. Strong experience in SQL with at least one enterprise database (SQL Server, Oracle, PostgreSQL, etc.). Experience in data migrations from on-premise data sources to Cloud platforms. Good to Have (Optional Skills) : Experience working with the Snowflake cloud data platform. Hands-on knowledge of Databricks for big data processing and analytics. Familiarity with Azure Data Factory (ADF) and other Azure data engineering tools
About the Company
Our client is a global digital transformation leader with a presence in over 50 countries. Specializing in Oracle Cloud, automation, data intelligence, and enterprise consulting, they help businesses modernize their systems and accelerate innovation at scale.
Job Description
Job Title : Lead Data Engineer
Experience : 10+ Years
Location : Hyderabad
Job Type : [Full-time ]
Job Overview :
We are looking for a skilled and motivated Lead Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for requirements gathering, designing, architecting the solution, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) & ELT data pipelines. The role involves working with customers directly, gathering requirements, discovery phase, designing, architecting the solution, using various GCP services, implementing data transformations, data ingestion, data quality, and consistency across systems, and post post-delivery support.
Experience Level : 10 to 12 years of relevant IT experience
Key Responsibilities :
- Design, develop, test, and maintain scalable ETL data pipelines using Python.
- Architect the enterprise solutions with various technologies like Kafka, multi-cloud services, auto-scaling using GKE, Load balancers, APIGEE proxy API management, DBT, using LLMs as needed in the solution, redaction of sensitive information, DLP (Data Loss Prevention) etc.
- Work extensively on Google Cloud Platform (GCP) services such as : ○ Dataflow for real-time and batch data processing
○ Cloud Functions for lightweight serverless compute ○ BigQuery for data warehousing and analytics
○ Cloud Composer for orchestration of data workflows (on Apache Airflow) ○ Google Cloud Storage (GCS) for managing data at scale
○ IAM for access control and security
○ Cloud Run for containerized applications Should have experience in the following areas : ○ API framework : Python FastAPI
○ Processing engine : Apache Spark ○ Messaging and streaming data processing : Kafka ○ Storage : MongoDB, Redis / Bigtable
○ Orchestration : Airflow ○ Experience in deployments in GKE, Cloud Run.
Create a job alert for this search
Lead Data Engineer • secunderabad, telangana, in
Related jobs
![Lead Data Engineer [T500-21385]](https://cdn-logos.talent.com/v1/logo/image?feedcode=jobrapido-pms&company_name=ANSR)
- Promoted
Lead Data Engineer
Saras Analyticssecunderabad, telangana, in We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Prudent Technologies and Consulting, Inc.Hyderabad, Telangana, India Seeking an experienced Senior Data Engineer with over 9 years of industry experience to architect, build, and manage advanced data pipelines and platforms in a cloud environment.The ideal candidate...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
DigitalzoneHyderabad, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 6 days ago
- Promoted
Tech Lead - Data Engineer
Recrohyderabad, telangana, in Data Engineering and Analytics team.The ideal candidate will be responsible for configuring, deploying, and maintaining Quantexa software on leading cloud platforms while leveraging their expertise...Show moreLast updated: 19 days ago
- Promoted
Lead Data Engineer
EverestDX Inchyderabad, telangana, in Everest DX – We are a Digital Platform Services company, headquartered in Stamford.Our Platform / Solution includes Orchestration, Intelligent operations with BOTs’, AI-powered analytics for Enterpri...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Quess Corp LimitedHyderabad, Telangana, India Mandatory Skills : Sql, Snowflake, Python, Matillion and Tableau.Design and build scalable data models, data pipelines, and ETL / ELT processes within the Snowflake environment.Write and optimize com...Show moreLast updated: 5 days ago
- Promoted
Lead Data Engineer
Cimpresshyderabad, telangana, in Our Team : Enterprise Business Solutions.Vista’s Enterprise Business Solutions (EBS) domain is working to make our company one of the most data-driven organizations to support Finance, Supply Chain,...Show moreLast updated: 6 days ago
- Promoted
Data Engineer
Response InformaticsHyderabad, Telangana, India Cloud & Infrastructure • AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.
Snowflake : Moderate understanding of Snowflake architecture.C...Show moreLast updated: 30+ days ago
- Promoted
- New!
Lead Data Engineer [T500-21385]
ANSRhyderabad, telangana, in ANSR is hiring for one of its clients.Headquartered in Berlin, Germany, 4flow provides consulting, software and services for logistics and supply chain management.
More than 1300 team members levera...Show moreLast updated: 6 hours ago
- Promoted
Data Engineer
Egenhyderabad, telangana, in We are looking for a skilled and motivated Lead Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team.The ideal candidate will...Show moreLast updated: 30+ days ago
- Promoted
Lead Data Engineer
S&P GlobalHyderabad, India Grade Level (for internal use) : .The Market Intelligence Industry Data Solutions business line provides data technology and services supporting acquisition, ingestion, content management, mastering,...Show moreLast updated: 1 day ago
- Promoted
Lead Data Engineer
Intellistaff Services Pvt. LtdHyderabad, Telangana, India Job Description : Team Lead Data Engineer (6-10 Years Experience) Location : India, Hyderabad Experience Required : 6-8 years Fulltime Key Responsibilities - Should be able to lead the team, work...Show moreLast updated: 3 days ago
- Promoted
Data Engineer
RecroHyderabad, Telangana, India Configure, deploy, and maintain.Design, implement, and optimize.Collaborate with cross-functional teams to ensure.Troubleshoot and resolve issues related to.
Required Skills & Qualifications : .Strong...Show moreLast updated: 30+ days ago
- Promoted
Lead Cloud Data Engineer
People Prime WorldwideHyderabad, Republic Of India, IN Specializing in Oracle Cloud, automation, data intelligence, and enterprise consulting, they help businesses modernize their systems and accelerate innovation at scale.
We are looking for a skilled ...Show moreLast updated: 6 days ago
- Promoted
Lead Data Engineer
Accordion Indiahyderabad, telangana, in Accordion is a global private equity-focused financial consulting firm specializing in driving value creation through services rooted in Data & Analytics and powered by technology.Accordion works a...Show moreLast updated: 3 days ago
- Promoted
Lead Data Pipeline Engineer
People Prime WorldwideHyderabad, Republic Of India, IN Specializing in Oracle Cloud, automation, data intelligence, and enterprise consulting, they help businesses modernize their systems and accelerate innovation at scale.
We are looking for a skilled ...Show moreLast updated: 6 days ago
- Promoted
Lead Data Engineer
ConfidentialHyderabad / Secunderabad, Telangana Experience building and optimizing data pipelines in a distributed environment.Experience supporting and working with cross-functional teams.
Proficiency working in Linux environment.Experience usin...Show moreLast updated: 30+ days ago
- Promoted
Lead Data Engineer
Info ServicesHyderabad, Republic Of India, IN Virtual + F2F technical interview.Must have strong experience with : .Scala, Spark, Databricks, Snowflake, AWS, Python, Data Security, Web Services, Big Data.
Please email your query to alvinp@infoser...Show moreLast updated: 19 days ago