Lead Data Pipeline EngineerStraive • Hyderabad, Republic Of India, IN
Lead Data Pipeline Engineer
Straive • Hyderabad, Republic Of India, IN
30+ days ago
Job description5-8 years of experience as a Data Engineer . Strong hands-on experience with Apache Spark 3.X . Proficient in Scala, Python, and advanced SQL . Experience with streaming frameworks (e.G., Spark Structured Streaming, Kafka). Knowledge of orchestration tools (e.G., Apache Airflow). Experience with cloud platforms like AWS Glue, EMR, Athena, Redshift, and S3 is a plus. Solid understanding of software engineering fundamentals (Git, CI / CD).
Job Summary
We're looking for a Senior Data Engineer with 5-8 years of experience to build and maintain scalable, production-grade data pipelines. The ideal candidate is a strong software engineer with hands-on experience in Spark (3.X), Scala, SQL, and Python. You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver data products, and mentoring junior engineers.
Key Responsibilities
- Design and build data pipelines using Apache Spark 3.X, Scala, Python, and SQL .
- Tune Spark jobs for performance and cost.
- Ensure code quality with unit tests, CI / CD, and code reviews.
- Collaborate with platform and DevOps teams to deploy and monitor pipelines.
- Troubleshoot production issues and perform root-cause analysis.
Qualifications
Create a job alert for this search
Data Pipeline Engineer • Hyderabad, Republic Of India, IN
Related jobs
Lead Data Pipeline Engineer
Multiscale Technologies India • Hyderabad, Republic Of India, IN
Data Engineer, Forward Deployed Applications.Location : Hyderabad, India (on-site at client location).Role Type : Individual Contributor.
We are hiring a Data Engineer to build reliable data pipelines...Show more
Last updated: 20 days ago • Promoted
AI Engineer (Data Pipelines & RAG)
BeGig • Hyderabad, IN
Job Role- AI Engineer (Data Pipelines & RAG).Work Mode- Remote(6 days working).We are looking for a hands-on AI / Data Engineer (4–7 years) to build and scale data pipelines powering GenAI and agenti...Show more
Last updated: 10 days ago • Promoted
Cloud Data Pipeline Engineer
RapidBrains • Hyderabad, Republic Of India, IN
Design, develop, and optimize large-scale data analytics and data warehouse solutions using Azure Data Factory and Azure Databricks.
Build and maintain data lake architectures to process multi-forma...Show more
Last updated: 8 hours ago • Promoted • New!
Data Pipeline Engineer
HCLTech • Hyderabad, Republic Of India, IN
Data Engineer - Databricks Job Description.The Data Engineer - Databricks will be responsible for building and optimizing our data pipelines, architectures, and data sets.He will work closely with ...Show more
Last updated: 16 days ago • Promoted
Lead Data Pipeline Architect
EXL • Hyderabad, Republic Of India, IN
AWS (S3, Redshift, Glue, EMR, Lambda, Kinesis, Athena).Collaborate with data scientists, analysts, and stakeholders to deliver reliable and scalable data solutions.
Ensure high availability, scalabi...Show more
Last updated: 16 days ago • Promoted
Data Engineer
IntraEdge • Hyderabad, IN
We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team.
You will be responsible for building scalable and reli...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
Saras Analytics • Hyderabad, Republic Of India, IN
We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Engineer
Luxoft • Hyderabad, Republic Of India, IN
This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.
The goal is to enhance data access...Show more
Last updated: 2 days ago • Promoted
Lead Data Engineer
Accordion India • Hyderabad, Telangana, India
Accordion is a global private equity-focused financial consulting firm specializing in driving value creation through services rooted in Data & Analytics and powered by technology.Accordion works a...Show more
Last updated: 20 days ago • Promoted
Data Engineer Lead
JRD Systems • Hyderabad, Telangana, India
The ideal candidate will have a strong background in.Databricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica.
This role will lead the design and delivery of.Databricks (PySpark, ...Show more
Last updated: 8 days ago • Promoted
Principal Data Pipeline Engineer
EverestDX Inc • Hyderabad, Republic Of India, IN
Everest DX – We are a Digital Platform Services company, headquartered in Stamford.Our Platform / Solution includes Orchestration, Intelligent operations with BOTs’, AI-powered analytics for Enterpri...Show more
Last updated: 10 days ago • Promoted
Lead Data Engineer
nexocean • Hyderabad, Telangana, India
Job Description : We are excited to announce an opening for Lead Data Engineer ! Skills Required : SQL, Python, Snowflake, ADF, ETL, Data Pipeline Experience Range : 8 - 12 years Job Description...Show more
Last updated: 3 hours ago • Promoted • New!
Databricks Data Engineer Lead – Sustainability Project
Blue Cloud Softech Solutions Limited • Hyderabad, IN
BCSS is seeking a Databricks Data Engineer to support its enterprise-wide Sustainability initiative.The engineer will be responsible for building data pipelines and models to support product-level ...Show more
Last updated: 9 days ago • Promoted
Data Pipeline Engineer
Travel & Payments • Hyderabad, Republic Of India, IN
At Yuno, we are building the payment infrastructure that enables all companies to participate in the global market.Founded by a team of seasoned experts in the payments and IT industries, Yuno prov...Show more
Last updated: 30+ days ago • Promoted
Senior Data Pipeline Engineer
nexocean • Hyderabad, Republic Of India, IN
We are excited to announce an opening for Lead Data Engineer !.SQL, Python, Snowflake, ADF, ETL, Data Pipeline.Work closely with Product owners / managers to understand the business needs.Collaborate...Show more
Last updated: 8 hours ago • Promoted • New!
Data Pipeline Engineer
People Prime Worldwide • Hyderabad, Republic Of India, IN
Our client is a global technology company headquartered in Santa Clara, California.OT) and more than 60 years in IT to unlock the power of data from your business, your people and your machines.We ...Show more
Last updated: 30+ days ago • Promoted
Lead Data Engineer
ValueMomentum • Hyderabad, Telangana, India
Primary skills : Databricks, PySpark, SQL.Secondary skills : Advanced SQL, Azure Data Factory, and Azure Data Lake.About the Job We are seeking a Tech Lead –.
Databricks Data Engineer with experience ...Show more
Last updated: 7 hours ago • Promoted • New!
Lead Data Engineer
Saras Analytics • Hyderabad, Telangana, India
We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show more
Last updated: 30+ days ago • Promoted