Lead Data Pipeline Engineer
StraiveHyderabad, Republic Of India, IN30+ days ago
Job description5-8 years of experience as a Data Engineer . Strong hands-on experience with Apache Spark 3.X . Proficient in Scala, Python, and advanced SQL . Experience with streaming frameworks (e.G., Spark Structured Streaming, Kafka). Knowledge of orchestration tools (e.G., Apache Airflow). Experience with cloud platforms like AWS Glue, EMR, Athena, Redshift, and S3 is a plus. Solid understanding of software engineering fundamentals (Git, CI / CD).
Job Summary
We're looking for a Senior Data Engineer with 5-8 years of experience to build and maintain scalable, production-grade data pipelines. The ideal candidate is a strong software engineer with hands-on experience in Spark (3.X), Scala, SQL, and Python. You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver data products, and mentoring junior engineers.
Key Responsibilities
- Design and build data pipelines using Apache Spark 3.X, Scala, Python, and SQL .
- Tune Spark jobs for performance and cost.
- Ensure code quality with unit tests, CI / CD, and code reviews.
- Collaborate with platform and DevOps teams to deploy and monitor pipelines.
- Troubleshoot production issues and perform root-cause analysis.
Qualifications
Create a job alert for this search
Data Pipeline Engineer • Hyderabad, Republic Of India, IN
Related jobs
- Promoted
Lead Data Engineer
Saras AnalyticsHyderabad, Telangana, India We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show moreLast updated: 30+ days ago
- Promoted
Backend and Data Pipeline Engineer
JRD Systemssecunderabad, telangana, in Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
ValueLabsHyderabad, Republic Of India, IN We are seeking a talented Data Engineer to join our dynamic team at ValueLabs.As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that sup...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
Integrated Personnel Services LimitedHyderabad, Republic Of India, IN Design, develop, and maintain scalable.Work closely with business stakeholders, analysts, and data scientists to understand data requirements and deliver reliable solutions.Optimize ETL workflows f...Show moreLast updated: 12 days ago
- Promoted
Lead Data Pipeline Engineer
Saras AnalyticsHyderabad, Republic Of India, IN We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value.
Our suite of work spans extraction, transformatio...Show moreLast updated: 30+ days ago
- Promoted
Lead Data Pipeline Engineer
ValueLabsHyderabad, Republic Of India, IN We are seeking a Data Engineer with strong expertise in Data Engineering—including data modeling, ETL pipeline development, data governance, and Data Build Tool (DBT)—along with proficiency in Pyth...Show moreLast updated: 21 days ago
- Promoted
Data Pipeline Engineer
OptumHyderabad, Republic Of India, IN Support the full data engineering lifecycle including research, proof of concepts, design, development, testing, deployment, and maintenance of data management solutions.
Utilize knowledge of variou...Show moreLast updated: 21 days ago
- Promoted
Snowflake Data Pipeline Engineer
Tata Consultancy ServicesHyderabad, Republic Of India, IN Minimum of 3+ years of working experience with.Datawarehouse and data lake service.Obvious track record of designing and implementing.
Exposed to ETL / ELT and governance tools (incl.Experience within...Show moreLast updated: 21 days ago
- Promoted
Lead Data Pipeline Engineer
EXLHyderabad, Republic Of India, IN The role involves building and optimizing ETL / ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI / ML, and business intelligence use cases.AWS (S...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
TMUS Global SolutionsHyderabad, Republic Of India, IN NASDAQ : TMUS), headquartered in Bellevue, Washington, is America’s supercharged Un-carrier, connecting millions through its strong nationwide network and flagship brands, T-Mobile and Metro by T-Mo...Show moreLast updated: 25 days ago
- Promoted
Lead Data Pipeline Engineer
SID Information TechnologiesHyderabad, Republic Of India, IN Role : Senior Data Engineer (Python, Spark / Databricks, SQL, AWS).Hybrid (3 days / week in-office).S3, Glue, Redshift, EMR, Lambda.
Strong SQL skills for data transformation and analytics.Expertise in s...Show moreLast updated: 30+ days ago
- Promoted
Backend and Data Pipeline Engineer
JRD SystemsHyderabad, IN Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Response InformaticsHyderabad, Telangana, India AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture.
CICD - Terraform or CloudFo...Show moreLast updated: 20 days ago
- Promoted
Tech Lead - Data Engineer
RecroHyderabad, Telangana, India Data Engineering and Analytics team.The ideal candidate will be responsible for configuring, deploying, and maintaining Quantexa software on leading cloud platforms while leveraging their expertise...Show moreLast updated: 13 days ago
- Promoted
Data Pipeline Engineer
Prudent Technologies and Consulting, Inc.Hyderabad, Republic Of India, IN We are seeking a skilled Data Engineer with strong experience in Python, Snowflake, and AWS.The ideal candidate will be responsible for building and optimizing scalable data pipelines, integrating ...Show moreLast updated: 30+ days ago
- Promoted
Tech Lead - Data Engineer
RecroHyderabad, Telangana, India Data Engineering and Analytics team.The ideal candidate will be responsible for configuring, deploying, and maintaining Quantexa software on leading cloud platforms while leveraging their expertise...Show moreLast updated: 12 days ago
- Promoted
Lead Data Pipeline Engineer
McDonald'sHyderabad, Republic Of India, IN One of the world’s largest employers with locations in more than 100 countries, McDonald’s Corporation has corporate opportunities in Hyderabad.
Our global offices serve as dynamic innovation and op...Show moreLast updated: 30+ days ago
- Promoted
Data Pipeline Engineer
Tanla Platforms LimitedHyderabad, Republic Of India, IN We are seeking a skilled Data Engineer with 5–10 years of experience to join our dynamic Data.The ideal candidate will design, build, and optimize large-scale data pipelines and warehouses that ena...Show moreLast updated: 21 days ago