Big Data Pipeline Engineer
TalentOnLeaseBengaluru, Republic Of India, IN2 days ago
Job description
Responsibilities :
- Big Data Orchestration : Airflow, Spark on Kubernetes, Yarn, Oozie
- Big Data Processing : Hadoop, Kafka, Spark & Spark Structured Streaming
- Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience
- Advanced Scala experience (e.G. Functional Programming, using Case classes, Complex Data Structures & Algorithms)
- Proficient in developing automated frameworks for unit & integration testing
- Proficient with Kubernetes, Docker, Helm and related container technologies
- Proficient in deploying and managing Spark workloads on Kubernetes clusters
- Candidates should have hands on experience in Spark and Scala at least 5+ years with good data engineering concepts
Create a job alert for this search
Data Pipeline Engineer • Bengaluru, Republic Of India, IN
Related jobs
- Promoted
Big Data Developer(Hive, Pyspark)
Tata Consultancy Serviceshosur, tamil nadu, in Big Data Developer(Hive, Pyspark).Years of Experience : 4 to 10 yrs.Date of Interview : 25th Oct, 2025.Mode of Interview : Virtual Drive( Cisco or WebEx).
Hands-on experience of Hadoop, Python, PySpark...Show moreLast updated: 18 days ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladehosur, tamil nadu, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
Python Data Engineer
Dexian Indiahosur, tamil nadu, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 5 days ago
- Promoted
Python Data Engineer
iVoyanthosur, tamil nadu, in One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs.
we use Fast API).Data processing and reporting usin...Show moreLast updated: 16 days ago
- Promoted
Data Engineer
Recrohosur, tamil nadu, in Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Big Data Developer
USThosur, tamil nadu, in The ideal candidate should demonstrate a deep understanding of big data concepts, programming fundamentals, and the ability to solve complex problems related to scalability, failure handling, and o...Show moreLast updated: 30+ days ago
- Promoted
Azure Data Engineer
LTIMindtreehosur, tamil nadu, in Role Senior Data Engineer 8 years of experience.Build reusable utilities templates and automation pipelines.Design scalable data engineering frameworks standards and best practices.Provide architec...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Tata Consultancy Serviceshosur, tamil nadu, in Required Technical Skill Set -.Create and maintain optimal data pipeline architecture,.Assemble large, complex data sets that meet functional / non-functional business requirements.Identify, design...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
VAANTECHhosur, tamil nadu, in Immediate Joiner (or within 15 days).Includes Night Shifts (US Shift).Preferred Candidates : From Chennai.Data Pipeline Optimization & Tuning.
Hadoop Infra / Cloud Platforms.Lead and mentor a team of...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Veraxionhosur, tamil nadu, in Python, Spark, DBT, and AWS-native services.Agile environment to deliver scalable, secure, and high-performance data solutions.
Python, DBT, and AWS services (Data Ops Live).Deliver end-to-end data ...Show moreLast updated: 16 days ago
- Promoted
Big Data Developer
Tata Consultancy Serviceshosur, tamil nadu, in Advanced working Scala ,SQL,Python / PySpark knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.Experience bu...Show moreLast updated: 30+ days ago
- Promoted
Freelance Data Engineer
upGradhosur, tamil nadu, in We are seeking a highly skilled and motivated.The ideal candidate will be responsible for designing, developing, and optimizing large-scale data pipelines and data warehouse solutions, utilizing a ...Show moreLast updated: 24 days ago
- Promoted
Senior Data Engineer
SAIVA AIhosur, tamil nadu, in We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Backend and Data Pipeline Engineer
JRD SystemsBengaluru, IN Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Engineer (Devops & Big Data)
AMISEQBengaluru, Karnataka, India You will be a member of our AI Platform Team, supporting the next generation AI architecture for various research and engineering teams within the organization.
You'll partner with vendors and the i...Show moreLast updated: 4 days ago
- Promoted
Data Engineer
LTIMindtreehosur, tamil nadu, in We are seeking a skilled Data Engineer with hands-on experience in.The ideal candidate will be responsible for developing scalable data pipelines, transforming and ingesting large-scale data, and e...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Response Informaticshosur, tamil nadu, in AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture.
CICD - Terraform or CloudFo...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
IntraEdgehosur, tamil nadu, in Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show moreLast updated: 30+ days ago