Senior Data Pipeline Engineer
InfosysBengaluru, Republic Of India, IN30+ days ago
Job descriptionAWS Glue, AWS Redshift / Spectrum, S3, API Gateway, Athena, Step and Lambda functions Experience in Extract Transform Load (ETL) and Extract Load & Transform (ELT) data integration pattern. Experience in designing and building data pipelines. Development experience in one or more object-oriented programming languages, preferably Python 5+ years of in depth hands on experience of developing, testing, deployment and debugging of Spark Jobs using Scala in Hadoop Platform In depth knowledge of Spark Core, working with RDDs, Spark SQL In depth knowledge on Spark Optimization Techniques and Best practices Good Knowledge of Scala Functional Programming : Try, Option, Future, Collections Good Knowledge of Scala OOPS : Classes, Traits and Objects (Singleton and Companion), Case Classes Good Understanding of Scala Language Features : Type System, Implicit / Givens Hands on experience of working in Hadoop Environment (HDFS / Hive), AWS S3, EMR Python programming skills Working experience on Workflow Orchestration tools like Airflow, Oozie Working with API calls in Scala Understanding and exposure to file formats such as Apache AVRO, Parquet, JSON Good to have knowledge of Protocol Buffers and Geospatial data analytics. Writing Test cases using frameworks such as scalatest. Good Knowledge of Build Tools such as : Gradle & SBT in depth Experience on using GIT, resolving conflicts, working with branches. Good to have worked on some workflow systems as Airflow Strong programming skills using data structures and algorithms. Excellent analytical skills Good communication skills
As a Software Developer you will work in a constantly evolving environment, due to technological advances and the strategic direction of the organization you work for. You will create, maintain, audit, and improve systems to meet particular needs, often as advised by a systems analyst or architect, testing both hard and software systems to diagnose and resolve system faults. The role also covers writing diagnostic programs and designing and writing code for operating systems and software to ensure efficiency. When required, you will make recommendations for future developments
Benefits of Joining Us
- Challenging Projects : Work on cutting-edge projects and solve complex technical problems.
- Career Growth : Advance your career quickly and take on leadership roles.
- Mentorship : Learn from experienced mentors and industry experts.
- Global Opportunities : Work with clients from around the world and gain international experience.
- Competitive Compensation : Receive attractive compensation packages and benefits.
- If you're passionate about technology and want to work on challenging projects with a talented team, becoming an Infosys Power Programmer could be a great career choice.
Mandatory Skills
Job Specs
Qualification
7-10 Yrs in the industry
BE / B.Tech CS or equivalent
Create a job alert for this search
Senior Data Engineer • Bengaluru, Republic Of India, IN
Related jobs
- Promoted
Senior Data Engineer
AvathonBengaluru, Karnataka, India AI with a powerful platform that enables businesses to harness the full potential of their operational data.Our technology seamlessly integrates and contextualizes siloed datasets, providing a 360-...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Straivehosur, tamil nadu, in The ideal candidate is a strong software engineer with hands-on experience in Spark (3.You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver da...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticshosur, tamil nadu, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer - Python - TDD - XP Practices
CareerXperts ConsultingBengaluru, Karnataka, India This role focuses on delivering robust, data-driven solutions using modern agile technical practices — including Continuous Integration & Deployment (CI / CD), Test-Driven Development (TDD), and Extr...Show moreLast updated: 1 day ago
- Promoted
- New!
Senior Data Engineer
XebiaBengaluru, Karnataka, India Job Title : Senior Data Engineer.Notice Period : immediate - 15 days.We are looking for Data Engineer with hands on experience on Python , Pyspark , Hadoop Hive, HDFS, Oozie, Yarn.Design, develop a...Show moreLast updated: 17 hours ago
- Promoted
Data Engineer
Recrohosur, tamil nadu, in Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Python Data Engineer
iVoyanthosur, tamil nadu, in One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs.
we use Fast API).Data processing and reporting usin...Show moreLast updated: 12 days ago
- Promoted
Python Data Engineer
Dexian Indiahosur, tamil nadu, in Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer
SAIVA AIBengaluru, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
SGS & Cohosur, tamil nadu, in Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer (AWS / Databricks)
AccoladeBengaluru, Karnataka, India The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Sonata SoftwareBengaluru, Karnataka, India Experience building data pipelines.Highly proficient in python, Data frames.Experience with AWS Lambda, Events (SQS), API calls, Kinesis, DynamoDB.
Highly proficient in SQL Query.Snowflake or experi...Show moreLast updated: 20 days ago
- Promoted
Senior Data Engineer
Guidanz Inchosur, tamil nadu, in BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show moreLast updated: 1 day ago
- Promoted
Backend and Data Pipeline Engineer
JRD SystemsBangalore, IN Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Mastekhosur, tamil nadu, in We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Saransh Inchosur, tamil nadu, in Data Engineer (Contract-to-Hire | Remote).Full-Time / Fixed-Term Contract (12 months, extendable based on performance).We are seeking a highly skilled.
The ideal candidate will be hands-on, detail-o...Show moreLast updated: 22 days ago
- Promoted
- New!
Senior Data Engineer – ETL & Pipeline Development (7 to 12 yrs)
AIMLEAPBengaluru, IN Senior Data Engineer – ETL & Pipeline Development.Remote (Work from Home) / Bangalore / India.Tech / MCA / Computer Science / IT.
IT / Data / AI / LegalTech / Enterprise Solutions.Pandas, Airflow, o...Show moreLast updated: 21 hours ago
- Promoted
Senior Data Engineer
PRI Globalbangalore, karnataka, in Sr Data Engineer, Strong Python, Snowflake, SQL, Git / Confluence and Jira.Min 5+ Yrs in Snowflake (Must).Min 5+ Yrs in Python Development (Must).
Experienced Data management specialist responsible fo...Show moreLast updated: 1 day ago