Talent.com
Data Pipeline Engineer
Data Pipeline EngineerAirtel Payments Bank • Haryāna, Republic Of India, IN
Data Pipeline Engineer

Data Pipeline Engineer

Airtel Payments Bank • Haryāna, Republic Of India, IN
5 days ago
Job description

About Us :

Airtel Payments Bank, India's first payments bank is a completely digital and paperless bank. The bank aims to take basic banking services to the doorstep of every Indian by leveraging Airtel's vast retail network in a quick and efficient manner.

  • At Airtel Payments Bank, we’re transforming the way banking operates in the country. Our core business is banking and we’ve set out to serve each unbanked and underserved Indian. Our products and technology aim to take basic banking services to the doorstep of every Indian. We are a fun-loving, energetic and fast growing company that breathes innovation. We encourage our people to push boundaries and evolve from skilled professionals of today to risk-taking entrepreneurs of tomorrow. We hire people from every realm and offer them opportunities that encourage individual and professional growth. We are always looking for people who are thinkers & doers;
  • people with passion, curiosity & conviction;

people who are eager tobreak away from conventional roles and do 'jobs never done before’.

About the Team :

We are a team of engineers and problem-solvers building scalable, modern data infrastructure. Our mission is to power intelligent decision-making through clean, reliable, and real-time data pipelines using technologies like Kafka, PySpark, Hadoop, Airflow, and AWS. If you love working with data at scale, building cloud-native solutions, and improving pipeline reliability, you'll thrive here.

Key Responsibilities :

  • Design, build, and maintain scalable ETL / ELT pipelines using Python, PySpark, and SQL.
  • Develop real-time data ingestion and streaming solutions using Apache Kafka and AWS Kinesis.
  • Leverage the Hadoop ecosystem and AWS EMR for distributed data processing.
  • Automate and orchestrate workflows using Apache Airflow (deployed via MWAA on AWS).
  • Build and expose data services and APIs using Flask, deployed via Nginx.
  • Implement centralized logging and monitoring with Filebeat and the ELK / Opensearch stack.
  • Work extensively with AWS services including S3, API Gateway, OpenSearch, Kinesis, and EMR.
  • Collaborate with data scientists, analysts, and platform engineers to ensure high-quality, accessible data.
  • Must-Have Skills :

  • 4–7 years of experience in data engineering.
  • Proficiency in Python and SQL (Experience with Oracle or similar RDBMS).
  • Good experience with Apache Kafka (producer / consumer architecture, stream processing concepts).
  • Hands-on with PySpark, Hadoop & Hive for big data processing.
  • Workflow orchestration using Apache Airflow (and optionally MWAA).
  • Building APIs with Flask and serving via Nginx.
  • Logging and observability using ELK Stack and Filebeat.
  • Good familiarity with AWS ecosystem : EMR, Kinesis, S3, OpenSearch, API Gateway, MWAA.
  • Good-to-Have (But not mandatory)

  • Experience with AWS Glue, Athena, and Redshift for serverless data processing and warehousing.
  • Familiarity with AWS Flink or other stream processing frameworks.
  • Exposure to AWS DMS (Data Migration Service) for database migrations and replication tasks.
  • Knowledge of AWS QuickSight for dashboarding and BI reporting.
  • Understanding of data lake architectures and event-driven processing on AWS
  • Why Join Us?

    Airtel Payments Bank is transforming from a digital-first bank to one of the largest Fintech company. There could not be a better time to join us and be a part of this incredible journey than now. We at Airtel payments bank don’t believe in all work and no play philosophy. For us, innovation is a way of life and we are a happy bunch of people who have built together an ecosystem that drives financial inclusion in the country by serving 300 million financially unbanked, underbanked, and underserved population of India. Some defining characteristics of life at Airtel Payments Bank are Responsibility, Agility, Collaboration and Entrepreneurial development : these also reflect in our core values that we fondly call RACE..

    Create a job alert for this search

    Data Pipeline Engineer • Haryāna, Republic Of India, IN

    Related jobs
    AWS Data Pipeline Engineer

    AWS Data Pipeline Engineer

    Recruin • Haryāna, Republic Of India, IN
    Design, develop, and maintain data pipelines using AWS services such as Glue, Lambda, Step Functions, EMR, Kinesis, and S3. Build and optimize data warehouses and data lakes on AWS (Redshift, Lake F...Show more
    Last updated: 3 days ago • Promoted
    Cloud Data Pipeline Specialist

    Cloud Data Pipeline Specialist

    Reqpedia • Haryāna, Republic Of India, IN
    Job Title : Azure Data Engineer.Minimum 6+ years of experience in a Cloud Data Engineering role.Hands-on experience with Azure Cloud data tools (ADF, SHIR,. Logic Apps, ADLS Gen2, Blob Storage), Data...Show more
    Last updated: 5 days ago • Promoted
    Data Engineer

    Data Engineer

    NAB • Haryāna, Republic Of India, IN
    Proficient in executing ETL processes between data environments.Proficient at coding in SQL (additional knowledge in Python and R is highly regarded). Proficient at creating analytics related docume...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    Terra Technology Circle Consulting Private Limited • Haryāna, Republic Of India, IN
    We are seeking a highly skilled and motivated.In this role, you will design, build, and optimize scalable data pipelines and architectures to support analytics, machine learning, and business intel...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Test Engineer

    Data Pipeline Test Engineer

    Tata Consultancy Services • Haryāna, Republic Of India, IN
    Required Skills and Experience : .IT experience with 3 years of ETL test experience.Strong testing background (leading, planning & execution of testing activities). Design, develop, maintain, execute ...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    GMG • Haryāna, Republic Of India, IN
    GMG is a global well-being company retailing, distributing and manufacturing a portfolio of leading international and home-grown brands across sport, food and health sectors.Its vision is to inspir...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    EXL • Haryāna, Republic Of India, IN
    We are looking for a Python & PySpark developer and data engineer who can design and build solutions for one of our Fortune 500 Client programs in the realm of Financial Master & Reference Data Man...Show more
    Last updated: 5 days ago • Promoted
    Principal Data Pipeline Engineer

    Principal Data Pipeline Engineer

    Antal International • Haryāna, Republic Of India, IN
    Udaipur 3 months then Gurugram, India.You’ll lead the development of high-performance data pipelines, ensure data integrity, and guide the engineering team with best practices and cutting-edge tech...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    TBO.COM • Haryāna, Republic Of India, IN
    TBO is a global platform that aims to simplify all buying and selling travel needs of travel partners across the world.The proprietary technology platform aims to simplify the demands of the comple...Show more
    Last updated: 3 days ago • Promoted
    Data Analyst

    Data Analyst

    EaseMyTrip.com • Haryana, India, India
    As a Data Analyst at EaseMyTrip.Your role involves harnessing large sets of data to identify trends, forecast demand, and optimize our services to enhance customer satisfaction.You will collaborate...Show more
    Last updated: 6 days ago • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    NAB • Haryāna, Republic Of India, IN
    Proficient in executing ETL processes between data environments.Proficient at coding in SQL (additional knowledge in Python and R is highly regarded). Proficient at creating analytics related docume...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    IGT Solutions • Haryāna, Republic Of India, IN
    We are seeking a highly skilled.Databricks, PySpark, ETL development, and SQL.The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and analytics...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    Sirius AI • Haryāna, Republic Of India, IN
    Sirius AI is a US headquartered AI Consulting services and products company with operations in India.Sirius AI focuses on Financial Services enterprises and solutions / services delivered across mu...Show more
    Last updated: 5 days ago • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    Impetus • Haryāna, Republic Of India, IN
    Lead Data Engineer – GCP (BigQuery - Composer - Python - PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a te...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    Azilen Technologies • Haryāna, Republic Of India, IN
    Manage and create design schema, SQL query tuning, and code review.Min 4+ years of professional experience in the field of data engineering with knowledge of the data platform and DWH development.D...Show more
    Last updated: 3 days ago • Promoted
    Principal Data Pipeline Engineer

    Principal Data Pipeline Engineer

    Pacific Data Integrators • Haryāna, Republic Of India, IN
    Shift time : Open to work in EST shift (5PM to 2AM IST).Lead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (ID...Show more
    Last updated: 5 days ago • Promoted
    Senior Data Pipeline Developer

    Senior Data Pipeline Developer

    Terra Technology Circle Consulting Private Limited • Haryāna, Republic Of India, IN
    We are seeking an experienced ETL Developer with strong hands on expertise in Spark,SQL,Databricks and snowflake and prior experience in pharmaceutical / life sciences domain.The ideal candidate will...Show more
    Last updated: 5 days ago • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    Amway • Haryāna, Republic Of India, IN
    The Data Platform team is responsible for the Global Data Platform built on Google Cloud Platform (GCP) technology.This platform, called the Analytics Capability Ecosystem (ACE), is focused on prov...Show more
    Last updated: 3 days ago • Promoted