Senior Python Data Engineer – ETL & Pipeline Development (5 to 12 yrs)

AIMLEAPMangalore, IN

2 hours ago

Job description

Senior Python Data Engineer – ETL & Pipeline Development

Experience : 5 to 12 Years

Location : Remote (WFH) / Bangalore / India

Mode of Engagement : Full-time

No. of Positions : 5

Educational Qualification : B.E / B.Tech / M.Tech / MCA / Computer Science / IT

Industry : IT / Data / AI / LegalTech / Enterprise Solutions

Notice Period : Immediate Joiners Preferred

What We Are Looking For :

5–12 years of hands-on experience building end-to-end ETL pipelines and data workflows from scratch (Python + SQL).
Strong expertise in Python for data transformation, automation, and orchestration (Pandas, Airflow, custom schedulers).
Ability to write complex SQL queries, optimize performance, and design scalable database schemas.
Experience integrating and consuming RESTful APIs for ingestion, transformation, and delivery pipelines.
Working knowledge of AWS / GCP / Azure (S3, Lambda, EC2, BigQuery, etc.).
Hands-on exposure to Docker, Kubernetes, and CI / CD for deploying ETL & data workloads.
Experience building or working with AI-driven / automated data pipelines (preferred).

Responsibilities :

Design, architect, and build complete ETL pipelines for multi-system data extraction, cleaning, transformation, and loading.

Develop custom Python-based data automation frameworks to improve quality and speed.

Build and optimize SQL queries, tables, and database structures for analytics & AI use cases.

Create and manage API-based data ingestion / integration workflows.

Deploy and maintain workloads using Docker, Kubernetes, and CI / CD tools.

Collaborate with product, AI, and analytics teams to deliver intelligent data solutions.

Set up data validation, monitoring, logging, and alerting mechanisms for production pipelines.

Continuously optimize pipelines for performance, cost, scalability, and reliability.

Qualifications :

Bachelor’s or Master’s degree in Computer Science / Engineering / IT.

Expertise in Python (FastAPI / Flask / Django), SQL, and REST API development.

Strong understanding of data modeling, ETL concepts, and pipeline orchestration (Airflow / Prefect / Dagster).

Experience with AWS (S3, Lambda, EC2, Glue, Athena) or GCP / Azure equivalents.

Hands-on experience with Docker, Kubernetes, and Git-based CI / CD workflows.

Strong debugging, problem-solving, and analytical capabilities with ownership mindset.

Create a job alert for this search

Senior Data Engineer • Mangalore, IN

Related jobs

Promoted

Senior Data Engineer

SAIVA AIMangalore, IN

We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago

Promoted

Senior Python Developer (Programming)

Jobseekerspage.commangalore, karnataka, in

HIRING - SENIOR PYTHON DEVELOPER.Senior Python Developer (Programming).We are seeking an experienced Python Developer with strong programming skills (not scripting) to develop and optimize data con...Show moreLast updated: 3 days ago

Promoted

Lead ETL ODI

Vienna ConsultancyMangalore, IN

We are seeking a skilled and experienced Data Integration Specialist with over 9 years of experience in designing and developing data solutions using Oracle Data Integrator (ODI).The ideal candidat...Show moreLast updated: 6 days ago

Promoted

Senior Data Engineer

IntelliasMangalore, IN

Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines. Kafka + Flink / Spark Streaming).Python for data engin...Show moreLast updated: 19 days ago

Promoted

Senior Data Engineer

Live Connectionsmangalore, karnataka, in

Hiring : Data Engineer | 9-12 Years Experience | Remote (India).Python, SQL, Big Data, Cloud, Snowflake.We are looking for an experienced. The ideal candidate will have deep experience in.Python, SQL...Show moreLast updated: 30+ days ago

Promoted

ETL & Tableau Data Engineer

Magma ConsultancyMangalore, IN

ETL & Tableau Data Engineer (Remote | Full-Time | Join in 2 Weeks).We are a forward-thinking technology and data consulting firm dedicated to helping organizations unlock the full potential of thei...Show moreLast updated: 3 days ago

Promoted

Senior Data Engineer

Intellify SolutionsMangalore, IN

Total years of experience : 10+ years.Required : Bachelor’s degree or relevant experience.Required : Prior experience as a software engineer or data engineer in a. Required : Cloud Data Warehouse experi...Show moreLast updated: 10 days ago

Promoted

Senior AI / ML Engineer

LuxoftMangalore, IN

Our client, one of the leading Agriculture Companies, is modernising their landscape and adopting AI and innovations in their process. We are seeking a highly skilled and innovative Python / AI Engine...Show moreLast updated: 16 days ago

Promoted
New!

Senior Data Engineer

K&K Talentsmangalore, karnataka, in

Promoted

AWS Data Engineer

Vista Applied Solutions Group IncMangalore, IN

Job Summary for AWS Data Engineer : .Job Qualification and Responsibilities for AWS Data Engineer : .Advanced SQL (performance tuning, joins, windows functions). Hands-on with AWS : Glue, Lambda, Redshif...Show moreLast updated: 5 days ago

Promoted

Python Software Engineer

Teamware Solutionsmangalore, karnataka, in

Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt Ltd.Role : Python Software Engineer - IC.Location : Bangalore / Hyderabad / pune / Mumbai. Shift : (Flexibility Required) / ...Show moreLast updated: 1 day ago

Promoted

Python with AI Engineer

Infomatics CorpMangaluru, Karnataka, India

We are seeking a talented and experienced AI and Python Engineer to design, develop, and deploy cutting-edge phone integration solutions for our Agentic AI platform within the health insurance sect...Show moreLast updated: 17 days ago

Promoted

Senior Data Engineer

MindlanceMangalore, IN

We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show moreLast updated: 3 days ago

Promoted

Senior Python Data Engineer

iVoyantmangalore, karnataka, in

Join a dynamic engineering team working on a high-impact tax reporting platform for the 2025 fiscal season.The core goal is to modernize and significantly accelerate the generation of Excel-based r...Show moreLast updated: 6 days ago

Promoted

Data Engineer

IntraEdgeMangalore, IN

Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show moreLast updated: 30+ days ago

Promoted

Data Engineer

AceolutionMangalore, IN

Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show moreLast updated: 30+ days ago

Promoted
New!

Senior Python - FastAPI Development Lead (8+ Years)

MindBrainmangalore, karnataka, in

Hiring : Python / FastAPI Development Lead.Remote | 🧑‍💻 8+ Years Experience | ⏳ Duration : 2 Months (Extendable).Python / FastAPI Development Lead. The ideal candidate will own the full API lifecycle—...Show moreLast updated: less than 1 hour ago

Promoted

AWS Data Engineer

Tata Consultancy Servicesmangalore, karnataka, in

Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 27 days ago