This job offer is not available in your country.
Module Lead Data Engineer
ConfidentialBengaluru / Bangalore, India9 days ago
Job descriptionDevelop efficient ETL pipelines as per business requirements, following the development standards and best practices. Perform integration testing of different created pipeline in AWS env. Provide estimates for development, testing & deployments on different env. Participate in code peer reviews to ensure our applications comply with best practices. Create cost effective AWS pipeline with required AWS services i.e S3,IAM, Glue, EMR, Redshift etc.
Job Description
- 6-8 years of good hands on exposure with Big Data technologies – pySpark (Data frame and SparkSQL), Hadoop, and Hive
- Good hands on experience of python and Bash Scripts
- Good understanding of SQL and data warehouse concepts
- Strong analytical, problem-solving, data analysis and research skills
- Demonstrable ability to think outside of the box and not be dependent on readily available tools
- Excellent communication, presentation and interpersonal skills are a must
- Hands-on experience with using Cloud Platform provided Big Data technologies (i.e. IAM, Glue, EMR, RedShift, S3, Kinesis)
- Orchestration with Airflow and Any job scheduler experience
- Experience in migrating workload from on-premise to cloud and cloud to cloud migrations
Roles & Responsibilities
Show more
Show less
Skills Required
Airflow, Hive, Hadoop, Sparksql, Python, Sql
Create a job alert for this search
Lead Data Engineer • Bengaluru / Bangalore, India
Related jobs
For an international project in Chennai, we are urgently looking for a Full Remote SeniorAzure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databrick...Show moreLast updated: 7 days ago
- Promoted
Director - Demand Generation
Intuitive.Cloudhosur, tamil nadu, in With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization.
Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 18 days ago
- Promoted
Senior Azure Data Engineer - Full remote - contractor in USD
All European Careershosur, tamil nadu, inRemote
- Promoted
- New!
Azure Data Engineer Lead
Creative Synergies GroupBengaluru, Karnataka, India Creative Synergies Group Overview.Creative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and D...Show moreLast updated: 4 hours ago
- Promoted
ETL LEAD with IBM DataStage experienced
IntraEdgehosur, tamil nadu, in ETL Developer – DataStage, AWS, Snowflake.We are looking for a talented and motivated ETL Developer / Senior Developer.You will work on building scalable and efficient data pipelines using.IBM Data...Show moreLast updated: 30+ days ago
- Promoted
Senior Manager - Data Engineering Lead
DIAGEO Indiahosur, tamil nadu, in Senior Manager - Data Engineering Lead.Bachelor’s or master’s degree in computer science, Data Engineering, or related field.
Experience in data engineering.Proven experience in cloud platforms (AWS...Show moreLast updated: 26 days ago
- Promoted
Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location
USThosur, tamil nadu, in CTC, notice period, and current location details to.Apache Hadoop, Airflow, Kubernetes, and Containers.Troubleshoot Hadoop log files and work with.
Required Skills & Qualifications : .Scala, Spark, Py...Show moreLast updated: 30+ days ago
- Promoted
Azure Data Engineer
Tata Consultancy Serviceshosur, tamil nadu, in Job Title : Azure Data Engineer.Required Skillset : Azure Databricks(Strong),scala(strong), Pyspark(Good to have).Azure Databricks using Scala, PySpark.
Software Development Life Cycle experience.Work...Show moreLast updated: 30+ days ago
- Promoted
- New!
Lead Data Engineer
ThinkWise Consulting LLPBengaluru, Karnataka, India We are seeking a Lead Data Engineer to serve as the technical lead for the team responsible for managing and engineering customer data.
The ideal candidate will have a strong background in data engi...Show moreLast updated: 4 hours ago
- Promoted
- New!
Data Engineer
TechVeritohosur, tamil nadu, in You will play a critical role in designing, building, and optimizing data workflows that enable scalable analytics and real-time insights.
The ideal candidate is hands-on, detail-oriented, and passi...Show moreLast updated: less than 1 hour ago
- Promoted
Data / ML Engineer
GleantapBengaluru, IN Gleantap is a customer engagement platform powering fitness, wellness, and service businesses.Python (scikit-learn, LightGBM, XGBoost).
Set up retraining and monitoring pipelines (Airflow, MLflow, o...Show moreLast updated: 22 days ago
- Promoted
Data Engineer
INFEC Serviceshosur, tamil nadu, in Design, develop, and optimize data pipelines and ETL processes on GCP or Azure.Work with structured and unstructured data, integrating sources such as databases, APIs, and streaming platforms.Imple...Show moreLast updated: 4 days ago
- Promoted
Reporting Leader (Big Data / MES)
Tata ElectronicsKolar, Karnataka, India Power BI, Tableau, and Grafana.Deliver multiple output format reports meeting OSAT customer needs.Develop portals for internal and external users to access reports with right level of access contro...Show moreLast updated: 7 days ago
- Promoted
Data Engineer
Insight Globalhosur, tamil nadu, in Notice Period : This is a ASAP hire - only apply if you can start 2 weeks after offer is extended!.Priority scheduling for candidates who message on LinkedIn with resume.
Design, develop, and maintai...Show moreLast updated: 30+ days ago
- Promoted
Lead Azure Data Engineer
RandomTreesBangalore, IN We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies.
If you're passionate about building r...Show moreLast updated: 26 days ago
- Promoted
- New!
Lead Data Engineer & ML Analyst
Eltropybangalore, karnataka, in We’re looking for someone with.Design and manage scalable ETL / ELT pipelines using AWS Glue, Redshift, S3, and Kafka / Kinesis.
Architect and implement data lake and warehouse solutions following best ...Show moreLast updated: 1 hour ago
- Promoted
Senior Data Engineer
CEShosur, tamil nadu, in As part of our data engineering team, you’ll build and maintain.This is a hands-on role where you’ll design and optimize modern data infrastructure, ensuring reliability, scalability, and performan...Show moreLast updated: 5 days ago
- Promoted
Lead Data Engineer
Eucloid Data Solutionshosur, tamil nadu, in Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 7 days ago
- Promoted
Senior Azure Data Engineer
Programmers.iohosur, tamil nadu, in Job Title : Senior Azure Data Engineer.Skills Required : Azure Synapse, Azure Data Factory, Azure Databricks, and ADLS Gen2, PySpark, SQL,Microsoft Fabrics.
We are seeking a highly skilled and experie...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Kumaran Systemshosur, tamil nadu, in Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 7 days ago
- Promoted
Data Integration & LLM Engineer
Chargebeehosur, tamil nadu, in We are seeking a highly motivated.This role is ideal for engineers who enjoy working at the intersection of.APIs, SaaS connectors, and ETL / ELT pipelines to ensure reliable and scalable data flows.B...Show moreLast updated: 7 days ago