Senior Data Pipeline EngineerEXL • Chennai, Republic Of India, IN
Senior Data Pipeline Engineer
EXL • Chennai, Republic Of India, IN
2 days ago
Job descriptionStrong knowledge on Python and Pyspark Expectation is to have ability to write Pyspark scripts for developing data workflows. Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. Expectation is to have strong problem-solving and troubleshooting skills. Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. 3-7 years of experience in Data Engineer. Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. Experience in GCP computing platform, such as developing ETL processes Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.
Responsibilities :
- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
- Create the data integration and data diagram documentation.
- Lead the data validation, UAT and regression test for new data asset creation.
- Create and maintain data models, including schema design and optimization.
- Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.
Qualifications and Skills :
Create a job alert for this search
Senior Data Engineer • Chennai, Republic Of India, IN
Related jobs
Senior Data Pipeline Engineer
HNR Tech • Republic Of India, IN
We are seeking an experienced Senior Data Engineer to design, build, and optimize our modern data ecosystem.The ideal candidate will have deep expertise in data architecture, pipeline orchestration...Show more
Last updated: 5 days ago • Promoted
Lead Data Pipeline Engineer
ListenFirst • Jaipur, Republic Of India, IN
We are looking for a skilled and motivated Senior Data Engineer with strong experience in data engineering and ETL processes.
The ideal candidate should be comfortable working with any object-orient...Show more
Last updated: 30+ days ago • Promoted
Senior Data Pipeline Engineer
Sutherland • Republic Of India, IN
Location : Bangalore | Hyderabad | Chennai | Mumbai.If interested please reach out to me with your updated resume @priti.
We’re transforming how we work by harnessing the power of AI, data, and digit...Show more
Last updated: 22 days ago • Promoted
Lead Data Pipeline Engineer
EXL • Pune, Republic Of India, IN
We are seeking a Senior Data Engineer to build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging.You will...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Engineer
IntraEdge • Republic Of India, IN
Python, PySpark, AWS services (Glue, Lambda), and Snowflake.The ideal candidate will design, build, and maintain scalable data pipelines, ensure efficient data integration, and enable advanced anal...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
Eucloid Data Solutions • Chennai, Republic Of India, IN
Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure for our cli...Show more
Last updated: 20 days ago • Promoted
Data Pipeline Engineer
Luxoft • Chennai, Republic Of India, IN
This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.
The goal is to enhance data access...Show more
Last updated: 1 day ago • Promoted
Data Pipeline Engineer
JMAN Group • Chennai, Republic Of India, IN
JMAN Group is a fast-growing data engineering & data science consultancy.We work primarily with Private Equity Funds and their Portfolio Companies to create commercial value using Data & Artificial...Show more
Last updated: 23 days ago • Promoted
Lead Data Pipeline Engineer
Persistent Systems • Pune, Republic Of India, IN
We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services.
The ideal candidate will have a strong backgro...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Engineer
UST • Pune, Republic Of India, IN
The ideal candidate should demonstrate a deep understanding of big data concepts, programming fundamentals, and the ability to solve complex problems related to scalability, failure handling, and o...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Engineer
EXL • Pune, Republic Of India, IN
DWH / ETL Developer : Strong experience in ETL development and data warehousing (SQL, Informatica).Responsibilities for Candidates.
Design, develop, and maintain ETL workflows ( Informatica ).Optimize ...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Engineer
Jaipur Living • Jaipur, Republic Of India, IN
At Jaipur Living, we blend time-honored craftsmanship with modern design to create luxurious rugs and home decor.Headquartered in Acworth, Georgia, we serve global customers with a mission-driven a...Show more
Last updated: 5 days ago • Promoted
Lead Data Pipeline Engineer
MakeMyTrip • Republic Of India, IN
At MakeMyTrip (MMT), technology is at the heart of everything we do.As a leading player in the travel industry, we leverage cutting-edge solutions like AI, machine learning, and cloud infrastructur...Show more
Last updated: 23 days ago • Promoted
Senior Data Pipeline Engineer
Searce Inc • Pune, Republic Of India, IN
Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’.It signifies our way of working : To improve to the finest degree of excellence, ‘solving for better’ every time.Searcians...Show more
Last updated: 8 days ago • Promoted
Data Pipeline Engineer
Yubi • Republic Of India, IN
As a Data Engineer, you will be part of a highly talented Data Engineering team.Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines.You...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
SGS & Co • Chennai, Republic Of India, IN
Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show more
Last updated: 23 days ago • Promoted
Lead Data Pipeline Engineer
Saks Global India • Republic Of India, IN
Data Engineer and Operations Lead.Saks Global is the largest multi-brand luxury retailer in the world, comprising Saks Fifth Avenue, Neiman Marcus, Bergdorf Goodman, Saks OFF 5TH, Last Call and Hor...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
RapidBrains • Republic Of India, IN
Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show more
Last updated: 13 days ago • Promoted