Senior Data Pipeline EngineerEXL • Chennai, Republic Of India, IN
Senior Data Pipeline Engineer
EXL • Chennai, Republic Of India, IN
3 days ago
Job descriptionStrong knowledge on Python and Pyspark Expectation is to have ability to write Pyspark scripts for developing data workflows. Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. Expectation is to have strong problem-solving and troubleshooting skills. Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. 3-7 years of experience in Data Engineer. Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. Experience in GCP computing platform, such as developing ETL processes Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.
Responsibilities :
- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
- Create the data integration and data diagram documentation.
- Lead the data validation, UAT and regression test for new data asset creation.
- Create and maintain data models, including schema design and optimization.
- Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.
Qualifications and Skills :
Create a job alert for this search
Senior Data Engineer • Chennai, Republic Of India, IN
Related jobs
Senior Data Pipeline Engineer
HNR Tech • Republic Of India, IN
We are seeking an experienced Senior Data Engineer to design, build, and optimize our modern data ecosystem.The ideal candidate will have deep expertise in data architecture, pipeline orchestration...Show more
Last updated: 6 days ago • Promoted
Lead Data Pipeline Engineer
ListenFirst • Jaipur, Republic Of India, IN
We are looking for a skilled and motivated Senior Data Engineer with strong experience in data engineering and ETL processes.
The ideal candidate should be comfortable working with any object-orient...Show more
Last updated: 30+ days ago • Promoted
Senior Data Pipeline Engineer
TECHY GEEKS • Chennai, Republic Of India, IN
Qualifications - 5+ years of experience in Data Engineering or related roles.Strong proficiency in SQL, with hands-on experience in Snowflake or similar cloud data warehouses.Expertise in ETL dev...Show more
Last updated: 16 hours ago • Promoted • New!
Senior Data Pipeline Engineer
Sutherland • Republic Of India, IN
Location : Bangalore | Hyderabad | Chennai | Mumbai.If interested please reach out to me with your updated resume @priti.
We’re transforming how we work by harnessing the power of AI, data, and digit...Show more
Last updated: 23 days ago • Promoted
Lead Data Pipeline Engineer
EXL • Pune, Republic Of India, IN
Develop high quality, secure and scalable data pipelines using spark, Scala / Python / Java on Hadoop or object storage like MinIO.
Leverage technologies and solutions to innovate with increasingly larg...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
Eucloid Data Solutions • Chennai, Republic Of India, IN
Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure for our cli...Show more
Last updated: 21 days ago • Promoted
Data Pipeline Engineer
Luxoft • Chennai, Republic Of India, IN
This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.
The goal is to enhance data access...Show more
Last updated: 2 days ago • Promoted
Data Pipeline Engineer
JMAN Group • Chennai, Republic Of India, IN
JMAN Group is a fast-growing data engineering & data science consultancy.We work primarily with Private Equity Funds and their Portfolio Companies to create commercial value using Data & Artificial...Show more
Last updated: 23 days ago • Promoted
Lead Data Pipeline Engineer
Persistent Systems • Pune, Republic Of India, IN
We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services.
The ideal candidate will have a strong backgro...Show more
Last updated: 30+ days ago • Promoted
Data Pipeline Engineer
EXL • Pune, Republic Of India, IN
DWH / ETL Developer : Strong experience in ETL development and data warehousing (SQL, Informatica).Responsibilities for Candidates.
Design, develop, and maintain ETL workflows ( Informatica ).Optimize ...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
MakeMyTrip • Republic Of India, IN
At MakeMyTrip (MMT), technology is at the heart of everything we do.As a leading player in the travel industry, we leverage cutting-edge solutions like AI, machine learning, and cloud infrastructur...Show more
Last updated: 23 days ago • Promoted
Data Pipeline Engineer
HireAlpha • Pune, Republic Of India, IN
To enable data-driven strategic and operational decision making through extracting actionable insights from large datasets, performing statistical and advanced analytics to uncover trends and patte...Show more
Last updated: 16 hours ago • Promoted • New!
Data Pipeline Engineer (GCP)
Capgemini Invent • Pune, Republic Of India, IN
At Capgemini Invent, we believe difference drives change.As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to...Show more
Last updated: 1 day ago • Promoted
Data Pipeline Engineer
HCLTech • Pune, Republic Of India, IN
We’re Expanding Our Digital & Data Talent Pool.Are you a professional ready to take on enterprise-scale transformation journey? We are on a mass hiring drive and looking for experts who can bring i...Show more
Last updated: 6 days ago • Promoted
Senior Data Pipeline Engineer
Searce Inc • Pune, Republic Of India, IN
Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’.It signifies our way of working : To improve to the finest degree of excellence, ‘solving for better’ every time.Searcians...Show more
Last updated: 8 days ago • Promoted
Data Pipeline Engineer
Yubi • Republic Of India, IN
As a Data Engineer, you will be part of a highly talented Data Engineering team.Responsible for developing reusable capabilities and tools to automate various types of data processing pipelines.You...Show more
Last updated: 30+ days ago • Promoted
Lead Data Pipeline Engineer
SGS & Co • Chennai, Republic Of India, IN
Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show more
Last updated: 23 days ago • Promoted
Lead Data Pipeline Engineer
Saks Global India • Republic Of India, IN
Data Engineer and Operations Lead.Saks Global is the largest multi-brand luxury retailer in the world, comprising Saks Fifth Avenue, Neiman Marcus, Bergdorf Goodman, Saks OFF 5TH, Last Call and Hor...Show more
Last updated: 30+ days ago • Promoted