Data Engineering
ConfidentialChennai, India9 days ago
Job descriptionStrong knowledge on Python and Pyspark Expectation is to have ability to write Pyspark scripts for developing data workflows. Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. Expectation is to have strong problem-solving and troubleshooting skills. Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. 3-7 years of experience in Data Engineer. Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks. Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.
Responsibilities :
- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
- Create the data integration and data diagram documentation.
- Lead the data validation, UAT and regression test for new data asset creation.
- Create and maintain data models, including schema design and optimization.
- Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.
Qualifications and Skills :
Skills Required
Pyspark, Hive, Hadoop, Hue, Apache Oozie, Databricks, Greenplum, Azure, Sql, Python
Create a job alert for this search
Data Engineering • Chennai, India
Related jobs
JMAN Group is a fast-growing data engineering & data science consultancy.We work primarily with Private Equity Funds and their Portfolio Companies to create commercial value using Data & Artificial...Show moreLast updated: 7 days ago
- Promoted
Data Engineering Role
100x.incChennai, IN At least 3 years of professional experience in Data Engineering.Demonstrated end-to-end ownership of ETL pipelines.Deep, hands-on experience with AWS services : EC2, Athena, Lambda, and Step Functio...Show moreLast updated: 30+ days ago
- Promoted
Vice President, Data Engineering & Architecture
Three AcrossChennai, IN Role : - Vice President – Data Engineering & Architecture.We are seeking a hands-on and visionary leader to serve as Vice President – Data Engineering & Architecture.
This role will drive digital tran...Show moreLast updated: 4 days ago
- Promoted
Senior Data Integration Engineer
IndsafriChennai, IN Senior Data Integration Engineer.We are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer intell...Show moreLast updated: 26 days ago
- Promoted
Lead, Data Engineering
Standard Chartered BankChennai, Tamil Nadu, India This job is with Standard Chartered Bank, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community.
Please do not contact the recruiter directly....Show moreLast updated: 14 days ago
- Promoted
Data Engineering
EXLChennai, Tamil Nadu, India Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead the data validation...Show moreLast updated: 30+ days ago
- Promoted
Engineering Lead - Cloud & Data Engineering
Nazztec Private LimitedChennai Job Title : Engineering Lead Cloud & Data Engineer Work Base Location : Chennai / Hyderabad / Pune Experience Ra...Show moreLast updated: 30+ days ago
- Promoted
Associate Director - Data Engineering
BrownloopChennai, IN Brownloop is a US-based fintech company focused on leveraging data, analytics, and AI solutions to help Private Equity firms make smarter investment decisions.
We work with some of the largest and m...Show moreLast updated: 1 day ago
Data Engineering Lead
JMAN GroupChennai, IndiaQuick Apply
- Promoted
Engineering Manager - II, Data Engineering Platform
TamaraChennai, IN Data Engineering Manager II : Real-Time Data & Experimentation Platform (Remote, India).Saudi Arabia's first fintech unicorn.
GCC, with a mission to empower dreams through customer-centric financial ...Show moreLast updated: 6 days ago
- Promoted
Engineering Manager - Data Platform
CoinbaseChennai, IN Ready to be pushed beyond what you think you’re capable of?.At Coinbase, our mission is to increase economic freedom in the world.
It’s a massive, ambitious opportunity that demands the best of us, ...Show moreLast updated: 20 days ago
- Promoted
Developer - Data Engineering & Cloud Analytics
Geakminds Technologies Private LimitedChennai Developer - Data Engineering & Cloud Analytics.Role Overview : Responsible for building, maintaining, and optimizing large-scale data pipelines and analytics solutions lev...Show moreLast updated: 28 days ago
- Promoted
Publicis Sapient - Data Engineering Manager / Architect
Publicis SapientChennai Manager Data Engineer As Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing GCP / Azure ...Show moreLast updated: 30+ days ago
- Promoted
Manager, Data Engineering
ConfidentialChennai Manage, mentor, and guide a team of data engineers, ensuring their professional development and optimizing team performance.
Foster a culture of collaboration, accountability, and continuous learnin...Show moreLast updated: 30+ days ago
- Promoted
Data Engineering Analyst
ConfidentialChennai, India Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives.The work you do with our team will directly improve health outcomes by connect...Show moreLast updated: 9 days ago
- Promoted
Principal Engineer - Data Science
Pull LogicChennai, IN Pull Logic is transforming the retail supply chain with a revolutionary availability-oriented paradigm that directly links customer preferences with real-time inventory.
Backed by cutting-edge resea...Show moreLast updated: 2 days ago
- Promoted
Project Manager – Data Engineering & Analytics
BrillioChennai, IN We are looking for a skilled Technical Project Manager to lead and deliver projects in data engineering and analytics.You will manage cross-functional teams to execute data platform, pipeline, and ...Show moreLast updated: 30+ days ago
- Promoted
Data Engineering Expert – Financial Data Pipeline Development
Elongo Global Solutionschennai, tamil nadu, in This role is crucial in enabling seamless integration and real-time flow of financial data, which will later power .If you have hands-on experience in .
Azure Functions, Kafka, MS Fabric.You’ll lead...Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineering
ConfidentialChennai, India Logitech is the Sweet Spot for people who want their actions to have a positive global impact while having the flexibility to do it in their own way.
We are looking for a candidate to join our team ...Show moreLast updated: 9 days ago