Data Engineering
EXLDelhi, India23 days ago
Job descriptionStrong knowledge on Python and Pyspark Expectation is to have ability to write Pyspark scripts for developing data workflows. Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. Expectation is to have strong problem-solving and troubleshooting skills. Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. 3-7 years of experience in Data Engineer. Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks. Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.
Responsibilities :
- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
- Create the data integration and data diagram documentation.
- Lead the data validation, UAT and regression test for new data asset creation.
- Create and maintain data models, including schema design and optimization.
- Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.
Qualifications and Skills :
Create a job alert for this search
Data Engineering • Delhi, India
Related jobs
![Director of Data Engineering [T500-20958]](https://cdn-logos.talent.com/v1/logo/image?feedcode=disruptedcloud-in&company_name=Arch%20Global%20Services%20India)
![Engineer II - Data Engineering [T500-20681]](https://cdn-logos.talent.com/v1/logo/image?feedcode=disruptedcloud-in&company_name=lululemon)
)
![Senior Manager, Data Engineering [T500-20771]](https://cdn-logos.talent.com/v1/logo/image?feedcode=disruptedcloud-in&company_name=McDonald's)
- Promoted
Engineering Manager
FxConsultingDelhi, India About the Role : We are seeking an experienced Engineering Manager with a strong background in Big Data technologies to lead a high-performing team of data engineers.
The ideal candidate has deep tec...Show moreLast updated: 17 days ago
- Promoted
- New!
Director of Data Engineering [T500-20958]
Arch Global Services IndiaDelhi, India About Arch : Arch Capital Group Ltd.Arch) is a leading global insurer with operations in more than a dozen countries.We write insurance, reinsurance and mortgage insurance on a worldwide basis, and ...Show moreLast updated: 13 hours ago
- Promoted
AI Engineering Manager
Manifest AIDelhi, India You will be responsible for building, delivering, and scaling products that impact millions of users.This is a fast-moving environment, so speed, clarity, and ownership matter a lot.Team Leadership...Show moreLast updated: 5 days ago
- Promoted
Engineer II - Data Engineering [T500-20681]
lululemonDelhi, India About lululemon : lululemon is an innovative performance apparel company for yoga, running, training, and other athletic pursuits.
Setting the bar in technical fabrics and functional design, we creat...Show moreLast updated: 23 days ago
- Promoted
Data Engineer
HR PLACEMENT CONSULTANTS (HRPC)Delhi, India Domain / Vertical - Insurance and / or Finance (preferred).Qualifications Educational - B.BCA / MCA – IT or Computer Science.
Experience programming skills in both Python and SQL.Experience with infrast...Show moreLast updated: 8 days ago
- Promoted
Principal Data Engineer
AutodeskDelhi, India We are seeking a highly experienced Principal Engineer to lead the design, development, and evolution of our Batch Processing platform, which powers Autodesk’s Analytics Data Platform (ADP).This ro...Show moreLast updated: 6 days ago
- Promoted
Data Engineering Role
100x.incMeerut, IN At least 3 years of professional experience in Data Engineering.Demonstrated end-to-end ownership of ETL pipelines.Deep, hands-on experience with AWS services : EC2, Athena, Lambda, and Step Functio...Show moreLast updated: 27 days ago
- Promoted
Senior Data Integration Engineer
IndsafriGhaziabad, IN Senior Data Integration Engineer.We are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer intell...Show moreLast updated: 16 days ago
- Promoted
AI / ML & Data Engineer
Mindfire SolutionsDelhi, IN We are looking for an experienced AI / ML & Data Engineer to design, develop, and deploy scalable machine learning models and data infrastructure on AWS.
You will work closely with cross-functional te...Show moreLast updated: 8 days ago
- Promoted
Engineer I - Data Engineering [T500-21053]
lululemonDelhi, India About lululemon : lululemon is an innovative performance apparel company for yoga, running, training, and other athletic pursuits.
Setting the bar in technical fabrics and functional design, we creat...Show moreLast updated: 6 days ago
- Promoted
Senior Manager, Data Engineering [T500-20771]
McDonald'sDelhi, India About McDonald’s : One of the world’s largest employers with locations in more than 100 countries, McDonald’s Corporation has corporate opportunities in Hyderabad.
Our global offices serve as dynamic...Show moreLast updated: 17 days ago
- Promoted
Data Engineer with Snowflake Experience- Contract
Gravity Infosolutions, Inc.gurugram, uttar pradesh, in Data Engineer with Snowflake & Data Vault 2.The project involves setting up a new software tool to meet audit requirements and is therefore critical.
Most data to be migrated is already in Snowflake...Show moreLast updated: 6 days ago
- Promoted
Azure Architect (Data Engineering)
Polestar AnalyticsDelhi, India Job Title : Azure Architect – Analytics & Data Services.Location : Noida / Kolkata / Bengaluru.Lead end-to-end architecture design for data and analytics solutions on Azure.
Define and implement Azure-bas...Show moreLast updated: 23 days ago
- Promoted
Senior Data Engineer
Saransh Incnoida, delhi, in Data Engineer (Contract-to-Hire | Remote).Full-Time / Fixed-Term Contract (12 months, extendable based on performance).We are seeking a highly skilled.
The ideal candidate will be hands-on, detail-o...Show moreLast updated: 17 days ago
- Promoted
Engineering Manager - Bigdata
ScaleneWorks People Solutions LLPDelhi, India We’re Hiring : Senior Engineering Manager – Big Data & Cloud.An established travel technology company is looking for an experienced Senior Engineering Manager to lead our Big Data & Cloud Engineerin...Show moreLast updated: 23 days ago
- Promoted
AWS Data Architect- Leadership
QuantiphiDelhi, India AWS Data Architect- Leadership Experience : .We are seeking a highly skilled and experienced Technical Leader for our AWS Data Engineering practice.
The ideal candidate will be responsible for drivin...Show moreLast updated: 23 days ago
- Promoted
Engineer II - Data Engineering [T500-20333]
lululemonDelhi, India About lululemon : lululemon is an innovative performance apparel company for yoga, running, training, and other athletic pursuits.
Setting the bar in technical fabrics and functional design, we creat...Show moreLast updated: 23 days ago
- Promoted
Technical Project Manager-Data Engineering
Reflections Info SystemsDelhi, India We are looking for 13+years experienced candidates for this role.Mode : Hybrid Preferred Location : Cochin / Trivandrum / Chennai Primary Skills : Overall 13+ Years of Experience in IT with 6+ years of e...Show moreLast updated: 23 days ago