This job offer is not available in your country.
Data Engineering Azure databricks
EXLBengaluru, IN2 days ago
Job descriptionStrong knowledge on Python and Pyspark Expectation is to have ability to write Pyspark scripts for developing data workflows. Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. Expectation is to have strong problem-solving and troubleshooting skills. Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. 4-7 years of experience in Data Engineer. Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks. Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.
Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.
Responsibilities :
- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
- Create the data integration and data diagram documentation.
- Lead the data validation, UAT and regression test for new data asset creation.
- Create and maintain data models, including schema design and optimization.
- Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.
Qualifications and Skills :
Create a job alert for this search
Data Engineering • Bengaluru, IN
Related jobs
For an international project in Chennai, we are urgently looking for a Full Remote SeniorAzure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databrick...Show moreLast updated: 9 days ago
- Promoted
Senior Azure Data Engineer - Full remote - contractor in USD
All European Careershosur, tamil nadu, inRemote
- Promoted
Senior Full Stack SDE with Data Engineering for Analytics
Truckmentumhosur, tamil nadu, in Truckmentum is seeking a Senior Full Stack Software Development Engineer (SDE) with deep data engineering experience to help us build cutting-edge software and data infrastructure for our AI-driven...Show moreLast updated: 30+ days ago
- Promoted
- New!
AZURE DATABRICKS
Response InformaticsBengaluru, Karnataka, India Alteryx, Databricks and Azure Data Engineering.Show moreLast updated: 1 hour ago
- Promoted
Azure Data Engineer
Tata Consultancy Serviceshosur, tamil nadu, in Job Title : Azure Data Engineer.Required Skillset : Azure Databricks(Strong),scala(strong), Pyspark(Good to have).Azure Databricks using Scala, PySpark.
Software Development Life Cycle experience.Work...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Manuh Technologieshosur, tamil nadu, in Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing.
Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 22 days ago
- Promoted
Senior Azure Data Engineer
Wissen InfotechBengaluru, Karnataka, India Azure Data Engineer Know anyone who is interested!! Share your resumes to adilakshmi.Wissen Infotech is hiring for Bangalore.
Job Title : Azure Data Engineer.Years of Experience : 8 plus years.Locatio...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
INFEC Serviceshosur, tamil nadu, in Design, develop, and optimize data pipelines and ETL processes on GCP or Azure.Work with structured and unstructured data, integrating sources such as databases, APIs, and streaming platforms.Imple...Show moreLast updated: 6 days ago
- Promoted
AWS Data engineer / Architect
Tata Consultancy ServicesBengaluru, Karnataka, India In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new La...Show moreLast updated: 20 days ago
- Promoted
Senior Manager - Data Engineering Lead
DIAGEO IndiaBengaluru, Karnataka, India Senior Manager - Data Engineering Lead.Bachelor’s or master’s degree in computer science, Data Engineering, or related field.
Experience in data engineering.Proven experience in cloud platforms (AWS...Show moreLast updated: 28 days ago
- Promoted
- New!
Azure Databricks Developer
Tata Consultancy ServicesBangalore Urban, Karnataka, India Job Title : Azure Databricks Developer.Notice Period : Immediate Joiners Preferred.Location : Bangalore / Hyderabad / Chennai / Pune / Indore ( Onsite ).
Troubleshoot, debug, and resolve issues in data...Show moreLast updated: 1 hour ago
- Promoted
Associate Architect - Data Engineering
Response InformaticsBengaluru, Karnataka, India We are seeking an experienced Data Architect to lead the transformation of enterprise data.Alteryx workflows into Azure Databricks.
Microsoft Azure ecosystem, including Azure.Data Factory, Databrick...Show moreLast updated: 27 days ago
- Promoted
Lead Azure Data Engineer
RandomTreesBangalore, IN We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies.
If you're passionate about building r...Show moreLast updated: 27 days ago
- Promoted
- New!
Snowflake Developer
Tata Consultancy ServicesGreater Bengaluru Area, India Greetings from TATA Consultancy Services!!.Must have at least 4- 8 years experience in implementing Data warehousing solutions with at least 2 year in Snowflake.
Hands-on experience with building pr...Show moreLast updated: 1 hour ago
- Promoted
Data Engineer
Kanerika Inchosur, tamil nadu, in Following are high level responsibilities that you will play but not limited to : .Analyze the Data Model and do GAP analysis with Business Requirements and Power BI.
Design and Model Power BI schema....Show moreLast updated: 20 days ago
- Promoted
AWS Data Architect- Leadership
QuantiphiBengaluru, Karnataka, India AWS Data Architect- Leadership.We are seeking a highly skilled and experienced Technical Leader for our AWS Data Engineering practice.
The ideal candidate will be responsible for driving Data Engine...Show moreLast updated: 30+ days ago
- Promoted
Analyst - Data Engineering
Response Informaticsbangalore, karnataka, in We are seeking a skilled and experienced Data Engineer to join our team and help build,.The ideal candidate will have deep.
Microsoft data engineering ecosystem, particularly leveraging tools such a...Show moreLast updated: 27 days ago
- Promoted
Senior Azure Data Engineer
Programmers.iohosur, tamil nadu, in Job Title : Senior Azure Data Engineer.Skills Required : Azure Synapse, Azure Data Factory, Azure Databricks, and ADLS Gen2, PySpark, SQL,Microsoft Fabrics.
We are seeking a highly skilled and experie...Show moreLast updated: 30+ days ago
- Promoted
Lead Data Engineer
Eucloid Data Solutionshosur, tamil nadu, in Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 9 days ago