Talent.com
This job offer is not available in your country.
Data Engineering Azure databricks

Data Engineering Azure databricks

EXLKochi, IN
1 day ago
Job description

Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.

Responsibilities :

  • Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
  • Create the data integration and data diagram documentation.
  • Lead the data validation, UAT and regression test for new data asset creation.
  • Create and maintain data models, including schema design and optimization.
  • Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.

Qualifications and Skills :

  • Strong knowledge on Python and Pyspark
  • Expectation is to have ability to write Pyspark scripts for developing data workflows.
  • Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum
  • Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.
  • Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.
  • Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.
  • Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.
  • Expectation is to have strong problem-solving and troubleshooting skills.
  • Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.
  • Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
  • 4-7 years of experience in Data Engineer.
  • Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.
  • Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.
  • Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.
  • Create a job alert for this search

    Data Engineering • Kochi, IN

    Related jobs
    • Promoted
    Azure Databricks

    Azure Databricks

    Tata Consultancy ServicesKochi, Kerala, India
    TCS present an excellent opportunity for Azure Databricks.Experience required : 7-10 Yrs.Hands-on experience of developing and operating ETL / ELT pipelines with Azure Data Factory, Azure Databricks ...Show moreLast updated: 5 days ago
    • Promoted
    Technical Project Manager-Data Engineering

    Technical Project Manager-Data Engineering

    Reflections Info SystemsKerala, India, India
    We are looking for 13+years experienced candidates for this role.Preferred Location : Cochin / Trivandrum / Chennai.Overall 13+ Years of Experience in IT with 6+ years of experience as a Technical Produ...Show moreLast updated: 6 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data SolutionsKochi, IN
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 9 days ago
    • Promoted
    Data Architect

    Data Architect

    InfoBeansKottayam, IN
    Data Architect – Snowflake & Matillion.Snowflake, Matillion, Data Modeling, SQL, Python / Bash, Cloud Platforms (AWS / Azure). Immediate to 15 days preferred.The ideal candidate is proactive, self-direc...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Manuh TechnologiesKottayam, IN
    Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineering Azure databricks

    Data Engineering Azure databricks

    EXLKottayam, IN
    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders. Work with stakeholders to unders...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer

    Data Engineer

    Otomeyt AIKochi, IN
    We are seeking a highly skilled 5+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.Kottayam, IN
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 27 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Insight GlobalKochi, IN
    Notice Period : This is a ASAP hire - only apply if you can start 2 weeks after offer is extended!.Priority scheduling for candidates who message on with resume. Design, develop, and maintain robust...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgeKochi, IN
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 30+ days ago
    • Promoted
    Azure AI Foundry Developer

    Azure AI Foundry Developer

    UndockedKottayam, IN
    At Undocked, we help companies excel in e-commerce by delivering bespoke optimizations and cutting-edge analytics.Our experiences in retail and supply chain product strategy, technology, and operat...Show moreLast updated: 9 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Otomeyt AIKottayam, IN
    We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 19 days ago
    • Promoted
    Principal Data Engineer

    Principal Data Engineer

    XebiaKottayam, IN
    We’re Hiring : Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week).Any Xebia Location (Hybrid – 3 days in office per week). Data Engineering with 4+ years team leadership...Show moreLast updated: 23 days ago
    • Promoted
    Lead Azure Data Engineer

    Lead Azure Data Engineer

    RandomTreesKochi, IN
    We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies. If you're passionate about building r...Show moreLast updated: 27 days ago
    • Promoted
    Senior Full Stack SDE with Data Engineering for Analytics

    Senior Full Stack SDE with Data Engineering for Analytics

    TruckmentumKochi, IN
    Truckmentum is seeking a Senior Full Stack Software Development Engineer (SDE) with deep data engineering experience to help us build cutting-edge software and data infrastructure for our AI-driven...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Engineering Manager-Big Data, Generative AI

    Senior Engineering Manager-Big Data, Generative AI

    Extreme NetworksKottayam, IN
    Over 50,000 customers globally trust our end-to-end, cloud-driven networking solutions.They rely on our top-rated services and support to accelerate their digital transformation efforts and deliver...Show moreLast updated: 9 days ago
    • Promoted
    Data Engineer

    Data Engineer

    HISH IT SERVICESKottayam, IN
    Location : Remote(Banglore,Chennai,Pune).Pay : 14LPA - 18 LPA(Based on Experience).Timings : A couple of hours overlap with EST, as the client is Canada-based (till 12AM IST).Start Date : 20th Octob...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Exult GlobalKochi, Kerala, India
    Proficient in Python (Including popular python packages e.Strong background in distributed data processing and storage (e. Large scale (TBs of data) data engineering skills - Model data, create prod...Show moreLast updated: 21 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran SystemsKochi, IN
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 9 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Velodata Global Pvt LtdKochi, Kerala, India
    Work Location - Trivandrum, Kochi& Remote.Note • • - Candidates are required to attend the final round of the interview.The selected candidate must work from the office during the first month.We are ...Show moreLast updated: 30+ days ago