Talent.com
Data Engineer with Databricks Expertise
Data Engineer with Databricks ExpertiseHello World Tech Consulting • Eluru, IN
Data Engineer with Databricks Expertise

Data Engineer with Databricks Expertise

Hello World Tech Consulting • Eluru, IN
1 day ago
Job description

Job Summary

We are seeking a highly skilled Data Engineer with Databricks certifications and

extensive practical experience working with the Databricks Lakehouse platform and its

supporting capabilities. The ideal candidate will possess a robust understanding of AWS

services relevant to data engineering and demonstrate proficiency in data pipeline

development, programming, data modeling, and governance.

Key Responsibilities

  • Design and implement data pipelines using AWS and Databricks.
  • Manage and orchestrate data workflows, ensuring eEicient data ingestion

processes.

  • Utilize Delta Lake for data storage and management, ensuring secure data

    • sharing across platforms with Delta Sharing.

  • Implement data governance and access control using Unity Catalog.
  • Apply the Medallion architecture (Bronze, Silver, Gold layers) to projects.
  • Develop and manage ETL pipelines using AWS Glue.
  • Leverage Amazon S3 for data storage and management.
  • Utilize AWS Lambda and Step Functions for serverless computing in data

    • processing.

  • Apply data warehousing solutions and best practices in data transformation.
  • Monitor and implement data quality checks and governance practices.
  • Collaborate with data scientists, analysts, and other stakeholders in an Agile

    • environment.

  • Deliver large-scale data processing and analytics projects within cloud

    • environments.

    Required Skills and Qualifications

  • Databricks certifications with practical experience in the Databricks Lakehouse

    • platform.

  • Proficient in AWS services : Glue, S3, Lambda, Step Functions, and data

    • warehousing solutions.

  • Strong programming skills in Python, Scala, or Java.
  • Proficient in SQL for querying and manipulating data.
  • Experience in data modeling for Lakehouse architecture.
  • Knowledge of ETL best practices and data transformation techniques.
  • Understanding of data governance frameworks and practices.
  • Proven track record of delivering data engineering projects, particularly in cloud

    • environments.

  • Experience with large-scale data processing and analytics projects.
  • Ability to work in Agile teams and collaborate eEectively with various

    • stakeholders.

    Preferred Qualifications

  • Advanced AWS certifications.
  • Experience with additional cloud platforms (e.g., Azure).
  • Familiarity with other data engineering tools and frameworks.
    • Create a job alert for this search

    Data Engineer • Eluru, IN