Talent.com
Lead I - Data Engineering

Lead I - Data Engineering

ConfidentialThiruvananthapuram, Thiruvananthapuram / Trivandrum, India
6 days ago
Job description

Overview

Overview We are seeking a highly skilled Senior Data Engineer with 5+ years of experience in building scalable data solutions on the Azure ecosystem. The ideal candidate will have strong hands-on expertise in Databricks, Spark (Java), modern data lake architectures, and end-to-end ETL development.

Mandatory Skills : Azure Cloud,Java,Pyspark

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure services and Databricks.
  • Build efficient data ingestion and transformation workflows using Databricks with PySpark and Delta Lake .
  • Develop and optimize ETL pipelines using Java and SQL .
  • Implement best practices for data modelling , data quality , governance , and pipeline performance .
  • Work with relational and lakehouse databases including PostgreSQL and Delta Lake .
  • Collaborate with cross-functional teams to support production workloads , including troubleshooting and performance tuning.

Required Skills & Experience

  • 5+ years of hands-on data engineering experience.
  • Strong working knowledge of the Azure ecosystem , particularly :
  • Azure Databricks
  • Azure Data Lake Storage (ADLS)
  • Azure Functions
  • Azure Data Factory (preferred)
  • Expertise in Apache Spark with Java and PySpark .
  • Advanced SQL skills, including query optimization.
  • Strong understanding of ETL / ELT methodologies and data warehousing concepts .
  • Experience with Delta Lake (schema evolution, ACID transactions, time travel).
  • Experience working with PostgreSQL or similar relational databases.
  • Strong analytical, problem-solving, and communication skills.
  • Ability to work proactively in a fast-paced environment.

    • Preferred Skills

  • Experience with CI / CD tools such as Azure DevOps and Git.
  • Knowledge of Infrastructure as Code (IaC) tools such as Terraform .
  • Exposure to streaming platforms like Kafka or Azure Event Hub .
  • Familiarity with data governance solutions such as Azure Purview .

    • If you meet the requirements outlined in the job description and are interested in this opportunity, please email your resume to [HIDDEN TEXT], mentioning your current CTC, expected CTC, and Notice Period.

    Skills Required

    Java, Pyspark, Postgresql, Apache Spark, Kafka, Sql, ELT, Git, Azure Data Factory, Azure Functions, Azure Cloud, Terraform, Spark, Databricks, Data Warehousing, Etl, Azure Devops

    Create a job alert for this search

    Data Engineering Lead • Thiruvananthapuram, Thiruvananthapuram / Trivandrum, India