Overview We are seeking a highly skilled Senior Data Engineer with 5+ years of experience in building scalable data solutions on the Azure ecosystem.
The ideal candidate will have strong hands-on expertise in Databricks, Spark (Java), modern data lake architectures, and end-to-end ETL development.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines using Azure and Databricks.
- Build efficient ingestion and transformation workflows using Databricks with Pyspark and Delta Lake
- Develop and optimize ETL pipelines using Java and SQL.
- Implement best practices for data modeling, data quality, governance, and pipeline performance.
- Work with relational and lakehouse databases including PostgreSQL and Delta Lake.
- Collaborate with cross-functional teams and support production workloads, troubleshooting, and optimization.
Required Skills & Experience :
5+ years of hands-on data engineering experience.Strong working knowledge of the Azure ecosystem, especially :
Azure DatabricksAzure Data Lake Storage (ADLS)Azure Functions of Azure Data Factory (optional but preferred)Expertise in Apache Spark with Java (PySpark).Advanced SQL skills including query optimization.Strong understanding of ETL / ELT methodologies and data warehousing concepts.Experience working with Delta Lake (schema evolution, ACID, time travel).Experience with PostgreSQL or similar databases.Strong problem-solving skills and attention to detail.Ability to communicate effectively with technical and non-technical teams.Proactive, self-driven, and able to work in fast-paced environments.Preferred Skills :
Experience with CI / CD tools (Azure DevOps, Git).Knowledge of infrastructure-as-code tools (Terraform).Exposure to streaming platforms (Kafka, Event Hub).Familiarity with data governance tools such as Purview.Skills : Azure Cloud,Java,Pyspark
MUST HAVE :
Skills : Azure Cloud,Java,Pyspark
Notice period - 0 to 15days only
(ref : hirist.tech)