Data Engineer

Role : Data Engineer

Experience : 5 to 7 Years

Location : Thiruvananthapuram

Work Mode : Hybrid

Notice Period : Immediate Joiners

Role Summary : Data Engineer

Key Responsibilities

1.Develop and manage Azure Data Factory (ADF) pipelines to ingest data from legacy and cloud systems into the Data Lake.

2.Build and optimize Databricks notebooks for ingestion and curation, enabling semantic views for reporting and event streaming to messaging queues.

3.Optimize workflows for performance and cost-efficiency, including job scheduling, cluster sizing, and PySpark / SQL code tuning.

4.Develop and manage Unity Catalog objects, ensuring proper access control and data organization.

5.Ensure compliance with PEP data governance and security standards across all data workflows.

6.Participate in code reviews, documentation, and DevOps deployments using Azure DevOps.

7.Integrate with various systems such as Snowflake, Salesforce, Oracle, SAP, ASQL, PostgreSQL, messaging queues, ADLS, API's and Unity Catalog for data ingestion and publishing.

8.Contribute to solution design, job optimization, and data workflow architecture.

9.Support migration efforts, including the transition of Teradata ETL logic to Azure Databricks and Unity Catalog.

Skills : Data Governance ,Azure Databricks ,Azure Data Factory ,Azure Devops

Communication skills should be very good