Senior Data Engineer

Senior Data Engineers

Experience - 6 +Yrs (5-7 relevant years)

Location : Trivandrum

Mandatory Skill - Pyspark, SQL, Azure Databricks

UK Shift Timing

Job Description

Key Accountabilities / Responsibilities :

• Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to End pipeline design, and code quality.
• Review and optimize PySpark , SQL , and Databricks code for performance, scalability, and maintainability.
• Offer engineering support and mentorship to data engineering teams within delivery squads, guiding them in building robust, reusable, and secure data solutions.
• Collaborate with architects to define data ingestion, transformation, and storage strategies leveraging Azure services such as Azure Data Factory , Azure Databricks, Azure Data Lakes
• Drive automation and CI / CD practices in data pipelines using tools such as Git, Azure DevOps , and DBT (good to have).
• Ensure optimal data quality and lineage by implementing proper testing, validation, and monitoring mechanisms within data pipelines.
• Stay current with evolving data technologies, tools, and best practices, continuously improving standards, frameworks, and engineering methodologies.
• Troubleshoot complex data issues, analyse system performance, and provide solutions to development and service challenges.
• Coach, mentor, and support team members through knowledge sharing sessions, technical reviews.

Required Skills & Experience :