Data Engineer Lead

About the Role

We are seeking an experienced Data Engineer Lead to design, develop, and maintain scalable data solutions on Azure and Databricks as part of our enterprise data modernization initiatives. The ideal candidate will have a strong background in data pipeline development, data integration frameworks, and cloud-based data engineering, with deep expertise in tools such as Databricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica.

This role will lead the design and delivery of high-performance, governed data architectures for large-scale enterprise clients, driving data reliability, compliance, and analytics readiness.

Key Responsibilities

• Design and implement scalable and reusable data pipelines using Databricks (PySpark, Delta Lake) and Azure Data Factory, ensuring high throughput and low latency.
• Develop and maintain Common Data Platforms (CDP) and Medallion Architecture-based data lakes to consolidate legacy systems and improve data availability.
• Create and manage backend data services using Python, Spark, or C#, ensuring performance, reliability, and scalability.
• Develop data ingestion frameworks using metadata-driven and schema-drift–handling designs to support multi-source data integration.
• Lead data integration and migration projects using Talend, Ab Initio, Alteryx, and Informatica, modernizing traditional ETL workflows into cloud-native pipelines.
• Implement data quality, lineage, and governance frameworks (Unity Catalog, Data Stewardship, Data Catalog, MDM).
• Design and maintain GDPR / CCPA-compliant data processing and ensure adherence to global data privacy standards.
• Build and optimize Feature Store Data Marts to support AI / ML analytics teams and downstream BI consumers.
• Collaborate with cross-functional teams (Data Scientists, BI Developers, Product Owners) to define and deliver data-driven business outcomes.
• Mentor junior engineers in best practices of data engineering, CI / CD, and infrastructure-as-code deployments.
• Utilize Redis and caching mechanisms to optimize performance and reduce load latency in analytic workloads.
• Take end-to-end ownership of assigned modules ensuring timely delivery, documentation, and performance optimization.

Technical Environment

Required Qualifications

Preferred Qualifications

Why Join Us