Job Description
Role : Lead Data Engineer
Job Responsibilities
- To collaborate with various teams / regions in driving facilitating data design, identifying architectural risks and key areas of improvement in data landscape, and developing and refining data models and architecture frameworks
- Modernize legacy data warehouse to modular, easy to maintain, scalable, cloud-based data process.
- Develop and test ETL components to high standards of data quality and performance as a hands-on development lead.
- Design data architecture patterns and ecosystems including data stores (operational systems, data lakes, data warehouses, data marts), ingress patterns (API, streaming, ETL / ELT), and egress patterns (analytics / decision tools, BI tools). Lead, consult or oversee multiple architectural engagements.
- Oversee and contribute to the creation and maintenance of relevant data artifacts (data lineages, source to target mappings, high level designs, interface agreements, etc.) in compliance with enterprise level architecture standards.
- To lead and deliver data centric projects with concentration on Data Quality and adherence to data standards and best practices.
- Conduct thorough assessments of the existing data & pipeline landscape and create comprehensive migration strategy & roadmap
- Conduct through assessment on the infra requirements of the target platform & guide the team on automating the infra creation aspects in the target system
- Collaborate with Security & Compliance teams to secure migration & address any security related issues
- Collaborate with testing team to design test & data reconciliation plan to ensure data & codes are migrated correctly
- Automate the code conversion mechanism to ensure quicker movement of code from source to target system
Qualifications :
Bachelor’s degree in a technical field (Comp. Science degree preferred not mandatory); Master’s degree is a plus.7+ years of experience in Data Engineering4+ years of experience in Informatica, SQL, PL / SQLExperience in building Data pipelines, Data Lakes and Data Warehouses leveraging Azure Data Factory, Azure SQL Server, ADLSStrong verbal and business communication skills.Technical experience and knowledge in Cloud Data Warehousing, data migration and data transformationMust have hands on experience on migrating on-Prem / legacy data process to cloud environment.AWS cloud experience is a plus, with familiarity of the services : Databricks, Spark SQL and PySparkExperience in Cloud Migration activities like Assessment & Planning, Migration Execution, Validation, Security & Compliance, etcExperience in any automated code conversion tool is a big plusExpertise in Data Engineering, Data Governance, Data Quality, Data Lake, Data Warehousing and Reporting / Analytics conceptsExpertise in data analysis and writing and developing complex queries using SQL.Experience in designing Big Data / Cloud Solution DesignsJob Location : Noida / Bangalore / Gurgaon / Hyderabad / Kochi
Shift Timing : 1 : 00PM IST – 10 : 00 PM IST
Experience : 7-12 years