For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databricks.
We are looking for a motivated contractor. Candidates need tbe fluent in English.
Tasks and responsibilities :
- Build data integration feeds according to the requirement;
- Review the business requirements and design the Pipelines that loads data into Azure Data Lakes or Azure Data warehouses, Azure Synapse, Azure Databricks or Cloud services. etc;
- Developing pipelines between different data sources including XML, flat files, SAP and databases.
- Strong Experience in Data Modeling, Data / Information Architecture, Metadata Management, Master Data management, Data Governance & Data Quality;
- Strong analytical abilities required to evaluate work and provide technical assistance;
- Perform Code optimization and strictly adhere IFC standards;
- Expertise in monitoring and improving performance of the Azure data feeds;
- Create Design, Functional / Technical specification documentations or any artifacts needed;
Profile :
Bachelor or Master degree;5 years of experience in Data Modeling, Data / Information Architecture, Metadata Management, Master Data management, Data Governance & Data Quality.Expertise on Azure platforms including experience in Azure Databricks, ADLS, Synapse and its various components is mandatory;In-dept knowledge and hands-on experience in Microsoft Azure data stack i.e., Azure Data Factory, Azure Synapse, Azure SQL Database, Azure Functions, Logic apps, Spark and Python / PySpark programming etc;Strong experience with designing ETL / ELT Pipelines / Data flows using Azure Data Factory as Primary skill and SSIS / Informatica as additional skills;Must have strong development & design skills in leading RDBMS environments e.g. MS SQL Server and Oracle;Expertise experience in complex SQL queries is required;Experience of Big data approach, architectural concepts, data sourcing / ingestion / curation and storage mechanisms;Experience with real-time / near-real-time Data warehousing concepts and Design approaches;Should have executed end-to-end projects focusing on analytics, data warehousing, enterprise reporting;Leveraging components like Azure Automation, Azure Power Shell Scripting, Az Copy, Devops / ARM Templates;Should have thorough understanding of Data-lakes, raw / enriched / curated layer concepts and ETL within Azure framework;Highly skilled in end-to-end Data warehousing and BI (analytics) design / development;Create Design, Functional / Technical specification documentations or any artifacts needed;Fluent in English;