For an international project in Chennai, we are urgently looking for a Full Remote SeniorAzure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databricks..
We are looking for a motivated contractor. Candidates need tbe fluent in English.
Tasks and responsibilities :
- Design and Develop data pipelines for enterprise search solutions using ADF and Databricks;
- Document Low Level Design, Unit Testing Plan, and Unit Test case documentation;
- Design and develop ETL / ELT pipelines in Azure Data Factory to extract data from on-premises, cloud, and third-party sources;
- Implement data cleansing, enrichment, and aggregation logic;
- Provide mentorship to team members, ensuring best practices in cloud architecture;
- Ensure compliance with security standards and best practices in cloud architecture and data handling, particularly when dealing with sensitive information;
- Assist in Integration Testing;
- Perform Code optimization and strictly adhere to bank coding standards;
- Assist in any documentation or any artifacts needed for OIS clearance;
- Participate in SCRUM meeting and coordination with onsite and offshore team;
- Perform timely resolution of issues and root cause analysis of each issue;
- Applied Security Compliance, which includes Application and data security assessment and remediation to attain compliance to WB Office Information Security Policies and Standards;
- Proactively manage your own work queue, time, and expectations, meet deadlines, and communicate needs and tradeoffs if your schedule is in jeopardy;
Profile :
Bachelor or Master degree;8 years of IT experience ;5 years in Data engineer related experience preferred;Knowledge and experience of executing data pipelines using ADF, Python and Databricks;Knowledge of programming languages like Python;Cloud native and hybrid web development projects with demonstrated experience in Azure technologies, ASE, Azure Data Factory, Databricks , Azure Data Lake and Caching technologies;Addressing performance and production issues, with extensive demonstrated knowledge in logging and monitoring;Sound technical knowledge and understanding of infrastructure design including private and public cloud;Establishing systems to supervise the operating efficiency of existing application systems and providing (or arranging for) proactive maintenance;Participating in systems design, working within an established framework;Ability to mentor and guide technical teams;Fluent in English;