Job Description : Accountabilities :
- Understands the business vision from the data solutions and works with all project stakeholders to enable that.
- Works alongside multiple other project team members to drive towards the common goal.
- Helps the business analysts to create data mapping documents that define the end to end pipeline.
- Builds and delivers data pipelines that ingests data (structured and unstructured) from multiple sources into the target data lake.
- Builds the data management layer in line with the business needs and standards defines.
- Triages problems in the pipelines or data layers and effectively fixes them based on the priority.
- Automates data loading during initial or incremental loads.
- Supports the run and maintain team during critical issues and helps them in faster resolution.
Dimensions :
A key element of this role is to work along with Trading and Supply teams in delivering an enterprise solution in data engineering area. This will mostly include green field implementation of data analytics solutions delivering high value to customer.
Special Challenges :
As a data engineer responsible for effectively handling data and deriving value out of it.Responsible for providing innovative solutions that are best in class but also quick to market.No direct reports. But is self-driven and can independently handle the work assigned.Drive effective best practices across multiple initiatives and teams and be innovative and influential in managing constraints.Requirements :
4 + years of IT industry experience, with at least 2 years on Big data technologies.Interest and passion in Big Data technologies and appreciates the value that can be brought in with an effective data management solution.Has worked on real data challenges and handled huge volume, velocity and variety of data.Understands Infrastructure concepts and has worked with multiple cloud tenants.Proficient with data warehousing and traditional ETL technologies.Strong experience in writing stored procedure, functions and triggers in SQL.Strong experience on Azure ecosystem and has worked with data management on Databricks, ADF, Blob Store, ADLS etc.Implement complex data processing algorithms in real time in and efficient manner using Spark and other technologies.Knowledge on data analytics area using Visualization and data modelling tools.Excellent analytical & problem-solving skills, willingness to take ownership and resolve technical challenges.Excellent communication and stakeholder management skillsIndustry certification in Data Engineering is an added advantage. technical skills like Pyspark , Azure etcMandatory skills :
AWS / Azure / SAP - MasterELT - MasterData Modeling - MasterData Integration & Ingestion - SkillData Manipulation and Processing - SkillGITHUB, Action, Azure DevOps - SkillData factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest - SkillRoles and Responsibilities :
The Senior Data Engineer is the one who designs and builds data foundations and end to end solutions for the Shell Business to maximize value from data. The role helps create a data-driven thinking within the organization, not just within IT teams, but also in the wider Business stakeholder community. A Senior Data engineer is expected to be a subject matter expert, who design & build data solutions and mentor junior engineers. They are also the key drivers to convert Vison and Data Strategy for IT solutions and deliver.
Key Characteristics :
Technology expert who constantly pursues knowledge enhancement and has inherent curiosity to understand work from multiple dimensions.Deep data focus with expertise in technology domainA skilled communicator capable of speaking to both technical developers and business managers. Respected and trusted by leaders and staff.Actively delivers the roll-out and embedding of Data Foundation initiatives in support of(ref : hirist.tech)
%20Pvt%20Ltd){kind=logo}