Senior Data Engineer - AWS Glue - Full Remote - Contractor in USD

All European Careersguwahati, assam, in

1 day ago

Job type

Remote

Job description

For an international organization in Geneva, Switzerland, we are urgently looking for a Full Remote Senior Data Engineer. The Senior Data Engineer is responsible for the implementation of the set of automated ETL pipelines, implementation of the analytics of the platform operations and importing new data sources and its maintenance. The Analytics Data Platform is a cloud-native application deployed on AWS through CloudFormation templates

using the following resources : Glue, S3, Lambda, Step Function, Airflow, Athena, RDS, Redshift, and Lake Formation.

Candidates need to be willing to work normal office hours in Switzerland. This positions is long-term.

Tasks and Responsibilities :

Refactor, develop, maintain the web analytics ETL pipeline

Use data pipeline jobs to extract data from various sources including on premise relational databases, APIs,

Use data pipeline jobs to curate extracted raw data

Use data pipeline jobs to apply transformation based on reporting needs

Data source connectors to ingest the data from various data sources

Write the spark SQLs to produce the datasets

Monitor, troubleshoot and remediate the above jobs

Data Modelling

Maintenance of the existing data models in the data lake and data warehouse

Design & Implement data models for new data sources as required

Refactor, maintain the CloudFormation templates and CDK as per the CICD guidelines / requirements

Other ad hoc deliverables required by the project

Profile :

University degree in computer science, engineering or other related discipline;

5 years of experience as Data Engineer;

Hands-on experience writing code for AWS Glue in python with PySpark and Spark SQL;

Relational databases such as PostgreSQL (preferably AWS Redshift) with expertise in SQL, query optimization;

Serverless Query Service, preferably AWS Athena;

Experience in implementing an end to end cloud native Data warehouse platform - data modelling;

ETL, data integration and data migration, preferably related to AWS Redshift;

Expertise in SQL, query optimization;

Experience in implementing data integration from different sources;

Experience with AWS IaC (CloudFormation & CDK) and CICD pipelines;

Experience in Python programming and using PySpark for distributed computing in AWS Cloud;

Experience in working with ERP applications such as PeopleSoft is an added advantage;

Development of Reports / Dashboards using Tableau is an added advantage;

Fluent in English;

Create a job alert for this search

Senior Data Engineer • guwahati, assam, in