Promote a DataOps approach to Data science, engineering, and analytics delivery processes to automate the provision of data, testing, and monitoring, and shorten CI / CD.
Collaborate with data & ML leads and create and build an optimal data pipeline architecture for the data solutions, including data science products.
Ensure the data pipelines are scalable and performant, as well as create and maintain a service to connect data products.
Create dashboards and other tools required to efficiently monitor our data and ML infrastructure, pipelines, ETL, and analytics delivery processes.
Building an end-to-end event instrumentation and alerting system to detect and alert to any anomaly in the system or in the data.
Assist in managing our data and ML infrastructure (upgrading, monitoring, optimising).
Collaborate with IT DevOps engineers and participate in enterprise DevOps activities.
Exchange your knowledge on infra and data standards with other developers and be part of our tech community. Promote the use of engineering best practices.
Contribute to innovative POCs with our data & engineering teams.
Remain flexible towards technology approaches to ensure that the best advantage is being taken by new technologies.
Requirements :
Strong drive to solve problems, communicate clearly, and contribute positively to a DevOps / DataOps culture.
Knowledge of the latest DevOps tools and practices.
Experience with data pipelines within AWS (Glue, DataPipeline, Athena, EMR, DMS, Spark).
Experience with Database Replication and databases like Aurora, MySQL, MariaDB, etc.
Efficient in building CI / CD pipelines for containerized Java / Python codestack.
Comfortable with Git workflow.
Experience with applications deployed in AWS.
Experience with configuration management and provisioning tools (e. g., Ansible, CloudFormation, Terraform).
Knowledge of one or more scripting languages - Orchestration / containerisation using Docker and Kubernetes.
Basic knowledge of data science & ML engineering.
Bachelor's Degree in computer science or a similar degree, or Big Data Background from top-tier universities.