Talent.com
DataOps Engineer - CI / CD

DataOps Engineer - CI / CD

NeemtreeBangalore
30+ days ago
Job description

Responsibilities :

  • Promote a DataOps approach to Data science, engineering, and analytics delivery processes to automate the provision of data, testing, and monitoring, and shorten CI / CD.
  • Collaborate with data & ML leads and create and build an optimal data pipeline architecture for the data solutions, including data science products.
  • Ensure the data pipelines are scalable and performant, as well as create and maintain a service to connect data products.
  • Create dashboards and other tools required to efficiently monitor our data and ML infrastructure, pipelines, ETL, and analytics delivery processes.
  • Building an end-to-end event instrumentation and alerting system to detect and alert to any anomaly in the system or in the data.
  • Assist in managing our data and ML infrastructure (upgrading, monitoring, optimising).
  • Collaborate with IT DevOps engineers and participate in enterprise DevOps activities.
  • Exchange your knowledge on infra and data standards with other developers and be part of our tech community. Promote the use of engineering best practices.
  • Contribute to innovative POCs with our data & engineering teams.
  • Remain flexible towards technology approaches to ensure that the best advantage is being taken by new technologies.

Requirements :

  • Strong drive to solve problems, communicate clearly, and contribute positively to a DevOps / DataOps culture.
  • Knowledge of the latest DevOps tools and practices.
  • Experience with data pipelines within AWS (Glue, DataPipeline, Athena, EMR, DMS, Spark).
  • Experience with Database Replication and databases like Aurora, MySQL, MariaDB, etc.
  • Efficient in building CI / CD pipelines for containerized Java / Python codestack.
  • Comfortable with Git workflow.
  • Experience with applications deployed in AWS.
  • Experience with configuration management and provisioning tools (e. g., Ansible, CloudFormation, Terraform).
  • Knowledge of one or more scripting languages - Orchestration / containerisation using Docker and Kubernetes.
  • Basic knowledge of data science & ML engineering.
  • Bachelor's Degree in computer science or a similar degree, or Big Data Background from top-tier universities.
  • Experience : 8+ years of experience.
  • (ref : hirist.tech)

    Create a job alert for this search

    Engineer • Bangalore