Senior DataOps Engineer - DevOps Tools

Overview :

curatAId is seeking a Senior DataOps Engineer (Minimum 6+ years of Experience) on behalf of our client, a fast-growing organization focused on data-driven innovation. This role combines data engineering expertise with DevOps practices to support the development and operation of a modern, cloud-based enterprise data platform.

The ideal candidate will be responsible for building and managing data infrastructure, developing scalable data pipelines, implementing data quality and governance frameworks and automating workflows for operational efficiency..

Title : DataOps Specialist.

Level : Consultant / Deputy Manager / Manager / Senior Manager.

Relevant Experience : Minimum of 6+ years of hands-on experience on Data Engineering with DevOps.

Must Have Skill : Data Engineering & DevOps.

Good to have Skill : Snowflake.

Location : Mumbai, Gurgaon, Bengaluru, Chennai, Kolkata, Bhubaneshwar, Coimbatore, Ahmedabad.

Qualifications :

• 6+ years of relevant DevOps / CI-CD pipelines in a data engineering context.
• 4+ years of overall data engineering and data infrastructure experience.
• Strong hands-on experience with data modelling, data warehousing, and building high-volume ETL / ELT pipelines.
• Must have experience with Cloud Data Warehouses like Snowflake, Amazon Redshift, Google Big Query or Azure Synapse. Experience with version control systems (GitHub, BitBucket, GitLab).
• Strong SQL expertise : Implement best practices for data storage management, security, and retrieval efficiency.
• Experience with pipeline orchestration tools (Fivetran, Stitch, Airflow, etc.).
• Coding proficiency in at least one modern programming language (Python, Java, Scala, etc.).
• Experience with Infrastructure as Code (IaC) tools like Terraform, Ansible, Chef, Puppet or AWS Cloud Formation.
• Experience with containerization and container orchestration technologies, like Docker or Kubernetes..
• Experience with cloud deployment automation and configuration management.
• Experience with networking, monitoring, and logging for cloud :
• Design, build, and manage robust and scalable data infrastructure on cloud platforms.
• Architect, develop, and maintain high-volume ETL / ELT pipelines to ingest, transform, and load data from various sources.
• Implement and enforce data quality frameworks, monitoring systems, and validation processes to ensure data accuracy and reliability.
• Establish and maintain data governance policies and procedures in collaboration with relevant stakeholders.
• Drive the adoption of DevOps principles and practices within the data engineering team, including CI / CD for data pipelines.
• Implement and manage CI / CD pipelines for data engineering workflows using tools like Jenkins, GitLab CI, or GitHub Actions.
• Utilize Infrastructure as Code (IaC) tools such as Terraform, Ansible, Chef, Puppet, or AWS CloudFormation to automate infrastructure provisioning and management.
• Implement best practices for data storage management, security, and efficient data retrieval.
• Work with pipeline orchestration tools (e.g., Fivetran, Stitch, Airflow) to automate and monitor data workflows.
• Apply strong SQL expertise for data querying, manipulation, and validation.
• Develop and maintain data models that support efficient data warehousing and analytics.
• Implement containerization and container orchestration technologies like Docker and Kubernetes for data services.
• Automate cloud deployment and configuration management for data-related services.
• Implement and manage networking, monitoring, and logging solutions for cloud-based data services.
• Collaborate closely with data scientists, analysts, and other engineering teams to understand their data needs and provide robust data solutions.
• Proactively identify and resolve performance bottlenecks and operational issues within the data platform.
• Stay up-to-date with the latest trends and technologies in Data Engineering, DevOps, and cloud computing.
• Contribute to the development of best practices and standards for data operations within the organization.

ref : hirist.tech)