Senior DevOps Engineer

Job Description :

Senior DevOps Engineer Summary The Senior DevOps Engineer will design and maintain automation frameworks that underpin Databricks-based data warehouse environments and associated Azure services. This role focuses on establishing robust CI / CD pipelines, automating infrastructure provisioning, and ensuring seamless deployment of code and configurations across development, test, and production environments. Emphasis is placed on reliability, repeatability, and security in deployments, enabling rapid delivery of updates while maintaining compliance and high quality.

Responsibilities

• Develop and manage CI / CD pipelines for Databricks notebooks, libraries, and Azure Data Factory (ADF) workflows.
• Implement Infrastructure-as-Code (IaC) using Terraform or Bicep to provision and manage Azure resources.
• Automate Databricks job and notebook deployments using CLI, REST APIs, and integration with Git-based workflows.
• Establish branching and merging strategies (e.g., GitFlow, trunk-based development) and enforce code review processes.
• Integrate automated testing, data validation, and quality gates into CI / CD pipelines.
• Manage release processes for Databricks jobs, ADF pipelines, and tabular models in Azure Analysis Services.
• Monitor pipeline runs, investigate failures, and resolve issues in collaboration with engineering teams.
• Secure CI / CD processes by leveraging Key Vault for secrets management, service principals, and role-based access controls.
• Document pipeline processes and provide training to development and analytics teams.

Required Qualifications

Preferred Qualifications

Certifications