Cloud Architect

"KPI Partners is looking for hands-on Azure Data Platform Architect with strong expertise in Databricks, DevOps (CI / CD), and IaC (Terraform / Bicep) .

This is a platform-focused architect role , not a data engineering or infrastructure admin role.

The ideal candidate must have strong hands-on expertise across Azure, Databricks, IaC, and DevOps , with the ability to design, automate, and optimize the data platform end-to-end. This is a platform-focused architectural role where the candidate acts as a Databricks Platform Owner , guiding teams, enforcing best practices, and driving automation, governance, and scalability across the environment.

The architect will own the Databricks platform setup, governance, automation, cost optimization, and best practices , while also enabling CI / CD and ADF monitoring.

Key Responsibilities

1. Databricks Platform Architecture

• Set up and configure Databricks workspaces, clusters, schemas, RBAC, and security.
• Define and enforce best practices :
• Cluster policies
• Workspace standards
• Cost control
• Job orchestration rules
• Automate Databricks (clusters, jobs, configurations) using Terraform / Bicep .
• Implement observability :
• Failed job alerts
• Cost and usage insights
• Performance reporting

2. ADF onitoring & Automation

3. DevOps (CI / CD) Enablement

4. Infrastructure-as-Code (IaC)

5. Data Platform Governance (Long Term)

6. Cost Optimization

Required Skills

Nice-to-Have