DATA ARCHITECT - Databricks

• Role Overview

• Platform/solution architect for enterprise lakehouse programs: governance, security, lineage, cost/performance guardrails and platform standards.

• Core Responsibilities
  • Own target-state lakehouse architecture: metastores, Unity Catalog topology, workspace strategy, governed sharing, lineage.
  • Define platform standards for DLT/Workflows, streaming, CDC/SCDs, data contracts, and schema strategy.
  • Establish cluster policies, pools, autoscaling, Photon adoption, cost guardrails & FinOps dashboards at org level.
  • Lead modernization (legacy DW/ETL Databricks) with migration patterns, parallel run, and cutover.
  • Set DataOps & observability standards: CI/CD, test suites, SLIs/SLOs, reliability scorecards; security/privacy posture.
• Must-have Skills (Screening Focus)
  • Expert Spark/Delta internals; large-scale performance engineering and cost governance.
  • Unity Catalog governance (RBAC/ABAC), row/column security; audit & compliance readiness.
  • Estate-wide DLT/Workflows standards; migration playbooks; ADRs & platform blueprints.
• Nice-to-have (Prefer, not Mandatory)
  • Delta Sharing, Marketplace, Lakehouse Federation; Model Serving/Vector Search awareness.
  • Multi-cloud architecture; FinOps maturity.

Skills Required
Spark, Delta, rbac