Responsibilities:
- Lead, hire and mentor a high-performing team; set priorities, delivery cadence and engineering standards to scale the capability.
- Own the roadmap and delivery of enterprise data platforms and analytics products, including data lakes/warehouses, ETL/ELT, streaming and data APIs.
- Provide hands-on technical leadership: design and review architecture, implement or optimize key components, and resolve production incidents as required.
- Drive adoption of Databricks best practice (Spark optimization, Delta Lake, Unity Catalog) and core cloud platform patterns.
- Productionise AI/ML: guide feature engineering, model deployment, monitoring, versioning, explainability and MLOps workflows.
- Establish and enforce CI/CD, automated testing, observability and incident response for data pipelines, models and analytics services.
- Define and enforce data governance, security, privacy and compliance standards in partnership with legal and security teams.
- Partner with business leaders, product managers and consulting teams to translate business problems into scalable data and AI solutions; manage stakeholder expectations and prioritisation.
- Manage vendor relationships and platform budgets; evaluate and procure third-party tools where appropriate.
- Foster a culture of data literacy, experimentation, inclusivity and continuous improvement.
Must have skills and qualifications:
- Degree in Computer Science, Engineering, Data Science, Statistics or equivalent practical experience.
- 15+ years designing and delivering data platforms or large-scale data systems; 5+ years managing engineering teams and senior technical leaders.
- Proven experience delivering end-to-end cloud data platform transformations at enterprise scale.
- Hands-on Databricks experience (Spark optimisation, Delta Lake, workspace/job orchestration, Unity Catalog) at scale.
- Strong practical experience building and operating data solutions on AWS (e.g., S3, Glue, Redshift/Athena, Lambda, EKS/ECS; infrastructure as code such as CloudFormation or Terraform).
- Solid understanding of AI/ML production patterns, including model development, deployment, monitoring, drift detection and MLOps.
- Strong software and data engineering skills: Python and SQL required; Scala/Java advantageous. Experience with Spark, data modelling, ETL/ELT and streaming fundamentals.
- Experience implementing CI/CD, container orchestration and observability for data systems.
- Knowledge of data governance, metadata/catalogue tools, lineage and data quality frameworks (i.e. Great Expectations or equivalent).
- Strong grasp of security, data privacy and regulatory requirements (e.g., GDPR, data residency).
- Professional certification such as Databricks or AWS (Solutions Architect / Specialty).
What makes you stand out:
- Consulting or client-facing delivery experience.
- Experience with streaming platforms (Kafka, Pub/Sub, Kinesis) and real-time architectures.
- Familiarity with generative AI, LLMs and conversational AI production patterns.
- Exposure to other cloud providers (Azure/GCP) or hybrid cloud architectures.
(ref:hirist.tech)