Key Responsibilities Architecture & Design :
- Define best practices for data pipelines, orchestration, and governance using Lakehouse architecture.
- Partner with product, business, and engineering teams to align architecture with business & AI Expertise :
- Hands-on implementation and optimization of Databricks AI tools, including :
- Agent Bricks (AI agent orchestration)
- Lakeflow Designer (data workflow automation)
- MLflow (ML lifecycle management)
- Genie or Mosaic AI (generative AI & enterprise search)
- At least one end-to-end implementation experience of AI / ML workloads using Architect large-scale data platforms leveraging Databricks for advanced analytics, AI / ML, and real-time data processing.
- Stay updated with the latest industry trends in AI, MLOps, LLMOps, and data architectures.
Leadership & Stakeholder Engagement :
Lead technical discussions and workshops with cross-functional stakeholders across global geographies.Provide technical thought leadership and mentor junior engineers / architects.Collaborate with enterprise architects, data scientists, and cloud engineers to ensure architectural consistency.Innovation & Best Practices :
Establish architectural blueprints, standards, and data governance practices.Evaluate new tools, frameworks, and methodologies for adoption in the data ecosystem.Drive performance tuning, cost optimization, and scalability in Databricks and AI solutions.Must-Have Skills :
10+ years in data engineering / architecture, with strong hands-on experience in Databricks.Expertise in Databricks AI suite (Agent Bricks, Lakeflow Designer, MLflow, Genie or Mosaic AI).Proven end-to-end AI / ML implementation experience on Databricks.Strong knowledge of Delta Lake, PySpark, SQL, and Data Lakehouse design patterns.Excellent understanding of MLOps / LLMOps, model deployment, and monitoring.Strong communication and stakeholder management skills with global exposure.Good-to-Have Skills :
Cloud platform experience (Azure, AWS, or GCP) with Databricks.Exposure to GenAI, Vector Databases, RAG frameworks.Familiarity with data security, compliance, and privacy regulations.Industry certifications (e.g., Databricks Certified Data Architect / ML Professional).(ref : hirist.tech)