Technical Architect-Databricks
ConfidentialPune, India4 days ago
Job description12+ years of experience in data engineering using Databricks or Apache Spark-based platforms. Proven track record of building and optimizing ETL / ELT pipelines for batch and streaming data ingestion. Hands-on experience with Azure services such as Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, or Azure SQL Data Warehouse. Proficiency in programming languages such as Python, Scala, or SQL for data processing and transformation. Expertise in Spark (PySpark, Spark SQL, or Scala) and Databricks notebooks for large-scale data processing. Familiarity with Delta Lake, Delta Live Tables, and medallion architecture for data lakehouse implementations. Build and query deltalake storage solutions Process streaming data with Azure Databricks structured streaming Design Azure Databricks security and data protection solutions Flatten nested structures and explode arrays with spark Transfer data outside using sparkpools using pyspark connector Optimizing spark jobs Implementing best practices in spark / databricks Experience with orchestration tools like Azure Data Factory or Databricks Jobs for scheduling and automation. Knowledge of Git for source control and CI / CD integration for Databricks workflows, cost optimization, performance tuning. Familiarity with Unity Catalog, RBAC, or enterprise-level Databricks setups. Ability to create reusable components, templates, and documentation to standardize data engineering workflows. Solutioning and presales - Architecting frameworks, defining roadmaps, and engaging with stakeholders. Experience in defining data strategy, evaluating new tools / technologies, and driving adoption across the organization. Must have experience of working with streaming data sources and Kafka (preferred). Bachelor's degree in computer science, Information Technology, or related field Proven experience as a Databricks Architect or similar role Complete knowledge in Azure Databricks platform architecture Databricks certification (e.g., Certified Data Engineer, Associate Developer) Expertise in Python / Scala / SQL / R Experience with cloud platforms like AWS, Azure, or GCP Strong understanding of data modeling and cloud integration Experience with cluster sizing and security implementation Excellent problem-solving and communication skills
Role Overview
The Technical Architect - Databricks designs and implements scalable data architectures and solutions. The jobholder has expertise in Databricks Lakehouse, data modeling, and cloud integration, ensuring high performance, security, and reliability.
Responsibilities
- Design and implement Databricks-based data architectures to meet business requirements.
- Develop and optimize data pipelines using PySpark, Scala, or SQL.
- Establish the Databricks Lakehouse architecture for batch and streaming data.
- Collaborate with cross-functional teams to integrate Databricks with cloud platforms (e.g., AWS, Azure, GCP).
- Ensure data security and compliance with best practices.
- Monitor and troubleshoot Databricks environments for performance and reliability.
- Stay updated on Databricks advancements and industry trends.
Key Technical Skills & Responsibilities
Eligibility Criteria
Skills Required
Spark SQL, Scala, Pyspark, Apache Spark, Kafka, Sql, Git, Azure Data Factory, Azure Synapse Analytics, Databricks
Create a job alert for this search
Technical • Pune, India
Related jobs
- Promoted
Data Architect
RecroPune, IN Central Data Office – Data Architect.SMEs, engineers, and developers to design and govern enterprise-level data architectures, ensuring alignment with corporate data principles and standards.This p...Show moreLast updated: 1 day ago
- Promoted
Databricks Solution Architect
InspireXTPune Description : About InspireXT : InspireXT is on a mission to build the most trusted supply chains in the world.Our Connected I...Show moreLast updated: 30+ days ago
- Promoted
Data Architect
NR ConsultingPune, IN We are seeking a highly experienced Data Architect with over 10 years in IT and at least 4+ years in data architecture or principal data engineering roles.
The ideal candidate will design, implement...Show moreLast updated: 1 day ago
- Promoted
Principal / Senior Data Architect
Aayspune, maharashtra, in Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 30+ days ago
- Promoted
Technical Architect
ConfidentialPune The Senior Tech Lead - Databricks leads the design, development, and implementation of advanced data solutions.Has To have extensive experience in Databricks, cloud platforms, and data engineering,...Show moreLast updated: 30+ days ago
- Promoted
Data Architect
Tech MahindraPune, Maharashtra, India Role : Azure Data Architect Exp : 10+yrs Location : Pan India Strong experience architecting and delivering data platforms on Azure Proficiency in Azure services including Azure Synapse Analytic...Show moreLast updated: 1 day ago
- Promoted
Enterprise Data Architect
DBiz.aiPune, IN You will ensure that data is modeled, structured and integrated across the organization to support analytics and AI / ML initiatives at scale.
You will work closely with business stakeholders and tech...Show moreLast updated: 12 days ago
- Promoted
Technology Architect
Mount Talent Consulting Pvt Ltd.pune, maharashtra, in Technical Architect(Performance Testing).Location- Pune,Hyderabad, Bangalore,Chennai.Skills- Performance,Java, Jmeter, Loadrunner, Dynatrace.Show moreLast updated: 18 days ago
- Promoted
Data Solution Architect
Persistent SystemsPune, Maharashtra, India The Architect will lead the design and implementation of scalable data solutions.This includes architecting cloud-native data pipelines, integrating tagging logic, and ensuring high-performance dat...Show moreLast updated: 18 days ago
- Promoted
Technical Architect Oracle Cloud
Proso.aiPune, Maharashtra, India Job Title : Technical Architect – Oracle Fusion Cloud (OIC, VBCS, OCI, ERP, HCM) Location : Work from Home Experience : 10+ Years Shift : US Rotational Shift Employment Type : Full-Time About the ...Show moreLast updated: 1 day ago
- Promoted
Azure Data Architect
Programmers.ioPune, Maharashtra, India Job Title : Azure Data Architect.Location : Hyderabad, pune (hybrid).Experience required : 12+ years.We are seeking a highly experienced S.
The ideal candidate should bring strong expertise in SQL, ETL...Show moreLast updated: 30+ days ago
- Promoted
Technical Architect Team Lead
QNOPY Inc.Pune, Maharashtra, India About the Role We are seeking an experienced Technical Team Lead with a strong background in delivering SaaS products to lead our engineering team.
The ideal candidate has 7+ years of hands-on exp...Show moreLast updated: 9 days ago
- Promoted
Technical Architect -Google Cloud Platform (GCP)-Data Engineering
ConfidentialPune, India Atos is a global leader in digital transformation,European number one in Cloud, Cybersecurity and High-Performance Computing, the Group provides end-to-end Orchestrated Hybrid Cloud, Big Data, Busi...Show moreLast updated: 4 days ago
- Promoted
Data Architect
Persistent Systemspune, maharashtra, in We are seeking a highly skilled and experienced Azure Databricks Architect to lead the design, development, and implementation of scalable data solutions using Azure Databricks.This role involves w...Show moreLast updated: 19 days ago
- Promoted
Databricks Data Architect
Tata Consultancy Servicespune, maharashtra, in Experience with machine learning frameworks and pipelines, particularly in Databricks.Familiarity with containerized environments (e.
Docker, Kubernetes) for deploying data workflows.Should be able ...Show moreLast updated: 1 day ago
- Promoted
Technical Architect
Antal Internationalpune, maharashtra, in My Client is a global leader in gaming with teams across the world creating original and memorable gaming experiences.Role : Technical Architect(C++).
Were hiring an Associate Technical Architect.Yo...Show moreLast updated: 22 days ago
- Promoted
YASH Technologies - Technical Lead - Azure Databricks
YASH TechnologiesPune Description : We are looking forward to hire Azure Databricks Professionals in the following areas : Job Description : <...Show moreLast updated: 28 days ago
- Promoted
Data Architect- Databricks
ConfidentialPune, India Data Architect (Databricks, Snowflake).Nagpur, Pune, Chennai, Bangalore.Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engine...Show moreLast updated: 4 days ago