Databricks SME

Salary - Upto 25 LPA

Key Responsibilities

Design, develop, and maintain data pipelines and ETL processes on Databricks .

Lead the architecture and implementation of scalable, secure, and high-performance data solutions.

Collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.

Optimize data workflows for performance, cost-efficiency, and reliability .

Integrate data from multiple structured and unstructured sources into Delta Lake and cloud data warehouses.

Implement best practices in data governance, security, and quality management.

Mentor junior engineers and provide technical leadership in data engineering best practices.

Troubleshoot and resolve issues in data pipelines, ensuring high data availability and integrity .

Required Skills & Experience

7+ years of experience in Data Engineering .

Proven Subject Matter Expertise in Databricks (including Databricks SQL, Delta Lake, and MLflow).

Strong experience with PySpark , Spark SQL , and Scala .

Hands-on experience with cloud platforms (Azure preferred; AWS / GCP a plus).

Proficiency in data modeling, ETL development , and data warehousing concepts.

Solid understanding of big data ecosystems (Hadoop, Kafka, etc.).

Strong SQL development skills with a focus on performance tuning .

Experience in implementing CI / CD pipelines for data solutions.