Talent.com
This job offer is not available in your country.
Databricks lead

Databricks lead

ConfidentialBengaluru / Bangalore, India
9 days ago
Job description

Designation - Databricks lead

Experience - 8+ Years

Key Responsibilities

  • Lead the design and implementation ofscalable data pipelines and ETL processes on Databricks.
  • Architect and optimize big data and analyticssolutions using Spark, Delta Lake, and MLflow.
  • Collaborate with stakeholders to translate businessrequirementsinto technical solutions.
  • Oversee the migration of on-premises data systemsto Databricks cloud environments.
  • Drive best practicesin data governance, security, and performance optimization.
  • Provide technical leadership, mentorship, and support to data engineering teams.
  • Work closely with data scientiststo productionize machine learning models.
  • Establish monitoring, logging, and automated testing for data pipelines.
  • Ensure compliance with industry standards,security protocols, and regulatory requirements.
  • Evaluate new featuresin Databricks and recommend adoption strategies.

Required Skills & Qualifications

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • 8+ years of experience in data engineering, with 3+ years in Databricks.
  • Strong expertise in Apache Spark, PySpark, SQL, and Delta Lake.
  • Proven experience with Databricks on Azure, AWS, or GCP.
  • Hands-on experience in designing large-scale ETL / ELT workflows.
  • Proficiency in cloud storage (S3, ADLS, GCS) and data warehousing solutions (Snowflake, Redshift,

    • Synapse, BigQuery).

  • Strong knowledge of DevOps, CI / CD, and Infrastructure as Code (Terraform / CloudFormation).
  • Experience in MLflow, feature engineering, and integrating machine learning pipelines.
  • Excellent communication, leadership, and stakeholder managementskills.
  • Databricks certification(s) such as Databricks Certified Data Engineer Professional or Databricks

    • Certified Machine Learning Professional preferred.

    Preferred Skills

  • Experience with streaming technologies(Kafka, Event Hubs, Kinesis).
  • Knowledge of data governance frameworks(Collibra, Purview).
  • Familiarity with BI / visualization tools(Power BI, Tableau, Looker).

    • Show more

    Show less

    Skills Required

    Devops, Pyspark, Apache Spark, Databricks, Sql, Etl, ELT

    Create a job alert for this search

    Lead • Bengaluru / Bangalore, India