Talent.com
Senior Or Lead Databricks Engineer
Senior Or Lead Databricks EngineerNewscape Consulting • Pune, Republic Of India, IN
Senior Or Lead Databricks Engineer

Senior Or Lead Databricks Engineer

Newscape Consulting • Pune, Republic Of India, IN
13 hours ago
Job description

🚀 We’re Hiring : Senior or Lead Databricks Developer

📍 Location : Baner, Pune (Onsite, 5 days a week)

🏢 Work Mode : Full-time | Office-based

⏳ Availability : Immediate Joiners / Notice Period ≤ 30 Days

⚠️ Note : Only local Pune candidates may apply.

  • 🧭 L2 interviews will be conducted in person — apply only if you are open to face-to-face interviews.

🌟 About Newscape Consulting-

Newscape is a fast-growing digital services company specializing in healthcare technology and data engineering. We help healthcare organizations modernize legacy systems, integrate cloud-native platforms, and drive digital transformation through data.

If you’re passionate about solving complex data challenges and contributing to meaningful innovation in healthcare, this is the place for you.

💼 Position Summary -

This role requires a seasoned Databricks professional with strong technical depth and leadership capability. You will design, architect, and optimize data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure, AWS, or GCP). As a Senior or Lead engineer, you will guide best practices, mentor junior engineers, and collaborate closely with data architects, data scientists, and business stakeholders.

Key Responsibilities

  • Architect, design, and maintain large-scale data pipelines using Databricks, PySpark, Spark SQL, and optionally Scala.
  • Lead the adoption and implementation of Databricks components including Workspace, Jobs, DLT (Delta Live Tables), Repos, and Unity Catalog.
  • Build and optimize Delta Lake solutions aligned with Lakehouse and Medallion architecture standards.
  • Partner with architects, engineering teams, and business stakeholders to translate complex business requirements into robust technical solutions.
  • Establish coding standards, best practices, and reusable components for Databricks-based development.
  • Drive CI / CD automation for Databricks deployments using Azure DevOps, GitHub Actions, or equivalent tools.
  • Ensure end-to-end data governance, data quality, metadata management, and lineage tracking leveraging Unity Catalog or Azure Purview.
  • Utilize orchestration tools such as Apache Airflow, Azure Data Factory, or Databricks Workflows for scheduling and monitoring.
  • Conduct performance tuning for Spark jobs, cluster optimization, and cost-efficient compute strategies.
  • Guide data modeling, SQL transformations, and data warehousing implementation using industry-standard best practices.
  • Mentor and provide technical leadership to junior and mid-level data engineers.
  • Participate in architecture reviews, solution design discussions, and roadmap planning.
  • Required Skills and Qualifications

  • Strong hands-on experience in data engineering with specific and extensive expertise in Databricks.
  • Deep proficiency with Databricks Workspace, Jobs, DLT, Repos, Unity Catalog, and Lakehouse architecture.
  • Advanced programming experience with PySpark and Spark SQL;
  • Scala is an added advantage.

  • Strong experience with at least one major cloud platform : Azure, AWS, or GCP.
  • Proven expertise with Delta Lake, Lakehouse, and Medallion architecture patterns.
  • Hands-on experience implementing CI / CD pipelines for Databricks using DevOps tooling.
  • Familiarity with orchestration tools such as Airflow, ADF, or Databricks Workflows.
  • Strong understanding of data governance, security frameworks, and enterprise data management.
  • Experience integrating Databricks datasets with Power BI, including creating optimized data models, implementing Direct Lake or DirectQuery connections, and enabling performant, enterprise-grade reporting solutions.
  • Experience leading technical teams or acting as a senior SME in critical data projects.
  • Excellent analytical abilities, communication skills, and stakeholder management capability.
  • Prior experience in healthcare data environments is a plus (not mandatory).
  • Work Model

  • This is a full-time, onsite role based in Baner, Pune.
  • 5 days onsite per week is mandatory.
  • 📨 Ready to make an impact?

    Apply now or DM us if you’re a Pune-based data enthusiast ready for your next big challenge!

    📩 How to Apply :

    Send your resume to 👉 swapnil.supe@newscapeconsulting.com

    📌 Subject Line : Application – Azure Databricks Developer

    📞 +91 8233829595

    Create a job alert for this search

    Senior Lead Engineer • Pune, Republic Of India, IN

    Related jobs
    Senior Data Engineer

    Senior Data Engineer

    Publicis Production • Republic Of India, IN
    We are seeking a proactive and self-motivated Senior Data Engineer with a proven track record in building scalable cloud-based data solutions across multiple cloud platforms to support our work in ...Show more
    Last updated: 9 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    BlackStone eIT • IN
    Remote
    Quick Apply
    BlackStone eIT is on the lookout for a.In this role, you will be responsible for the design, development, and optimization of our data infrastructure. You'll play a crucial part in supporting data a...Show more
    Last updated: 30+ days ago
    Senior Data Engineer

    Senior Data Engineer

    Persistent Systems • Pune, Republic Of India, IN
    Data Engineer with Pyspark, SQL, ETL Data Engineer.Job Type : Full Time Employment.Candidate should have 5+ years of experience in Data Engineering. Must have experience in Pyspark, SQL, ETL, data pi...Show more
    Last updated: 30+ days ago • Promoted
    Senior / Lead Data Engineer

    Senior / Lead Data Engineer

    Allianz Services • Pune, Republic Of India, IN
    Designation - Senior / Lead Data Engineer.Experience range - 5 to 10 Years.Job Location - Pune or Trivandrum.We are seeking a motivated Data Engineer with experience to design, develop, and maintai...Show more
    Last updated: 5 hours ago • Promoted • New!
    Data Engineer

    Data Engineer

    IntraEdge • India, India
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show more
    Last updated: 30+ days ago • Promoted
    Senior Snowflake Data Engineer

    Senior Snowflake Data Engineer

    Luxoft • India, India
    We are seeking a highly skilled Snowflake Data Engineer with 7 years of IT experience to design, build, and optimize scalable data pipelines and cloud-based solutions across AWS, Azure, and GCP.The...Show more
    Last updated: 10 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Guidanz Inc • Republic Of India, IN
    BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Straive • Republic Of India, IN
    Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced.Analytics datasets based on your stakeholders’ requirements. Design data pipelines to curate sourced da...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Guidanz Inc • India, India
    BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show more
    Last updated: 1 day ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Ironbook AI • India, India
    We are seeking an experienced and driven Lead Data Engineer to spearhead the.AI use cases across the organization.Minimum 7 years of experience in data engineering, with at.Strong hands-on experien...Show more
    Last updated: 12 hours ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    Ironbook AI • India, India
    The ideal candidate will have strong experience with cloud platforms, modern ETL / ELT tools, and deep technical skills in Python, SQL, and distributed data frameworks. Design, develop, and maintain s...Show more
    Last updated: 12 hours ago • Promoted • New!
    Senior Data Solutions Engineer

    Senior Data Solutions Engineer

    Ironbook AI • Republic Of India, IN
    We are seeking an experienced and driven Lead Data Engineer to spearhead the.AI use cases across the organization.Minimum 7 years of experience in data engineering, with at.Strong hands-on experien...Show more
    Last updated: 9 hours ago • Promoted • New!
    Lead Data Engineer

    Lead Data Engineer

    Confidential • India
    Expertise in big data technologies such as Apache Spark and real-time streaming technologies like Apache Kafka.Strong programming skills in Python, Java, C++, SQL etc. Advanced knowledge of a major ...Show more
    Last updated: 12 hours ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    EXL • Pune, Republic Of India, IN
    We are seeking a Senior Data Engineer to build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging.You will...Show more
    Last updated: 30+ days ago • Promoted
    Senior Oracle Data Engineer

    Senior Oracle Data Engineer

    Strattek Consulting • India, India
    This role centers on architecting robust data pipelines across on-prem and cloud Oracle ecosystems, automating ingestion from varied sources including APIs. Exposure to Oracle FDI (Fusion Data Intel...Show more
    Last updated: 15 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Coditas • Pune, Republic Of India, IN
    We are looking for Senior Data Engineers who have the right attitude, aptitude, skills, empathy, compassion, and hunger for learning. Build products in the data analytics space.A passion for shippin...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    eClerx • Pune, Republic Of India, IN
    We are looking for an experienced Senior Data Engineer of hands-on experience in designing, building, and managing complex data architectures and BI solutions. The ideal candidate will have deep exp...Show more
    Last updated: 22 days ago • Promoted
    Data Support Engineer ( Azure, Databricks )

    Data Support Engineer ( Azure, Databricks )

    Kumaran Systems • India, India
    We’re looking for a skilled Data Support Engineer with.The ideal candidate will serve as the escalation point for data ingestion, pipeline, and consumption issues while ensuring high system stabili...Show more
    Last updated: 9 days ago • Promoted