Talent.com
Senior OR lead Databricks Engineer
Senior OR lead Databricks EngineerNewscape Consulting • pune, maharashtra, in
Senior OR lead Databricks Engineer

Senior OR lead Databricks Engineer

Newscape Consulting • pune, maharashtra, in
14 hours ago
Job description

🚀 We’re Hiring : Senior or Lead Databricks Developer

📍 Location : Baner, Pune (Onsite, 5 days a week)

🏢 Work Mode : Full-time | Office-based

⏳ Availability : Immediate Joiners / Notice Period ≤ 30 Days

⚠️ Note : Only local Pune candidates may apply.

  • 🧭 L2 interviews will be conducted in person — apply only if you are open to face-to-face interviews.

🌟 About Newscape Consulting-

Newscape is a fast-growing digital services company specializing in healthcare technology and data engineering. We help healthcare organizations modernize legacy systems, integrate cloud-native platforms, and drive digital transformation through data.

If you’re passionate about solving complex data challenges and contributing to meaningful innovation in healthcare, this is the place for you.

💼 Position Summary -

This role requires a seasoned Databricks professional with strong technical depth and leadership capability. You will design, architect, and optimize data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure, AWS, or GCP). As a Senior or Lead engineer, you will guide best practices, mentor junior engineers, and collaborate closely with data architects, data scientists, and business stakeholders.

Key Responsibilities

  • Architect, design, and maintain large-scale data pipelines using Databricks, PySpark, Spark SQL, and optionally Scala.
  • Lead the adoption and implementation of Databricks components including Workspace, Jobs, DLT (Delta Live Tables), Repos, and Unity Catalog.
  • Build and optimize Delta Lake solutions aligned with Lakehouse and Medallion architecture standards.
  • Partner with architects, engineering teams, and business stakeholders to translate complex business requirements into robust technical solutions.
  • Establish coding standards, best practices, and reusable components for Databricks-based development.
  • Drive CI / CD automation for Databricks deployments using Azure DevOps, GitHub Actions, or equivalent tools.
  • Ensure end-to-end data governance, data quality, metadata management, and lineage tracking leveraging Unity Catalog or Azure Purview.
  • Utilize orchestration tools such as Apache Airflow, Azure Data Factory, or Databricks Workflows for scheduling and monitoring.
  • Conduct performance tuning for Spark jobs, cluster optimization, and cost-efficient compute strategies.
  • Guide data modeling, SQL transformations, and data warehousing implementation using industry-standard best practices.
  • Mentor and provide technical leadership to junior and mid-level data engineers.
  • Participate in architecture reviews, solution design discussions, and roadmap planning.
  • Required Skills and Qualifications

  • Strong hands-on experience in data engineering with specific and extensive expertise in Databricks.
  • Deep proficiency with Databricks Workspace, Jobs, DLT, Repos, Unity Catalog, and Lakehouse architecture.
  • Advanced programming experience with PySpark and Spark SQL; Scala is an added advantage.
  • Strong experience with at least one major cloud platform : Azure, AWS, or GCP.
  • Proven expertise with Delta Lake, Lakehouse, and Medallion architecture patterns.
  • Hands-on experience implementing CI / CD pipelines for Databricks using DevOps tooling.
  • Familiarity with orchestration tools such as Airflow, ADF, or Databricks Workflows.
  • Strong understanding of data governance, security frameworks, and enterprise data management.
  • Experience integrating Databricks datasets with Power BI, including creating optimized data models, implementing Direct Lake or DirectQuery connections, and enabling performant, enterprise-grade reporting solutions.
  • Experience leading technical teams or acting as a senior SME in critical data projects.
  • Excellent analytical abilities, communication skills, and stakeholder management capability.
  • Prior experience in healthcare data environments is a plus (not mandatory).
  • Work Model

  • This is a full-time, onsite role based in Baner, Pune.
  • 5 days onsite per week is mandatory.
  • 📨 Ready to make an impact?

    Apply now or DM us if you’re a Pune-based data enthusiast ready for your next big challenge!

    📩 How to Apply :

    Send your resume to 👉 swapnil.supe@newscapeconsulting.com

    📌 Subject Line : Application – Azure Databricks Developer

    📞 +91 8233829595

    Create a job alert for this search

    Senior Lead Engineer • pune, maharashtra, in

    Related jobs
    Data Engineer

    Data Engineer

    IntraEdge • Pune, IN
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show more
    Last updated: 30+ days ago • Promoted
    Lead Engineer

    Lead Engineer

    Hyqoo • Pune, IN
    Design, deploy, and manage AWS cloud infrastructure, including EC2 instances, S3 buckets, VPCs, RDS databases, and Lambda functions. Assist in the design, implementation, and maintenance of backup, ...Show more
    Last updated: 25 days ago • Promoted
    PySpark Data Engineer

    PySpark Data Engineer

    EXTRAGIG • Pune, IN
    Contract Assistant – Data Engineer Support (Remote, EST Hours).PySpark Data Engineer with daily activities.This is a remote contract role. Execute creative software and data solutions, including des...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Ironbook AI • Pune, IN
    We are seeking an experienced and driven Lead Data Engineer to spearhead the.AI use cases across the organization.Minimum 7 years of experience in data engineering, with at.Strong hands-on experien...Show more
    Last updated: 13 hours ago • Promoted • New!
    Data Engineer

    Data Engineer

    TerraGiG • Pune, IN
    Lead the design, development, and implementation of data solutions using AWS and Snowflake.Collaborate with cross-functional teams to understand business requirements and translate them into techni...Show more
    Last updated: 30+ days ago • Promoted
    GCP Data Engineer

    GCP Data Engineer

    Adastra • Pune, IN
    We are looking for a proactive and solution-oriented GCP Data Engineer to join our team.This role requires hands-on experience in Google Cloud Platform (GCP), especially with BigQuery and Airflow, ...Show more
    Last updated: 17 days ago • Promoted
    AWS Data Architect

    AWS Data Architect

    ACL Digital • Pune, IN
    AWS (S3, Redshift, Glue, Lake Formation, IAM).Proficient in data modeling, performance tuning, and security best practices. .AWS Certified Solutions Architect preferred.Show more
    Last updated: 21 days ago • Promoted
    Senior / Lead Data Engineer

    Senior / Lead Data Engineer

    Allianz Services • Pune, Maharashtra, India
    Designation - Senior / Lead Data Engineer.Experience range - 5 to 10 Years.Job Location - Pune or Trivandrum.We are seeking a motivated Data Engineer with experience to design, develop, and maintai...Show more
    Last updated: 3 hours ago • Promoted • New!
    Data Engineer

    Data Engineer

    MyRemoteTeam Inc • Pune, IN
    MyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent.We empower businesses by providing world-class software engineers, operations suppo...Show more
    Last updated: 13 hours ago • Promoted • New!
    Lead Data Engineer

    Lead Data Engineer

    Guidanz Inc • Pune, IN
    BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show more
    Last updated: 1 day ago • Promoted
    Azure Databricks

    Azure Databricks

    Kumaran Systems • Pune, IN
    We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault. The candidate should be proficient in Dat...Show more
    Last updated: 16 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    EXL • Pune, Maharashtra, India
    Develop high quality, secure and scalable data pipelines using spark, Scala / Python / Java on Hadoop or object storage like MinIO. Leverage technologies and solutions to innovate with increasingly larg...Show more
    Last updated: 30+ days ago • Promoted
    Data Support Engineer ( Azure, Databricks )

    Data Support Engineer ( Azure, Databricks )

    Kumaran Systems • Pune, IN
    We’re looking for a skilled Data Support Engineer with.The ideal candidate will serve as the escalation point for data ingestion, pipeline, and consumption issues while ensuring high system stabili...Show more
    Last updated: 9 days ago • Promoted
    Senior Solutions Architect (Data)

    Senior Solutions Architect (Data)

    Hillview Consulting Solutions • Pune, IN
    If candidate is in Mumbai this would be onsite in Andheri East, Mumbai, Maharashtra.We’re looking for a senior, hands-on. You’ll own architecture for ETL / ELT, data warehousing, analytics pipelines, ...Show more
    Last updated: 16 days ago • Promoted
    Azure Data Engineer

    Azure Data Engineer

    Paritas Recruitment • Pune, IN
    Azure Data Engineer - 6 Month+ Rolling Contract.Remote - (Sunday to Thursday working days).Paritas is working with a global IT Consultancy & leading Energy client who are seeking an experienced and...Show more
    Last updated: 1 day ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Ironbook AI • pune, maharashtra, in
    The ideal candidate will have strong experience with cloud platforms, modern ETL / ELT tools, and deep technical skills in Python, SQL, and distributed data frameworks. Design, develop, and maintain s...Show more
    Last updated: 10 hours ago • Promoted • New!
    Lead Data Engineer

    Lead Data Engineer

    Confidential • Pune, IN
    Expertise in big data technologies such as Apache Spark and real-time streaming technologies like Apache Kafka.Strong programming skills in Python, Java, C++, SQL etc. Advanced knowledge of a major ...Show more
    Last updated: 13 hours ago • Promoted • New!
    Senior Data Architect- Snowflake

    Senior Data Architect- Snowflake

    USEReady • Pune, IN
    USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage. USEReady partners with cloud...Show more
    Last updated: 14 days ago • Promoted