Senior Databricks Engineer

Newscape Consultingpune, India

3 days ago

Job description

About the Role

We are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data ecosystems to power business insights and AI / ML initiatives.

Key Responsibilities

Design &

Development :
Build and maintain scalable ETL / ELT data pipelines using Apache Spark and Databricks.
Develop Delta Lake architectures and optimize data storage for performance and cost.
Automate data ingestion, transformation, and orchestration processes across multiple data sources.
Build different stages of medallion architecture such as Bronze, Silver and Gold layer

Data Architecture & Modeling :

Collaborate with data architects to design data lakehouse and warehouse solutions.

Implement data quality frameworks, versioning, and governance standards (e.g., Unity Catalog).

Optimize Spark jobs, cluster configurations, and query performance.

Cloud & Infrastructure :

Work with Azure Databricks or AWS to configure and manage cloud-based data platforms.

Develop and manage CI / CD pipelines for Databricks notebooks, workflows, and jobs.

Implement security best practices for data access, encryption, and compliance.

Collaboration & Leadership :

Partner with data scientists, analysts, and product teams to enable data-driven decisions.

Provide technical mentorship to junior engineers.

Contribute to strategic planning and architecture reviews.

Required Qualifications

Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or related field.

6+ years of experience in data engineering or software development.

4+ years hands-on experience with Databricks and Apache Spark.

Strong proficiency in Python, SQL, and PySpark.

Experience with Delta Lake, Data Lakehouse, and Data Warehousing concepts.

Solid understanding of data modeling, ETL design, medallion architecture and distributed data processing.

Familiarity with CI / CD tools (Azure DevOps, GitHub Actions, Jenkins, etc.).

Cloud platform expertise in Azure, AWS, or GCP.

Preferred Qualifications

Experience with machine learning pipelines in Databricks (MLflow).

Knowledge of data governance and compliance frameworks.

Experience integrating Databricks with BI tools like Power BI, Tableau, or Looker.

Certifications such as :

Databricks Certified Data Engineer Associate or Professional

Azure Data Engineer Associate

AWS Big Data Specialty

Soft Skills

Strong problem-solving and analytical thinking.

Excellent communication and collaboration abilities.

Proactive mindset with ownership of deliverables.

Passion for learning emerging data technologies.

Why Join Us

Work with cutting-edge data technologies on large-scale projects.

Opportunity to shape the organization’s data strategy and architecture.

Collaborative and innovative culture focused on continuous growth.

Create a job alert for this search

Senior Engineer • pune, India

Related jobs

Promoted

Senior Data Engineer

Coditaspune, maharashtra, in

Understand Business requirements and implement scalable solutions.Translate complex technical and functional requirements into detailed designs. Develop a highly scalable, reliable, and high-perform...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

MSBC GroupPune, Maharashtra, India

As a Data Engineer, your primary role is to create and maintain high-performance data pipelines, warehouses, and cloud-based systems. This includes creating efficient batch and streaming pipelines, ...Show moreLast updated: 24 days ago

Promoted

Senior Data Engineer- AWS

InfoCeptspune, maharashtra, in

Key Result Areas and Activities : .Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements. Should be able to design comple...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

Mastekpune, maharashtra, in

We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show moreLast updated: 30+ days ago

Promoted

Senior Data Platform Engineer

Persistent Systemspune, maharashtra, in

We are seeking a Senior Platform Engineer to architect, build, and manage enterprise-scale data platforms that power advanced analytics and AI initiatives. This role blends cloud engineering, DevOps...Show moreLast updated: 22 days ago

Promoted

Databricks Data Engineer

AscendionPune, Maharashtra, India

Job Title : Senior Data Engineer Experience : 3+ years Location : Gurgaon / Pune / Bangalore Skills : PySpark, SQL, Databricks, AWS. Role Summary : We are looking for 3–4 experienced Databricks Devel...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

Guidanz IncPune, IN

BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show moreLast updated: 3 days ago

Promoted

Senior Data Engineer (AWS / Databricks)

Accoladepune, maharashtra, in

The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago

Promoted

Senior Databricks Engineer

Newscape ConsultingPune, Maharashtra, India

The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant data ecos...Show moreLast updated: 2 days ago

Promoted

Senior Snowflake Data Engineer

iVoyantPune, IN

One of our clients is looking for an experienced Senior Snowflake Data Engineer to join their team.We are seeking a Senior Data Engineer with 8+ years of experience in end-to-end data engineering a...Show moreLast updated: 3 days ago

Promoted

Senior Data Engineer

EXLPune, Maharashtra, India

We are seeking a Senior Data Engineer to build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging.You will...Show moreLast updated: 2 days ago

Promoted

Senior Data Engineer (ADB + Snowflake)

Persistent SystemsPune, India

We are looking for a highly skilled and innovative Senior Data Engineer with strong expertise in Azure Databricks, PySpark, and Snowflake, along with hands-on experience integrating AI tools such a...Show moreLast updated: 21 days ago

Promoted

Senior Data Engineer

SAIVA AIPune, IN

We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

Apptwarepune, maharashtra, in

The ideal candidate will take ownership of our end-to-end data strategy, ensuring scalability, reliability, and performance across multiple data systems. Redshift or PostgreSQL, DynamoDB, and Docume...Show moreLast updated: 2 days ago

Promoted

Senior Data Engineer

Straivepune, maharashtra, in

The ideal candidate is a strong software engineer with hands-on experience in Spark (3.You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver da...Show moreLast updated: 30+ days ago

Promoted
New!

Senior Data Engineer – Cloud Data Solutions

Elongo Global SolutionsPune, Maharashtra, India

About the Role We’re seeking a Senior Data Engineer with strong expertise in SQL, Talend, Google Cloud Platform (GCP) BigQuery , and Microsoft Azure to join our growing data engineering team.I...Show moreLast updated: 7 hours ago

Promoted

Senior Data Engineer

Persistent Systemspune, maharashtra, in

We are looking for an experienced Azure Databricks and Snowflake Data Engineer to lead and execute complex user migrations from legacy systems to modern cloud-based solutions.This role involves col...Show moreLast updated: 24 days ago

Promoted
New!

Senior Data Engineer ( Palantir)

Lear Corporationpune, maharashtra, in

Lear, a global automotive technology leader in Seating and E-Systems, is Making every drive better by delivering intelligent in-vehicle experiences for customers around the world.With over 100 year...Show moreLast updated: 3 hours ago