Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.
Design, develop, and optimize AWS pipeline as applicable.
Implement and manage GitHub asset bundles for version control, CI / CD workflows, and collaborative development.
Leverage Databricks capabilities including Delta Lake, MLflow, and notebooks to build robust data solutions.
Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality, production-ready datasets.
Ensure data quality, integrity, and security across all stages of the pipeline.
Monitor and troubleshoot performance issues in Spark jobs and Databricks clusters.
Document technical designs, workflows, and best practices.
Required Qualifications
Minimum 5+ years of hands-on experience with Spark in a production environment.
Strong proficiency in Databricks platform and ecosystem.
Solid understanding of GitHub asset bundle management, including branching strategies, pull requests, and release workflows.
Proficient in Python, Scala, or SQL for data engineering tasks.
Experience with Delta Lake, Structured Streaming, and Databricks Workflows.
Familiarity with cloud platform (AWS) and AWS data services.
Excellent problem-solving and communication skills.
Create a job alert for this search
Data Engineer • India
Related jobs
Promoted
Data Engineer
Kline + CompanyNagpur, IN
Here at Kline our data capabilities have grown exponentially over the last four years.Having gone through a rapid digitization process and becoming a cloud-native corporation, we are looking for to...Show moreLast updated: 7 days ago
Promoted
Senior Data Engineer
InfogainNagpur, IN
Big Data Engineer (Lead) : As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers.
You will work closely with clients and team members to understand their req...Show moreLast updated: 30+ days ago
Promoted
Data Engineer
Randstad EnterpriseNagpur, IN
Shift Timing : 2 : 00 Pm - 11 : 00 Pm.Experience : 2- 4 years relevant only ( this is a Junior position with us ).GCP - 2 years minimum working Experience.
Worked with global stakeholders.Randstad Sourc...Show moreLast updated: 30+ days ago
Promoted
Senior Data Engineer
Confidential JobsNagpur, IN
This role involves collaborating with cross-functional teams to ensure data reliability, scalability, and performance.The candidate will work closely with data scientists, analysts and software eng...Show moreLast updated: 2 days ago
Promoted
New!
Senior Data Engineer
HealthyMe AINagpur, IN
We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show moreLast updated: 18 hours ago
Promoted
Principal Data Engineer
XebiaNagpur, IN
We’re Hiring : Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week).Any Xebia Location (Hybrid – 3 days in office per week).
Data Engineering with 4+ years team leadership...Show moreLast updated: 29 days ago
Promoted
Data Engineer
XebiaNagpur, IN
We’re Hiring : Data Engineer | Xebia.Any Xebia location (Hybrid, 3 days office per week).Immediate to 2 weeks – only apply if you can join early.
Databricks, Python, SQL, and Postgres.The ideal candi...Show moreLast updated: 12 days ago
Promoted
Data Engineer
Data-Hat AINagpur, IN
Data Engineering & AI Solutions .Lead Data Solutions Architect .International travel required (up to 30–40%) .We are hiring a senior-level Data Engineer to lead the design, development, and optimiz...Show moreLast updated: 2 days ago
Promoted
Senior Data Engineer
CESNagpur, IN
As part of our data engineering team, you’ll build and maintain.This is a hands-on role where you’ll design and optimize modern data infrastructure, ensuring reliability, scalability, and performan...Show moreLast updated: 12 days ago
Promoted
Lead Data Engineer
Eucloid Data SolutionsNagpur, IN
Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 14 days ago
Promoted
Data Engineer
IntraEdgeNagpur, IN
Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 30+ days ago
Promoted
Senior Data Engineer
CareerXperts ConsultingNagpur, IN
We are looking for a highly skilled.The ideal candidate will have strong expertise in modern data engineering tools and frameworks, along with the ability to collaborate closely with data scientist...Show moreLast updated: 2 days ago
Promoted
Lead Data Engineer
Searce IncIndia
Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’.It signifies our way of working : To improve to the finest degree of excellence, ‘solving for better’ every time.Searcians...Show moreLast updated: 2 days ago
Promoted
Data Engineer
RevXNagpur, IN
RevX helps app businesses acquire and reengage users via programmatic to retain, monetize, and accelerate revenue.We're all about taking your app businesses to a new growth level.We rely on data sc...Show moreLast updated: 30+ days ago
Promoted
Data Engineer
Kumaran SystemsNagpur, IN
Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 14 days ago
Promoted
Data Engineer
Vriba SolutionsIndia, India
Design, develop & maintain ETL / ELT pipelines.Ingest & transform data from APIs, DBs, files, streams.Build real-time & batch processing solutions.
Data validation, quality & cleansing.Translate busin...Show moreLast updated: 2 days ago
Promoted
Data Platform Engineer
Intuitive.CloudNagpur, IN
With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization.
Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 12 days ago
Promoted
Data Engineer
EveriseNagpur, IN
Join us on our mission to elevate customer experiences for people around the world.As a member of the Everise family, you will be part of a global experience company that believes in being people-f...Show moreLast updated: 13 days ago
Promoted
Data / ML Engineer
GleantapNagpur, IN
Gleantap is a customer engagement platform powering fitness, wellness, and service businesses.Python (scikit-learn, LightGBM, XGBoost).
Set up retraining and monitoring pipelines (Airflow, MLflow, o...Show moreLast updated: 29 days ago
Promoted
Data Engineer
Bahwan CyberTekNagpur, IN
Job Title : Data Engineer – Google Cloud Platform (GCP).We are seeking a skilled and motivated Data Engineer with hands-on experience in building scalable data pipelines and cloud-native data soluti...Show moreLast updated: 24 days ago