We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.
The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities :
Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
Write efficient HiveQL and SQL queries to handle large-scale datasets.
Perform performance tuning and optimization of distributed data systems.
Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
Manage and schedule workflows using Apache Airflow or Oozie.
Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills :
Proven experience in Big Data Engineering with a focus on PySpark.
Strong knowledge of HDFS, Hive, Sqoop, and related tools.
Proficiency in SQL / HiveQL for large datasets.
Expertise in performance tuning and optimization of distributed systems.
Familiarity with Agile methodology and collaborative team practices.
Experience with workflow orchestration tools (Airflow / Oozie).
Strong problem-solving, analytical, and communication skills.
Good to Have :
Knowledge of data modeling and data warehousing concepts.
Exposure to DevOps practices and CI / CD pipelines for data engineering.
Experience with other Big Data frameworks such as Spark Streaming or Kafka.
Create a job alert for this search
Data Big Data • India
Related jobs
Data Science Specialist
SkillsBridge.AI • Nagpur, IN
The Data Science team at skillsbridge is at the forefront of innovation, leveraging AI and machine learning to transform skill assessment, career development, and talent management.We build intelli...Show more
Last updated: 2 days ago • Promoted
Data Migration Specialist
Atlas - the Recruitment Platform • Nagpur, IN
Before you start reading, please read this carefully!.I've spent a good amount of time thinking about what to write : D.Anyone who successfully passes our 30 minute SQL assessment will receive $30 h...Show more
Last updated: 30+ days ago • Promoted
Data Science Specialist
LawSikho • Nagpur, IN
N8N Developer / Automation Engineer.The ideal candidate should have strong experience with.Integrate multiple systems and APIs (REST, GraphQL, etc.
Monitor, troubleshoot, and optimize existing workf...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
System Soft Technologies • Nagpur, IN
Location : Remote (3–4-hour time zone overlaps with EST if off shore).Experience with next flow is required, as the consultant will make targeted enhancements to existing workflows and pipelines.Whi...Show more
Last updated: 14 hours ago • Promoted • New!
Senior Data Engineer
DataOrbit AI • Nagpur, IN
DataOrbit AI is a leading AI consulting firm dedicated to empowering small and medium investment managers to achieve AI readiness.
By uniting strategy, data preparation, technology, and execution, w...Show more
Last updated: 1 day ago • Promoted
Senior Bigdata Developer_ Exp : 6+ Years
Atyeti Inc • nagpur, maharashtra, in
Analyze, organize and process raw data using Bigdata technologies and Scala.Perform data validation, cleaning and transformation using big data technologies, Spark and Scala.Ingest and manage data ...Show more
Last updated: 7 hours ago • Promoted • New!
Data Analyst
Kindred • Nagpur, IN
Kindred is a tech Social Enterprise that transforms e-commerce transactions into a force for good.Kindred's tech empowers 500 million users with 550,000+ online savings, from 100,000+ advertisers, ...Show more
Last updated: 14 hours ago • Promoted • New!
Data Analyst
DraconX • Nagpur, IN
AI business automation and AI-driven SaaS platforms.Specializing in MVP development, custom software, and automation systems, we help startups and enterprises accelerate growth and innovation.Our e...Show more
Last updated: 20 days ago • Promoted
Data Annotator
Innodata Inc. • Nagpur, IN
Innodata (NASDAQ : INOD) is a leading data engineering company.With more than 2,000 customers and operations in 13 cities around the world, we are an AI technology solutions provider-of-choice for 4...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Live Connections • Nagpur, IN
Hiring : Data Engineer | 9-12 Years Experience | Remote (India).Python, SQL, Big Data, Cloud, Snowflake.We are looking for an experienced.
The ideal candidate will have deep experience in.Python, SQL...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
Aceolution • Nagpur, IN
Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show more
Last updated: 30+ days ago • Promoted
Senior Data Architect- Snowflake
USEReady • Nagpur, IN
USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage.
USEReady partners with cloud...Show more
Last updated: 4 days ago • Promoted
Senior Snowflake Data Engineer
Luxoft • nagpur, maharashtra, in
We are seeking a highly skilled Snowflake Data Engineer with 7 years of IT experience to design, build, and optimize scalable data pipelines and cloud-based solutions across AWS, Azure, and GCP.The...Show more
Last updated: 7 hours ago • Promoted • New!
Data Analyst (DOMO Specialist) || Healthcare industry || Upto 30 LPA
Talentiser • India, India
As a Data Analyst, you will be responsible for designing, developing, and maintaining dashboards, reports, and data visualizations in.
You will partner closely with business and technical teams to p...Show more
Last updated: 3 days ago • Promoted
Senior Data Engineer
Mindlance • Nagpur, IN
We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show more
Last updated: 4 days ago • Promoted
Data Engineer (GCP)
HISH IT SERVICES • Nagpur, IN
We have a new urgent GCP Data Engineer opportunity open to support a migration initiative from Teradata to Cerebro (BigQuery).
This role requires a hands-on developer who can collaborate closely wit...Show more
Last updated: 12 days ago • Promoted
Senior Data Engineer
SAIVA AI • Nagpur, IN
We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
Last updated: 30+ days ago • Promoted
Data Migration Specialist
Berg Search • Nagpur, IN
A revolutionary recruitment platform leveraging cutting-edge generative AI technology to streamline your workflow.They're redefining recruitment through automation and intelligence.The platform int...Show more