Big Data SpecialistAtomic North • India

Big Data Specialist

Atomic North • India

19 hours ago

Job description

Big Data Engineer

Location : Pune / Nagpur (WFO)

Experience Required : 8+ Years

Long term onsite to Abu Dhabi

Job Summary :

We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.

The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.

Key Responsibilities :

Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.

Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.

Write efficient HiveQL and SQL queries to handle large-scale datasets.

Perform performance tuning and optimization of distributed data systems.

Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.

Manage and schedule workflows using Apache Airflow or Oozie.

Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills :

Proven experience in Big Data Engineering with a focus on PySpark.

Strong knowledge of HDFS, Hive, Sqoop, and related tools.

Proficiency in SQL / HiveQL for large datasets.

Expertise in performance tuning and optimization of distributed systems.

Familiarity with Agile methodology and collaborative team practices.

Experience with workflow orchestration tools (Airflow / Oozie).

Strong problem-solving, analytical, and communication skills.

Good to Have :

Knowledge of data modeling and data warehousing concepts.

Exposure to DevOps practices and CI / CD pipelines for data engineering.

Experience with other Big Data frameworks such as Spark Streaming or Kafka.

Create a job alert for this search

Data Big Data • India

Related jobs

Data Science Specialist

SkillsBridge.AI • Nagpur, IN

The Data Science team at skillsbridge is at the forefront of innovation, leveraging AI and machine learning to transform skill assessment, career development, and talent management.We build intelli...Show more

Last updated: 2 days ago • Promoted

Data Migration Specialist

Atlas - the Recruitment Platform • Nagpur, IN

Before you start reading, please read this carefully!.I've spent a good amount of time thinking about what to write : D.Anyone who successfully passes our 30 minute SQL assessment will receive $30 h...Show more

Last updated: 30+ days ago • Promoted

Data Science Specialist

LawSikho • Nagpur, IN

N8N Developer / Automation Engineer.The ideal candidate should have strong experience with.Integrate multiple systems and APIs (REST, GraphQL, etc. Monitor, troubleshoot, and optimize existing workf...Show more

Last updated: 30+ days ago • Promoted

Data Engineer

System Soft Technologies • Nagpur, IN

Location : Remote (3–4-hour time zone overlaps with EST if off shore).Experience with next flow is required, as the consultant will make targeted enhancements to existing workflows and pipelines.Whi...Show more

Last updated: 14 hours ago • Promoted • New!

Senior Data Engineer

DataOrbit AI • Nagpur, IN

DataOrbit AI is a leading AI consulting firm dedicated to empowering small and medium investment managers to achieve AI readiness. By uniting strategy, data preparation, technology, and execution, w...Show more

Last updated: 1 day ago • Promoted

Senior Bigdata Developer_ Exp : 6+ Years

Atyeti Inc • nagpur, maharashtra, in

Analyze, organize and process raw data using Bigdata technologies and Scala.Perform data validation, cleaning and transformation using big data technologies, Spark and Scala.Ingest and manage data ...Show more

Last updated: 7 hours ago • Promoted • New!

Data Analyst

Kindred • Nagpur, IN

Kindred is a tech Social Enterprise that transforms e-commerce transactions into a force for good.Kindred's tech empowers 500 million users with 550,000+ online savings, from 100,000+ advertisers, ...Show more

Last updated: 14 hours ago • Promoted • New!

Data Analyst

DraconX • Nagpur, IN

AI business automation and AI-driven SaaS platforms.Specializing in MVP development, custom software, and automation systems, we help startups and enterprises accelerate growth and innovation.Our e...Show more

Last updated: 20 days ago • Promoted

Data Annotator

Innodata Inc. • Nagpur, IN

Innodata (NASDAQ : INOD) is a leading data engineering company.With more than 2,000 customers and operations in 13 cities around the world, we are an AI technology solutions provider-of-choice for 4...Show more

Last updated: 30+ days ago • Promoted

Senior Data Engineer

Live Connections • Nagpur, IN

Hiring : Data Engineer | 9-12 Years Experience | Remote (India).Python, SQL, Big Data, Cloud, Snowflake.We are looking for an experienced. The ideal candidate will have deep experience in.Python, SQL...Show more

Last updated: 30+ days ago • Promoted

Data Engineer

Aceolution • Nagpur, IN

Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show more

Last updated: 30+ days ago • Promoted

Senior Data Architect- Snowflake

USEReady • Nagpur, IN

USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage. USEReady partners with cloud...Show more

Last updated: 4 days ago • Promoted

Senior Snowflake Data Engineer

Luxoft • nagpur, maharashtra, in

We are seeking a highly skilled Snowflake Data Engineer with 7 years of IT experience to design, build, and optimize scalable data pipelines and cloud-based solutions across AWS, Azure, and GCP.The...Show more

Last updated: 7 hours ago • Promoted • New!

Data Analyst (DOMO Specialist) || Healthcare industry || Upto 30 LPA

Talentiser • India, India

As a Data Analyst, you will be responsible for designing, developing, and maintaining dashboards, reports, and data visualizations in. You will partner closely with business and technical teams to p...Show more

Last updated: 3 days ago • Promoted

Senior Data Engineer

Mindlance • Nagpur, IN

We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show more

Last updated: 4 days ago • Promoted

Data Engineer (GCP)

HISH IT SERVICES • Nagpur, IN

We have a new urgent GCP Data Engineer opportunity open to support a migration initiative from Teradata to Cerebro (BigQuery). This role requires a hands-on developer who can collaborate closely wit...Show more

Last updated: 12 days ago • Promoted

Senior Data Engineer

SAIVA AI • Nagpur, IN

We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more

Last updated: 30+ days ago • Promoted

Data Migration Specialist

Berg Search • Nagpur, IN

A revolutionary recruitment platform leveraging cutting-edge generative AI technology to streamline your workflow.They're redefining recruitment through automation and intelligence.The platform int...Show more

Last updated: 10 days ago • Promoted