Big Data SpecialistAtomic North • Gulbarga, IN

No longer accepting applications

Big Data Specialist

Atomic North • Gulbarga, IN

4 days ago

Job description

Big Data Engineer

Location : Pune / Nagpur (WFO)

Experience Required : 8+ Years

Long term onsite to Abu Dhabi

Job Summary :

We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.

The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.

Key Responsibilities :

Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.

Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.

Write efficient HiveQL and SQL queries to handle large-scale datasets.

Perform performance tuning and optimization of distributed data systems.

Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.

Manage and schedule workflows using Apache Airflow or Oozie.

Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills :

Proven experience in Big Data Engineering with a focus on PySpark.

Strong knowledge of HDFS, Hive, Sqoop, and related tools.

Proficiency in SQL / HiveQL for large datasets.

Expertise in performance tuning and optimization of distributed systems.

Familiarity with Agile methodology and collaborative team practices.

Experience with workflow orchestration tools (Airflow / Oozie).

Strong problem-solving, analytical, and communication skills.

Good to Have :

Knowledge of data modeling and data warehousing concepts.

Exposure to DevOps practices and CI / CD pipelines for data engineering.

Experience with other Big Data frameworks such as Spark Streaming or Kafka.

Create a job alert for this search

Data Big Data • Gulbarga, IN

Related jobs

Data Engineer

Tata Consultancy Services • Gulbarga, IN

TCS has been a great pioneer in feeding the fire of Techies like you.We are a global leader in the technology arena and there’s nothing that can stop us from growing together.Your role is of key im...Show more

Last updated: 30+ days ago • Promoted

Snowflake Data Engineer

Newpage Solutions • Gulbarga, IN

Location : Remote | Type : Contract.Newpage Solutions is a global digital health innovation company helping people live longer, healthier lives. We partner with life sciences organizations—including p...Show more

Last updated: 13 days ago • Promoted

Senior Data Analyst

Ruder Finn India • Gulbarga, IN

Ruder Finn is seeking a Senior Data Analyst (Data Analyst III) to join its award-winning Emerging Technology & Analytics team. The chosen candidate will support the agency’s new business efforts by ...Show more

Last updated: 24 days ago • Promoted

Data Annotator

Innodata Inc. • Gulbarga, IN

Innodata (NASDAQ : INOD) is a leading data engineering company.With more than 2,000 customers and operations in 13 cities around the world, we are an AI technology solutions provider-of-choice for 4...Show more

Last updated: 24 days ago • Promoted

Data Migration Specialist

Atlas - the Recruitment Platform • Gulbarga, IN

Before you start reading, please read this carefully!.I've spent a good amount of time thinking about what to write : D.Anyone who successfully passes our 30 minute SQL assessment will receive $30 h...Show more

Last updated: 30+ days ago • Promoted

Data Analyst

DraconX • Gulbarga, IN

AI business automation and AI-driven SaaS platforms.Specializing in MVP development, custom software, and automation systems, we help startups and enterprises accelerate growth and innovation.Our e...Show more

Last updated: 13 days ago • Promoted

Data Migration Specialist

Berg Search • Gulbarga, IN

A revolutionary recruitment platform leveraging cutting-edge generative AI technology to streamline your workflow.They're redefining recruitment through automation and intelligence.The platform int...Show more

Last updated: 3 days ago • Promoted

Immediate Opening for Data Engineer (Databricks & D365)

GyanSys Inc. • Gulbarga, IN

Need to have databricks experience as well as experience with handling D365 data.Need strong SQL skills, strong analytical skills.Show more

Last updated: 4 days ago • Promoted

Big Data Developer SME

Information Tech Consultants • Gulbarga, IN

Masters or Ph D in Statistics, Mathematics, Computer Science, or another quantitative field.Fluent English (written / spoken). Experience Developing Machine Learning / Data Science models, from coding...Show more

Last updated: 6 days ago • Promoted

Senior Data Associate

Qode • Gulbarga, IN

To apply : Review the JD Below, and if interested, submit your application here : .We are building a next-generation multi-asset trading and research infrastructure that ingests millions of ticks per ...Show more

Last updated: 9 hours ago • Promoted • New!

Senior Data Engineer

SAIVA AI • Gulbarga, IN

We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more

Last updated: 30+ days ago • Promoted

Data Engineer (GCP)

HISH IT SERVICES • Gulbarga, IN

We have a new urgent GCP Data Engineer opportunity open to support a migration initiative from Teradata to Cerebro (BigQuery). This role requires a hands-on developer who can collaborate closely wit...Show more

Last updated: 5 days ago • Promoted

Data Engineer

Aceolution • Gulbarga, IN

Data Engineer – Python Expert(Freelance Role).We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) developmen...Show more

Last updated: 30+ days ago • Promoted

Data Scientist

upGrad Rekrut • Gulbarga, IN

You will work on end-to-end data-driven solutions — from data cleaning to model deployment — enabling key insights for strategic decisions. Analyze large, complex datasets to extract meaningful busi...Show more

Last updated: 4 days ago • Promoted

SAP Data Quality & Master Data Specialist

TekGenio • Gulbarga, IN

SAP Data Quality & Master Data Specialist (MRO / MDO / Content.We are seeking an experienced(4Y+) SAP Data Quality & Master Data Specialist to manage,. SAP systems with a focus on MRO (Maintenance, Repa...Show more

Last updated: 1 day ago • Promoted

Senior Data Engineer

HealthyMe AI • Gulbarga, IN

We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show more

Last updated: 30+ days ago • Promoted

Senior Data Engineer - Data Acquisition

InfoBeans • Gulbarga, IN

We are seeking a highly skilled.Senior Data Engineer – Data Acquisition (ODS).The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformatio...Show more

Last updated: 6 days ago • Promoted

Lead Data Engineer

NexGen Tech Solutions • Gulbarga, IN

Lead Data Engineer– Wealth Management.Must Have : Wealth Management Domain.We are seeking a highly skilled.Experience in building end-to-end Data Lake and Modern Data Platform programs covering the ...Show more

Last updated: 9 hours ago • Promoted • New!