Big Data Engineer - Hadoop / Spark

Role Overview :

As a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.

Mandatory Skill Set :

• Extensive hands-on experience with the Hadoop ecosystem (HDFS, MapReduce, YARN).
• Strong expertise in Apache Spark for large-scale data processing and analytics.
• Proficiency in Hive writing complex HiveQL queries and managing Hive Metastore.
• Experience in developing and maintaining Apache Kafka-based streaming solutions.
• In-depth knowledge of HDFS storage management and optimization.
• Proven programming skills in Python or Scala for data pipeline development and automation.
• Strong expertise in writing complex SQL queries for data transformation and retrieval.
• Experience in workflow orchestration using Apache Airflow or Oozie to manage dependencies, schedule jobs, and monitor workflows.

Key Expectations :

Important Note :

We require candidates who have demonstrable, hands-on project experience in Big Data Engineering knowledge alone is not acceptable. Candidates with only academic or certification-level exposure will not be considered.

What We Offer :

(ref : hirist.tech)