Big Data Engineer - ETL

Job Overview :

We are looking for a skilled and experienced Big Data Developer to join our data engineering team. The ideal candidate will have a strong foundation in Big Data technologies and a proven track record of building scalable and efficient data pipelines. You will play a critical role in enabling data-driven decision-making across the Responsibilities :

• Design, develop, and optimize big data pipelines using tools and technologies from the Hadoop ecosystem.
• Work with large datasets to extract, transform, and load (ETL) data for analytical and operational purposes.
• Build scalable data ingestion frameworks for real-time and batch processing.
• Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver high-quality solutions.
• Ensure data quality, governance, and compliance standards are maintained.
• Monitor and troubleshoot big data infrastructure and performance issues.
• Document design decisions, processes, and technical Technical Skills :
• Strong experience with Big Data technologies : Hadoop, Spark, Hive, HBase, Pig, Sqoop
• Hands-on experience with data processing frameworks like : Apache Spark (RDD / DataFrame / SQL APIs), Kafka, Flink
• Proficiency in programming languages such as : Java, Scala, Python
• Strong knowledge of SQL and NoSQL databases (e.g., MongoDB, Cassandra).
• Experience in developing and managing data workflows using Apache Airflow, Oozie, or similar tools.
• Familiarity with cloud platforms (AWS / Azure / GCP) and cloud-native big data services (e.g., EMR, Databricks, BigQuery).
• Knowledge of CI / CD pipelines, version control tools like Git, and DevOps practices is a plus.
• Experience in data modeling, data governance, and performance :
• Bachelors or Masters degree in Computer Science, Information Technology, or a related field.
• 58 years of experience in Big Data development and data engineering roles.
• Strong analytical, problem-solving, and communication skills.
• Ability to work in a collaborative, Agile / Scrum environment.

ref : hirist.tech)