Senior Kafka Data Engineer - Cloudera / Hadoop Ecosystem

Description : Role Overview

We are seeking a highly skilled Senior Kafka Data Engineer to design, build, and manage robust data pipelines that power both batch and real-time data processing across our enterprise data ecosystem. This role requires deep technical expertise in Cloudera, Azure Databricks, Kafka, and other cloud-based data platforms. The ideal candidate will be passionate about building scalable and high-performing data solutions, ensuring data quality, and enabling data-driven decision-making across the organization.

Key Responsibilities :

Data Pipeline Design & Development :

• Design, develop, test, and maintain end-to-end batch and streaming data pipelines using Cloudera, Apache Spark, Kafka, and Azure Data Services such as ADF, Databricks, and Cosmos DB.
• Build efficient ETL and ELT frameworks to transform raw data into structured, usable formats for downstream analytics and reporting.
• Implement data ingestion frameworks from multiple structured and unstructured sources (APIs, databases, streams, files, etc.).
• Automate and orchestrate complex data workflows using Azure Data Factory and Airflow (if applicable).

Performance Optimization & Data Quality :

Collaboration & Solution Design :

Innovation & Continuous Improvement :

Mentorship & Leadership :

Qualifications & Skills :

Required Technical Skills :

Preferred Skills :

Soft Skills :

(ref : hirist.tech)