Role : Kafka Administrator
Location : Pune , India
Exp : 10-15 years
Experience Required :
- 10+ years overall, with 5+ years in Kafka infrastructure management and operations.
- Must have successfully deployed and maintained Kafka clusters in production environments, with proven experience in securing, monitoring, and scaling Kafka for enterprise-grade data streaming.
Overview :
We are seeking an experienced Kafka Administrator to lead the deployment, configuration, and operational management of Apache Kafka clusters supporting real-time data ingestion pipelines.
The role involves ensuring secure, scalable, and highly available Kafka infrastructure for streaming flow records into centralized data platforms.
Key Responsibilities :
Architect and deploy Apache Kafka clusters with high availability.Implement Kafka MirrorMaker for cross-site replication and disaster recovery readiness.Integrate Kafka with upstream flow record sources using IPFIX-compatible plugins.Configure Kafka topics, partitions, replication, and retention policies based on data flow requirements.Set up TLS / SSL encryption, Kerberos authentication, and access control using Apache Ranger.Monitor Kafka performance using Prometheus, Grafana, or Cloudera Manager and ensure proactive alerting.Perform capacity planning, cluster upgrades, patching, and performance tuning.Ensure audit logging, compliance with enterprise security standards, and integration with SIEM tools.Collaborate with solution architects and Kafka developers to align infrastructure with data pipeline needs.Maintain operational documentation, SOPs, and support SIT / UAT and production rollout activities.Required Skills & Qualifications :
Proven experience in Apache Kafka, Kafka Connect, Kafka Streams, and Schema Registry.Strong understanding of IPFIX, nProbe Cento, and network flow data ingestion.Hands-on experience with Apache Spark (Structured Streaming) and modern data lake or DWH platforms.Familiarity with Cloudera Data Platform, HDFS, YARN, Ranger, and Knox.Deep knowledge of data security protocols, encryption, and governance frameworks.Excellent communication, documentation, and stakeholder management skills.(ref : hirist.tech)