Kafka Engineer

Company Description

BAPON is a forward-thinking IT consulting company dedicated to empowering businesses with intelligent automation, seamless digital integration, and transformative cloud solutions. We are Cyber Essential Plus and ISO 9001 & ISO 27001 certified, ensuring the highest standards of quality, security, and compliance. Our core services include intelligent automation, digital integration, cloud transformation, and IT strategy & planning. We specialize in delivering innovative IT solutions that enhance operational efficiency and drive growth, particularly for startups and emerging companies.

Role Description

This is a contract role for a Kafka Engineer, based in Chennai. The Kafka Engineer will be responsible for designing, implementing, and managing Kafka-based data pipelines, maintaining and enhancing Kafka clusters, and ensuring high availability and performance of Kafka infrastructure. Regular tasks include monitoring Kafka environments, troubleshooting issues, collaborating with development teams to integrate Kafka solutions, and optimizing data streaming processes.

Responsibilities

• Design, deploy, and manage Apache Kafka clusters across on-prem and cloud environments.
• Handle end-to-end setup, including installation, configuration, security hardening, and monitoring.
• Build automated deployment pipelines for Kafka using tools like Ansible, Terraform, Helm, or Kubernetes.
• Ensure high availability, disaster recovery, and fault tolerance for Kafka clusters.
• Manage topics, partitions, ACLs, quotas, and schemas in line with business requirements.
• Integrate Kafka with downstream systems (databases, data lakes, stream processors, analytics platforms).
• Implement monitoring and alerting (Prometheus, Grafana, Confluent Control Center, or similar).
• Optimize performance, troubleshoot issues, and provide root cause analysis for production incidents.
• Document deployment processes and provide knowledge transfer to operations teams.

Required Skills

Nice to Have

Show more

Show less

Skills Required

Networking, Prometheus, Grafana, Security, Linux Administration, Shell Scripting, Apache Kafka