Kafka Administrator - IAC Terraform

Key Responsibilities :

• Kafka Consulting & Architecture : Consult with inquiring teams on how to effectively leverage Kafka within their data pipelines, providing expert guidance and solutions.
• Cluster Management (IaC) : Architect, build, and provide ongoing support for existing and new Kafka clusters, strictly adhering to Infrastructure-as-Code (IaC) best practices.
• Traffic Routing & Integration : Partner with Splunk teams to strategically route traffic through Kafka, utilizing open-source agents and collectors deployed via Chef.
• Health & Performance Remediation : Proactively identify and remediate any health issues within Kafka clusters to ensure optimal performance and stability.
• Operational Automation : Identify opportunities to automate operational processes within the team, developing and implementing solutions where possible.
• Monitoring & Alerting : Create new and / or update existing monitoring dashboards and alerts as needed to ensure comprehensive visibility into Kafka health.
• CI / CD Pipeline Management : Manage and optimize a continuous improvement / continuous development (CI / CD) pipeline for Kafka-related deployments.
• Innovation & PoC : Perform Proof-of-Concepts (PoCs) on new components and technologies to expand and enhance the teams Kafka offerings.
• Documentation : Maintain thorough documentation of Kafka configurations, procedures, and architectural Skills & Qualifications :
• Overall Experience : 7 to 10 years of professional experience in IT infrastructure or software development.
• Kafka Administration : Minimum 2+ years of hands-on experience with Kafka clustering & administration.
• Infrastructure-as-Code (IaC) : Minimum 2+ years of experience building, deploying, and supporting multiple Kafka clusters using IaC best practices.
• Automation : Proven experience developing automated processes within and around Kafka to supplement service operations.
• Data Pipeline Architecture : Experience working with multiple teams to build and architect data pipeline solutions where Kafka will be a key component.
• Operating Systems : Proficient experience with Linux / Unix and system management.
• IaC Tools : Hands-on IaC experience with Virtual / Physical Servers using tools like Chef, Ansible, Jenkins, Artifactory, etc.
• CI / CD : Strong understanding of Git workflows and continuous improvement / continuous development (CI / CD) concepts.
• Communication : Strong verbal and written communication skills.
• Analytical & Technical : Strong technical acumen and analytical skills.
• Methodology : Experience working in Agile / Lean Qualifications :
• Knowledge and experience with Splunk, Elastic, Kibana, and Grafana for monitoring and visualization.
• Knowledge and experience with log collection agents such as Open-Telemetry, Fluent Bit, FluentD, Beats, and LogStash.
• Knowledge and experience with Kubernetes / Docker for container orchestration.
• Knowledge and experience with Kafka-Connect.
• Knowledge and experience with cloud platforms like AWS or Azure.
• Knowledge and experience with Streaming Analytics.

ref : hirist.tech)