Kafka Admin

Perform Admin functions, Setting Kafka cluster and responsible for cluster migration. Monitoring activities (Support team) and Hands on Monitoring tool Confluent Control Centre, Cloudera Manager Deployment and Management of Confluent Kafka. Performance monitoring and investigation and Streams based application problem investigation. Container Orchestration (Kubernetes) Container technologies (Docker) Infrastructure as code (Terraform, Concourse, Veracode) Service Monitoring and Graphing (Prometheus + Grafana, Splunk) Cluster maintenance activities all Environments

Adding node and removing node on prod cluster and rebalancing DevOps Pipeline setup CI / CD for Kafka Ansible, Deployments, AZURE data bricks, On premises Hadoop, Spark etc. all the data pipelines To work on Kafka Prerequisites of infra and network Confluent Kafka license configuration for the components / nodes in UAT and PROD

Non Prod cluster activities and migration plan Support and adhere to the standards and make as like Production.

Disaster recovery setup and regular failover drill runs on the cluster.

Develop Prerequisite scripts and automation scripts for health check and monitoring the cluster.

Automate monitoring and proactive alerts. App Dynamics Integration from Kafka and Log Analyses (Splunk Integration for the logs) Prepare and execute load / performance test cases. Automation of monitoring and ensure cluster healthy with the best practices.

Security configuration and protocol as per the Mashreq ISG requirements LDAP and Cluster access strategy

24x7 availability will be for Critical Production issue marked Severity 1 (or P1 in Confluent terms) only and NOT Sev2 or Sev3 Qualifications BE or degree in Computer Science or a related technical field. 7+ years of experience in Confluent Kafka / Apache Kafka cluster installation and maintenance. Expert admin experience in Linux and distributed cluster environments. Good Experience in Ansible and DevOps for Admin activities.

Skills Required

Hadoop, Azure Data Bricks, Prometheus, Grafana, Terraform, Docker, Linux, Ansible, Spark, Apache Kafka, Splunk, Kubernetes