Hadoop Administrator - Cluster Management

WFH and WFO

Job description :

Role : Hadoop Administrator

• General operational expertise such as good troubleshooting skills, understanding of systems capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
• Very good hands on experience in managing Hadoop cluster skills like HDFS, Spark, Hive, H-Catalog, Pig, Flume, HBase, Kafka, NiFi, YARN, Impala, Zookeeper, Oozie, SOLR, Oozie, Sentry, Encryption etc
• Experience in Hadoop cluster security and Data Governance LDAP, Kerberos, encryption at rest & in-motion, Data privacy / protection, implementing RBAC rules and Define and setup ACL policies.
• Ability to deploy Hadoop cluster, add and remove nodes, keep track of jobs, monitor critical parts of the cluster, configure name node high availability, schedule and configure it and take backups.
• Good understanding of SDLC and agile methodologiesInstallation and configuration of Hadoop clusters, including HDFS, MapReduce, Hive, Pig, HBase, and other related tools
• Managing and monitoring Hadoop clusters to ensure high availability and performance
• Planning and implementing data backup and disaster recovery strategies for Hadoop clusters
• Proactively monitoring and tuning Hadoop cluster performance to optimize resource utilization and prevent bottlenecks
• Providing technical support to developers and end-users as needed
• Awareness of latest technologies and trends
• Logical thinking and problem solving skills along with an ability to collaborate
• Good knowledge of Linux as Hadoop runs on Linux, Administration, Shell, awk scripting
• Experience with open source Hadoop cluster and its configuration management
• Good understanding of OS concepts, process management and resource scheduling.
• Basics of networking, CPU, memory and storage.
• Perform Tuning and Increase Operational efficiency on a continuous basis
• Monitor health of the platforms and Generate Performance Reports and Monitor and provide continuous improvements
• Perform optimization capacity planning of a large multi-tenant cluster
• Allocate and manage compute, memory, storage, number of name-node objects for individual pools and user groups.
• Upgrading Hadoop software and its ecosystem

(ref : hirist.tech)