Hadoop Administrator

Job Description

• Manage, configure, and administer Hadoop ecosystem components including HDFS, YARN, Zookeeper, Hue, Ranger, Spark, and Hive.
• Perform cluster monitoring, tuning, troubleshooting, and performance optimization across on-prem and cloud environments.
• Implement security practices including Kerberos authentication, LDAP integration, and encryption (at rest / in transit).
• Set up and maintain high availability, backup, and disaster recovery solutions .
• Administer and optimize AWS EMR clusters , including scaling strategies and cost optimization.
• Work with AWS services : Glue (ETL pipelines), Athena (query optimization), S3 (data lake management), and IAM (access and policy management).
• Develop and maintain automation scripts (Shell, Python) and Infrastructure-as-Code solutions (CloudFormation).
• Support Hive Meta store , query optimization, and Spark performance tuning .
• Oversee logging, monitoring, and auditing practices using tools such as CloudWatch and other monitoring solutions.
• Collaborate with data engineering and analytics teams to support ongoing projects and optimize workflows.

Requirements

Requirements

Proven hands-on experience in Hadoop Administration (on-prem & AWS). Strong knowledge of Linux system administration. Proficiency in Shell scripting (Bash) and Python scripting. Experience with cluster management, scaling, and troubleshooting. Expertise in AWS EMR, Glue, Athena, S3, IAM, and infrastructure automation (CloudFormation). Solid understanding of security frameworks and data governance. Strong problem-solving, analytical, and communication skills.