Hadoop Lead

This is a Big Data Engineer role with a strong focus on data warehousing and analytics within the AWS cloud platform. The position requires experience in building and managing data pipelines, using a range of technologies for data transformation, and leading projects from design to implementation.

Key Responsibilities

• Data Pipeline & Transformation : You'll have experience with data pipelines using technologies like Apache Kafka, Storm, Spark , or AWS Lambda . The role requires at least 2 years of experience writing PySpark for data transformation. You'll also work with terabyte data sets using relational databases and SQL .
• Data Warehousing & ETL : The position demands at least 2 years of experience with data warehouse technical architectures, ETL / ELT processes, and data security. You'll be responsible for designing data warehouse solutions and integrating various technical components.
• Project Leadership : You'll have 2 or more years of experience leading data warehousing and analytics projects, specifically utilizing AWS technologies like Redshift, S3 , and EC2 .
• Methodologies & Tools : You'll use Agile / Scrum methodologies to iterate on product changes and work through backlogs. Exposure to reporting tools like QlikView or Tableau is a plus, as is familiarity with Linux / Unix scripting .

Skills Required

Aws, Apache Kafka, Spark, Aws Lambda, Pyspark, S3, Ec2