Big Data Lead

Big Data Lead - CREQ Description Job Description :

5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive

Strong knowledge of working with large data sets and high-capacity big data processing platform

Strong experience in Unix and Shell scripting

Advanced knowledge of the Hadoop ecosystem and its components

In-depth knowledge of Hive, Shell scripting, Python, Spark

Ability to write MapReduce jobs

Experience using Job Schedulers like Autosys

Hands on experience in HiveQL

Good knowledge on Hadoop Architecture and HDFS

Strong knowledge of working with large data sets and high-capacity big data processing platform

Strong experience in Unix and Shell scripting

Experience with jenkins for Continuous Integration

Experience using Source Code and Version Control Systems like Bitbucket, Git

Good to have experience on Agile Development

Responsibilities : Develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained

Ensures solutions are well designed with maintainability / ease of integration and testing built-in from the outset

Participates and guides team in estimating work necessary to realize a story / requirement through the software delivery lifecycle

Responsible for developing and delivering complex software requirements to accomplish business goals

Ensures that software is developed to meet functional, non-functional, and compliance requirements

Codes solutions, Unit testing and ensure the solution can be integrated successfully into the overall application / system with clear robust, and well-tested interfaces

Required Skills : Hadoop, Hive, HDFS, Spark, Python, Unix

Nice to have skills : AML Domain knowledge

Primary Location Chennai, Tamil Nadu, India Job Type Experienced Primary Skills PySpark Years of Experience 5 Qualification

Job Description :

5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive

Strong knowledge of working with large data sets and high-capacity big data processing platform

Strong experience in Unix and Shell scripting

Advanced knowledge of the Hadoop ecosystem and its components

In-depth knowledge of Hive, Shell scripting, Python, Spark

Ability to write MapReduce jobs

Experience using Job Schedulers like Autosys

Hands on experience in HiveQL

Good knowledge on Hadoop Architecture and HDFS

Strong knowledge of working with large data sets and high-capacity big data processing platform

Strong experience in Unix and Shell scripting

Experience with jenkins for Continuous Integration

Experience using Source Code and Version Control Systems like Bitbucket, Git

Good to have experience on Agile Development

Responsibilities : Develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained

Ensures solutions are well designed with maintainability / ease of integration and testing built-in from the outset

Participates and guides team in estimating work necessary to realize a story / requirement through the software delivery lifecycle

Responsible for developing and delivering complex software requirements to accomplish business goals

Ensures that software is developed to meet functional, non-functional, and compliance requirements

Codes solutions, Unit testing and ensure the solution can be integrated successfully into the overall application / system with clear robust, and well-tested interfaces

Required Skills : Hadoop, Hive, HDFS, Spark, Python, Unix

Nice to have skills : AML Domain knowledge

Travel No