Senior Hadoop / Bigdata Engineer

Candidate should be able to :

Work closely with our QA Team to ensure data integrity and overall system quality

Work closely with Technology Leadership, Product Managers, and Reporting Team for understanding the functional and system requirements

Write Shell / Python scripts for jobs scheduling and data wrangling

Write Scoop Jobs to Import / Export data from Hadoop

Enhance existing Spark and Java applications, and provide support

Generate data reports using HiveQL, Spark SQL or PySpark

Develop greenfield data applications using Spark, Java, Python, JDBS / ODBC, and other Hadoop / BigData technologies

Design and develop Cloudera HDFS-based solutions using Spark (with interfaces Java, Python, and Spark SQL), Hive QL, Flume, Talend, IBM MQ, and Kafka

Candidate should have :

Ability to identify problems, and effectively communicate solutions to peers and management

Strong debugging skills to troubleshoot production issues

Experience in working with real-time data feeds

Understanding of Data architecture, replication, and administration

Familiarity with Linux OS

Exposure to RDBMS : Microsoft SQL Server, Oracle, DB2

Comfortability working in a team environment

AWS Cloud Analytics experience

Must have 5+ years of experience in using Hadoop / BigData technologies like Spark, Spark SQL, Hive, Flume, Parquet, and Avro file formats, Sqoop, etc.

Must have 5+ years of experience in developing applications using Java, Junit, Maven, and its eco-system

Must have 2+ years of experience in developing Shell / Python scripts

BS / BA degree in Computer Science, Information Systems or related field