Associate Big Data Engineer

Responsibilities

• Contribute towards supporting the build / implementation of data ingestion and curation processes developed using Big data tools such as Spark (Scala / python), Hive, HDFS, Kafka, Pig, Spark, HDFS, Oozie, Sqoop, Flume, Zookeeper, Kerberos, Sentry, Impala, CDP 7.x etc. under the guidance of Big Data Engineers.
• Support the ingestion of huge data volumes from various platforms for Analytics needs and prepare high-performance, reliable, and maintainable ETL code with support / review guidance of senior team members.
• Provide relevant support in monitoring performance and advising any necessary infrastructure changes to senior team members for their review inputs.
• Understand the defined data security principals and policies developed using Ranger and Kerberos.
• Gain broader understanding on how to support application developers and progressively work on efficient big data application development using cutting edge technologies.
• Collaborate with Business systems analyst, technical leads, Project managers and business / operations teams in building data enablement solutions across different LOBs and use cases basis
• Understand & support the creation of reusable frameworks which will optimize the development effort involved.
• Contribute towards supporting the build / implementation of data ingestion and curation processes developed using Big data tools such as Spark (Scala / python), Hive, HDFS, Kafka, Pig, Spark, HDFS, Oozie, Sqoop, Flume, Zookeeper, Kerberos, Sentry, Impala, CDP 7.x etc. under the guidance of Big Data Engineers.
• Support the ingestion of huge data volumes from various platforms for Analytics needs and prepare high-performance, reliable, and maintainable ETL code with support / review guidance of senior team members.
• Provide relevant support in monitoring performance and advising any necessary infrastructure changes to senior team members for their review inputs.
• Understand the defined data security principals and policies developed using Ranger and Kerberos.
• Gain broader understanding on how to support application developers and progressively work on efficient big data application development using cutting edge technologies.
• Collaborate with Business systems analyst, technical leads, Project managers and business / operations teams in building data enablement solutions across different LOBs and use cases basis
• Understand & support the creation of reusable frameworks which will optimize the development effort involved.

Skills Required

ETL code, Ranger Kerberos, LOBs