PySpark QA

PySpark QA - CREQ Description

JOB DESCRIPTION

Skill : PySpark QA

Role / Tier : Lead Software Engineer / Tier 2

Experience : 6 – 9 years

Job Description : Primary Skills

BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PYSpark.

build CI / CD pipelines

Spark APIs to cleanse, explore, aggregate, transform, store & analyse

installing, configuring, debugging and troubleshooting Hadoop clusters

Secondary Skills

Cloud-based infrastructure , AWS services EC2, EMR, S3, Lambda, EBS, IAM, Redshift, RDS

deploying and managing ETL pipelines, RDBMS technologies (PostgreSQL, MySQL, Oracle, etc.).

knowledge of data frames, Pandas , data visualization tools & data mining .

knowledge of JIRA, Bitbucket GitHub

JD

Seeking a developer to develop, enhance, in a Pyspark -QA role.

Responsibility

Experience with BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PYSpark.

Hands-on experience in build CI / CD pipelines is required Outstanding coding, debugging and analytical skills, Core problem solving skills

eliminate possible solutions and select an optimal solution.

Experience in installing, configuring, debugging and troubleshooting Hadoop clusters is desired skill.

Primary Location Hyderabad, Andhra Pradesh, India Job Type Experienced Primary Skills Data Modelling Years of Experience 5 Travel No