Pyspark Developer

Tata Consultancy ServicesKolkata, West Bengal, India

14 days ago

Job description

TCS is Hiring Pyspark Developer For Chennai, Hyderabad, Kolkata location

Experience : 6 - 10 years

Mode of interview : Virtual

Location : Chennai, Hyderabad, Kolkata

JOB DESCRIPTION : Must have :

Minimum 3 Years of experience in design, development and deployment of big data applications and ETL jobs using PySpark APIs / sparkSQL.
Experience in design, build and deployment of Python based applications.
Experience in writing complex SQL queries / procedures using Relational databases like SQL Server or Oracle
Experience in version control system like Git and CI / CD pipeline is a must.
Experience in Delta lake APIs is a plus
Experience in Docker and Kubernetes is a plus
Knowledge of AWS services like S3, Athena, Glue, Lambda, Redshift or Cloud platform is a plus

Responsibilities :

Designing, implementing, and maintaining jobs written in PySpark for extracting raw data, applying transformation, and writing into file / tables on Apache Spark cluster.

Writing complex queries as per the business rules and data extraction logic

Query tuning and Performance optimization of various components as and when required.

Work with development leads, System Architects and other teams (as needed) to manage dependencies, risks and issues.

Contributing in all phases of the development lifecycle

Support testers and defect resolution

Update status / risks / issues / impediments in daily scrum calls to the customer and track it to closure.

Writing testable, scalable and efficient code

Working on the necessary documentations as needed

Create a job alert for this search

Pyspark Developer • Kolkata, West Bengal, India