PySpark Developer

Job Title : PySpark Developer

Location : Chennai, Hyderabad, Kolkata

Work Mode : Monday - Friday (5 days WFO)

Experience : 5+ Years in Backend Development

Notice Period : Immediate to 15 days

Must-Have Experience : Python, PySpark, Amazon Redshift, PostgreSQL

About the Role :

We are looking for an experienced PySpark Developer with strong data engineering capabilities to design, develop, and optimize scalable data pipelines for large-scale data processing. The ideal candidate must possess in-depth knowledge of PySpark, SQL, and cloud-based data ecosystems, along with strong problem-solving skills and the ability to work with cross-functional teams.

Roles & Responsibilities :

• Design and develop robust, scalable ETL / ELT pipelines using PySpark to process data from various sources such as databases, APIs, logs, and files.
• Transform raw data into analysis-ready datasets for data hubs and analytical data marts.
• Build reusable, parameterized Spark jobs for batch and micro-batch processing.
• Optimize PySpark job performance to handle large and complex datasets efficiently.
• Ensure data quality, consistency, and lineage, and maintain thorough documentation across

all ingestion workflows.

logic aligned with business requirements.

storage.

Must-Have Skills :

structured (JSON, XML), and unstructured data (logs, PDFs).

data modeling concepts.

Good-to-Have Skills :

Atlas).

(ref : hirist.tech)