Description
We are seeking an experienced PySpark Developer to join our dynamic team in India. The ideal candidate will have a strong background in data engineering and a passion for building scalable data solutions.
Responsibilities :
- Develop and maintain data processing pipelines using PySpark, SQL, and Hadoop.
- Collaborate with data scientists and analysts to optimize data workflows.
- Implement data transformation and aggregation processes.
- Ensure secure data access and compliance with data governance policies.
- Perform Spark job tuning and performance optimization.
- Write unit tests and documentation for Spark transformations.
- Work on continuous improvement of data processing frameworks.'
Skills Required :
Proficiency in PySpark, SQL, and Hadoop.Experience with big data technologies and frameworks.Strong problem-solving and analytical skills.Ability to work in a collaborative team environment.Excellent communication skills.Skills Required
Pyspark, Spark SQL, Python Scripting, Machine Learning, Data Modeling, Cloud Services, Data Warehousing, Scala Programming