Lead Cloud Data Engineer - PySpark / SQL

About the Job :

We're hiring an cloud data engineering (preferably Azure) data pipelines and Spark.

• Work with Databricks platform using Spark for big data processing and analytics.
• Write optimized and efficient code using PySpark, Spark SQL and Python.
• Develop and maintain ETL processes using Databricks notebooks and workflows.
• Implement and optimize data pipelines for data transformation and integration.
• Knowledge of one or more SQL variant, preferably PL / SQL and Spark SQL.
• Write complex SQL queries for data retrieval, manipulation, and analysis.
• Debugging code when required and troubleshooting any Python, Pyspark or SQL related queries.
• Good Experience with version control (Git) and ci / cd.
• Excellent problem-solving ability with solid communication and collaboration skills

Key Responsibilities :

Technical Requirements :

(ref : hirist.tech)