Data Scientist - Python / PySpark

About the job

We are seeking a skilled Data Scientist with 2 to 5 years of experience, specializing in Machine Learning, PySpark, and Databricks, with a proven track record in long-range demand and sales forecasting. This role is crucial for the development and implementation of an automotive OEMs next-generation Intelligent Forecast Application.

The position will involve building, optimising, and deploying large-scale machine learning models for complex, long-term forecasting challenges using distributed computing frameworks, specifically PySpark on the Databricks platform. The work will directly support strategic decision-making across the automotive value chain, including areas like long-term demand planning, production scheduling, and inventory optimization.

The ideal candidate will have hands-on experience developing and deploying ML models for forecasting, particularly long-range predictions, in a production environment using PySpark and Databricks.

This role requires strong technical skills in machine learning, big data processing, and time series forecasting, combined with the ability to work effectively within a technical team to deliver robust and scalable long-range forecasting solutions.

Roles & Responsibilities :

• Machine Learning Model Development & Implementation for Long-Range Forecasting : Design, develop, and implement scalable and accurate machine learning models specifically for long-range demand and sales forecasting challenges.
• Apply advanced time series analysis techniques and integrate them with machine learning models leveraging PySpark for data processing and model training on large datasets within the Databricks environment.
• Implement probabilistic forecasting methods using PySpark to capture uncertainty in long-range predictions.
• Develop robust solutions for hierarchical and grouped long-range forecasting on distributed data.
• Data Processing and Feature Engineering with PySpark : Build and optimize large-scale data pipelines for ingesting, cleaning, transforming, and engineering features relevant to long-range forecasting from diverse, complex automotive datasets using PySpark on Databricks.
• Deployment and MLOps on Databricks : Develop and implement robust code for model training, inference, and deployment of long-range forecasting models directly within the Databricks platform.
• Apply MLOps principles compatible with Databricks workflows for model versioning, monitoring, retraining, and managing the lifecycle of long-range ML forecasting models in production.
• Collaborate with Data Engineering and IT Operations to ensure seamless deployment and operational efficiency of the forecasting application on Databricks.
• Performance Evaluation & Optimization : Evaluate long-range forecasting model performance using relevant metrics (e.g., MAE, RMSE, MAPE, considering metrics suitable for longer horizons) and optimize models and data processing pipelines for improved accuracy and efficiency within the PySpark / Databricks ecosystem.
• Technical Collaboration : Work effectively as part of a technical team, collaborating with other data scientists, data engineers, and software developers to integrate ML long-range forecasting solutions into the broader forecasting application built on Databricks.
• Communicate technical details and forecasting results effectively within the technical team.

Qualifications :

Technical Skills :

Preferred Qualifications :

Soft Skills :

(ref : hirist.tech)