Job Summary :
We are seeking a highly skilled and motivated professional with strong expertise in Large Language Models (LLMs), API prompt engineering, RAG (Retrieval-Augmented Generation) pipelines, and fine-tuning models. The ideal candidate should also be proficient in Python, PySpark, SQL, and have hands-on experience with at least one major cloud provider (AWS, Azure, or GCP). Familiarity with Snowflake, Databricks, and Informatica is mandatory.
Key Responsibilities :
- Design and build scalable RAG pipelines for intelligent data retrieval and question answering systems.
- Develop and optimize LLM-based applications, including prompt engineering, fine-tuning, and deployment of models.
- Collaborate with data scientists and engineers to integrate LLM APIs and ML models into business workflows.
- Build and manage ETL / ELT pipelines using PySpark, SQL, and Informatica.
- Work with Databricks and Snowflake to manage data pipelines and perform advanced analytics.
- Implement secure, scalable, and cost-effective data solutions on a major cloud platform (AWS, Azure, or GCP).
- Optimize data processing jobs for performance and scalability.
Required Skills and Qualifications :
Strong experience with LLMs (OpenAI, Hugging Face, etc.), including prompt engineering, API integration, fine-tuning, and embedding models.Proven expertise in building RAG pipelines.Advanced programming skills in Python and PySpark.Proficiency in SQL and performance tuning of complex queries.Hands-on experience with one major cloud platform (AWS, Azure, or GCP).Practical knowledge of Databricks, Snowflake, and Informatica.Experience in building scalable data pipelines and working with large datasets.Ability to collaborate in cross-functional teams and communicate technical concepts to non-technical stakeholders.(ref : hirist.tech)