Data Engineer II

Job Title : Data Engineer II

Job Summary :

We are seeking a highly skilled and motivated Data Engineer II - AI / ML to join our innovative team, with a focus on OpenAI technologies, Azure AI services, and cutting-edge LLM and RAG architectures. The ideal candidate will possess deep technical expertise in large language models, hands-on experience with Azure's AI ecosystem, and a strong background in Python-based machine learning, CI / CD pipelines, and version control workflows.

This role is pivotal in designing, developing, and deploying intelligent AI solutions that drive value through automation, natural language processing (NLP), and advanced retrieval mechanisms.

Key Responsibilities :

• Design and implement AI-powered applications using OpenAI APIs (GPT, Whisper, Codex).
• Build and deploy scalable AI / ML solutions using Azure OpenAI, Azure Storage, Azure Container Apps, Azure Functions, and Azure Cognitive Services.
• Architect and implement LLM-based pipelines, including fine-tuning, prompt engineering, and RAG (Retrieval-Augmented Generation) using vector databases (e.g., Azure AI Search, Pinecone, FAISS).
• Build intelligent document processing systems with embeddings, semantic search, and LLM chaining (LangChain, Semantic Kernel).
• Develop and maintain robust APIs and AI services using Python, FastAPI, or Flask.
• Implement CI / CD pipelines for model deployment and service delivery using Azure DevOps, GitHub Actions, or similar tools.
• Utilize Git and version control systems to manage source code, workflows, and model versioning.
• Collaborate with cross-functional teams to deliver AI solutions in a production environment.
• Monitor, evaluate, and optimize AI systems for performance, accuracy, and cost-efficiency.

Required Skills & Qualifications :

Nice to Have :