No longer accepting applications
Senior Data Engineer - Cloud Architecture
beBeeDataNagpur, Maharashtra, India2 days ago
Job descriptionStrong programming skills in Python (PySpark, pandas, REST APIs). Proven expertise in Databricks (workflows, notebooks, Delta Lake). Advanced SQL knowledge for data modeling and optimization. Experience with Azure Data Factory, Azure Data Lake Gen2, and Azure Synapse Analytics. Proficiency in Postgres (schema design, indexing, tuning). Exposure to Vector Databases (pgvector, Qdrant, Pinecone, etc.) and Generative AI concepts. Familiarity with CI / CD pipelines and Git-based version control.
About the Role
We are seeking an experienced Data Architect with strong skills in Databricks, Python, SQL, and Postgres. The ideal candidate will have exposure to Vector Databases and Generative AI use cases such as RAG pipelines and embedding-based search.
Key Responsibilities :
- Design, develop, and maintain data pipelines and ETL workflows using Databricks and Azure Data Factory.
- Build efficient and scalable data models and Delta Lake architectures for analytics and AI applications.
- Develop and optimize Python-based data processing scripts (PySpark, pandas, APIs).
- Manage and tune SQL and Postgres databases for schema design, indexing, and query performance.
- Integrate and experiment with Vector Databases (pgvector, Qdrant, Pinecone) for advanced AI search solutions.
- Collaborate with data scientists and architects on Generative AI applications leveraging Azure OpenAI or similar platforms.
- Implement CI / CD pipelines and Git-based workflows for continuous data deployment and version control.
Required Skills :
Create a job alert for this search
Cloud Data Engineer • Nagpur, Maharashtra, India