Data Engineer

Data Engineer – AI-Powered Marketing Personalization Platform

We’re seeking an experienced Data Engineer to help build and scale our next-generation AI-powered marketing personalization platform (V2.0) . You’ll design and implement a robust multi-database infrastructure that enables real-time personalization, vector search, graph analytics, and large-scale data processing.

This is a greenfield opportunity to architect data pipelines from the ground up using vector and graph databases and LLM-based systems . You’ll play a key role in migrating our existing platform while creating a scalable foundation powering AI agents across thousands of marketing campaigns.

Core Responsibilities

Data Architecture & Infrastructure (40%)

• Design and implement multi-database systems (MongoDB, Redis, Milvus, Neo4j, BigQuery)
• Build scalable real-time pipelines and ETL / ELT workflows
• Implement data governance, quality, and high-throughput optimization

Vector & Graph Systems (25%)

ML Data Infrastructure (20%)

Analytics & Monitoring (15%)

Tech Stack

Databases : MongoDB, Redis, Milvus, Neo4j, BigQuery

Processing : Airflow / Prefect, Pandas / Polars, dbt, Spark

ML Pipeline : vLLM, MLflow, Sentence Transformers, PyTorch, TensorFlow

Cloud & Infra : GCP, Docker, Kubernetes, Terraform, GitHub Actions

Languages : Python (3.10+), SQL, Bash

Requirements

Must-Have

1M records / day)

Preferred

Key Projects

Phase 1 – Foundation : Migrate 10M+ vectors, implement MongoDB schemas, Neo4j models, and BigQuery warehouse

Phase 2 – Optimization : Build data quality monitoring, caching (Redis), and

Phase 3 – ML Infrastructure : Create LLM fine-tuning pipelines, GNN feature stores, and A / B testing systems

Why Join Us