Key Responsibilities :
Data Pipeline & ETL Architecture :
- Design and implement robust pipelines to source pricing data automatically from hyperscaler APIs (AWS, Azure, GCP, Oracle, etc.) and regional providers.
- Develop validation and cleansing processes to ensure accuracy and consistency.
- Automate ingestion into Typesense (or alternative search engines).
Search & Marketplace Intelligence :
Architect and optimize intelligent search (fuzzy, semantic, NLP-driven).Design ranking algorithms that factor in price, performance, region, and compliance.Build features like deal scoring, recommendation engines, and AI-driven cost optimization.AI / ML Services :
Explore and deploy NLP / LLM-powered search for contextual queries (best GPU for AI training in Europe under $2 / hr).Experiment with anomaly detection for pricing errors and trend analysis.Work with the product team to build predictive models for cost-saving & Performance :Architect data systems that can scale globally with millions of records and fast response times.Implement monitoring, error-handling, and recovery strategies for critical data pipelines.Required Skills & Experience :
Data Engineering & ETL :
7+ years in data architecture, pipelines, or ETL frameworks (Airflow, dbt, Spark, Kafka, etc.).Proven experience building pipelines that integrate multiple external APIs.Strong skills in data validation, mapping, and schema design.Search Engines & Marketplace Tech :
Hands-on experience with Typesense, Elasticsearch, or Solr.Background in marketplace or price aggregation systems (e.g., e-commerce, travel, fintech, or cloud pricing).AI & NLP :
Experience with NLP / semantic search (e.g., vector embeddings, Pinecone, Weaviate, FAISS, OpenAI / LLM APIs).Knowledge of relevance ranking, recommendation systems, and & Cloud :Strong coding skills in Python / Go / Node.js.Deep understanding of cloud platforms (AWS / Azure / GCP) and their pricing models.Experience with containerized & serverless environments (Docker, Kubernetes, Lambda).Bonus :
Prior experience in FinOps, cloud cost management, or SaaS marketplaces.Exposure to graph databases for relationship-driven recommendations.What We Offer :
Ground-floor opportunity in a fast-growing startup.Work with a team solving one of the biggest challenges in cloud adoption.Competitive salary + equity.Remote flexibility with global exposure.(ref : hirist.tech)