Founding Data Engineer

InSiteVerseKannur, IN

30+ days ago

Job description

Equity Only (No Monetary Compensation) | Pre-Seed Stage Startup

About Us

We are building a next-generation AI-driven trading platform that ingests massive streams of financial, alternative, and sentiment data and transforms them into actionable insights. Our mission is to combine robust data engineering pipelines with Generative AI and advanced ML models to democratize access to sophisticated trading strategies.

As part of our founding technical team , you will play a crucial role in ensuring our platform's data backbone is reliable, scalable, and production-ready.

The Role

We are looking for a Founding Data Pipeline Engineer passionate about designing and building scalable data pipelines in the cloud. You will work closely with the Founding Gen AI Engineer to ensure clean, validated, and well-structured data is available for model training, inference, and strategy generation.

You will be hands-on with custom Python development , running data ingestion and transformation jobs in Azure Functions and similar serverless / cloud-native environments. You will also architect and optimize data storage and analytics pipelines, leveraging Azure OLAP services such as Kusto (KQL) and Data Lake, as well as Databricks.

What You’ll Do

Data Pipeline Development :

Design and implement scalable ingestion pipelines to handle high-volume, real-time, and batch data feeds.
Write custom Python code to run within Azure Function Apps for event-driven and scheduled ingestion tasks.
Integrate APIs, event streams, and unstructured data into unified pipelines.

Data Transformation & Storage : Implement preprocessing, validation, and enrichment logic to prepare datasets for AI / ML models.

Leverage Azure Data Lake, Synapse Analytics, and Kusto (KQL) for efficient querying, storage, and OLAP-style analytics.

Ensure data quality, consistency, and schema evolution in a fast-changing environment.

Collaboration with AI / ML : Partner with the Founding Gen AI Engineer to ensure AI-ready datasets (structured, time-series, textual, and alternative data).

Enable backtesting and live trading workflows by delivering low-latency, reliable data pipelines.

Operations & Observability :

Establish monitoring, alerting, and logging for all pipelines to ensure reliability.

Design pipelines that can adapt to scaling needs, failure recovery, and cost efficiency in the cloud.

Startup Contribution :

Take ownership of the entire data lifecycle , from ingestion to OLAP.

Contribute to technical decisions and long-term architecture.

Operate in a pre-funding, equity-only environment with full ownership of outcomes.

What We’re Looking For

5–8 years of experience in data engineering or pipeline development , with strong proficiency in Python .

Hands-on experience building serverless / cloud-native pipelines (Azure Functions, AWS Lambda, or similar).

Strong expertise with Azure Data stack (Kusto / KQL, Synapse, Data Lake, Event Hubs, Data Factory).

Proven ability to design and manage data ingestion, transformation, and storage pipelines at scale.

Solid understanding of batch vs. streaming data processing and how to optimize for low-latency use cases.

Bonus : familiarity with financial data structures or trading systems (but not required).

Entrepreneurial, self-starter mindset with the ability to deliver hands-on solutions in an early-stage startup .

Bring your own device / laptop - but need to sign up for strict Confidentiality / NDA / Non-Compete Agreements

Why Join Us

Be a founding technical member , building the data backbone of a disruptive AI trading platform.

Collaborate closely with the Founding Gen AI Engineer to power cutting-edge ML models with high-quality data.

Work with modern, cloud-native, and serverless data pipelines in Azure.

Full equity-based compensation at pre-angel stage – a high-risk, high-reward journey.

Shape the core infrastructure of a company at the ground floor.

Create a job alert for this search

Data Engineer • Kannur, IN