Talent.com
Data Engineer – Financial Infrastructure & Analytics

Data Engineer – Financial Infrastructure & Analytics

MerilBangalore Urban, Karnataka, India
15 days ago
Job description

About the Role

As a Quantitative Data Engineer , you will be the backbone of the data ecosystem powering our quantitative research, trading, and AI-driven strategies . You will design, build, and maintain the high-performance data infrastructure that enables low-latency, high-fidelity access to market, fundamental, and alternative data across multiple asset classes.

This role bridges quant engineering, data systems, and research enablement , ensuring that our researchers and traders have fast, reliable, and well-documented datasets for analysis and live trading. You’ll be part of a cross-functional team working at the intersection of finance, machine learning, and distributed systems .

Responsibilities

  • Architect and maintain scalable ETL pipelines for ingesting and transforming terabytes of structured, semi-structured, and unstructured market and alternative data.
  • Design time-series optimized data stores and streaming frameworks to support low-latency data access for both backtesting and live trading.
  • Develop ingestion frameworks integrating vendor feeds (Bloomberg, Refinitiv, Polygon, Quandl, etc.), exchange data, and internal execution systems.
  • Collaborate with quantitative researchers and ML teams to ensure data accuracy, feature availability, and schema evolution aligned with modeling needs.
  • Implement data quality checks, validation pipelines, and version control mechanisms for all datasets.
  • Monitor and optimize distributed compute environments (Spark, Flink, Ray, or Dask) for performance and cost efficiency.
  • Automate workflows using orchestration tools (Airflow, Prefect, Dagster) for reliability and reproducibility.
  • Establish best practices for metadata management, lineage tracking, and documentation.
  • Contribute to internal libraries and SDKs for seamless data access by trading and research applications.

In Trading Firms, Data Engineers Typically :

  • Build real-time data streaming systems to capture market ticks, order books, and execution signals.
  • Manage versioned historical data lakes for backtesting and model training.
  • Handle multi-venue data normalization (different exchanges and instruments).
  • Integrate alternative datasets (satellite imagery, news sentiment, ESG, supply-chain data).
  • Work closely with quant researchers to convert raw data into research-ready features .
  • Optimize pipelines for ultra-low latency where milliseconds can impact P&L.
  • Implement data observability frameworks to ensure uptime and quality.
  • Collaborate with DevOps and infra engineers to scale storage, caching, and compute.
  • Tech Stack

  • Languages : Python, SQL, Scala, Go, Rust (optional for HFT pipelines)
  • Data Processing : Apache Spark, Flink, Ray, Dask, Pandas, Polars
  • Workflow Orchestration : Apache Airflow, Prefect, Dagster
  • Databases & Storage : PostgreSQL, ClickHouse, DuckDB, ElasticSearch, Redis
  • Data Lakes : Delta Lake, Iceberg, Hudi, Parquet
  • Streaming : Kafka, Redpanda, Pulsar
  • Cloud & Infra : AWS (S3, EMR, Lambda), GCP, Azure, Kubernetes
  • Version Control & Lineage : DVC, MLflow, Feast, Great Expectations
  • Visualization / Monitoring : Grafana, Prometheus, Superset, DataDog
  • Tools for Finance : kdb+ / q (for tick data), InfluxDB, QuestDB
  • What You Will Gain

  • End-to-end ownership of core data infrastructure in a high-impact, mission-critical domain.
  • Deep exposure to quantitative research workflows , market microstructure , and real-time trading systems .
  • Collaboration with elite quantitative researchers, traders, and ML scientists.
  • Hands-on experience with cutting-edge distributed systems and time-series data technologies .
  • A culture that emphasizes technical excellence, autonomy, and experimentation.
  • Qualifications

  • Bachelor’s or Master’s in Computer Science, Data Engineering, or related field.
  • 2+ years of experience building and maintaining production-grade data pipelines .
  • Proficiency in Python , SQL , and frameworks like Airflow , Spark , or Flink .
  • Familiarity with cloud storage and compute (S3, GCS, EMR, Dataproc) and versioned data lakes (Delta, Iceberg) .
  • Experience with financial datasets , tick-level data , or high-frequency time series is a strong plus.
  • Strong understanding of data modeling, schema design, and performance optimization .
  • Excellent communication skills with an ability to support multidisciplinary teams .
  • Create a job alert for this search

    Infrastructure Engineer • Bangalore Urban, Karnataka, India

    Related jobs
    • Promoted
    Data Engineer(Azure / AWS, Python / Pyspark, SQL)

    Data Engineer(Azure / AWS, Python / Pyspark, SQL)

    Sail Analyticshosur, tamil nadu, in
    Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift). Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
    • Promoted
    AI & Data Analytics Engineer

    AI & Data Analytics Engineer

    AmphenolBangalore Urban district, India, India
    We are seeking a highly motivated.This role will play a critical part in modernizing and streamlining our testing processes through automation,. The Analyst will collaborate with auditors and stakeh...Show moreLast updated: 12 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Response Informaticsbangalore, karnataka, in
    AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture. CICD - Terraform or CloudFo...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Recrohosur, tamil nadu, in
    Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake. AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer (AWS / Databricks)

    Senior Data Engineer (AWS / Databricks)

    Accoladehosur, tamil nadu, in
    The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Sonata SoftwareBengaluru, Karnataka, India
    We are looking for Data Engineer-.Experience in Snowflake - SnowSQL,Snowpipe,Streams,Tasks,Procedures,TImeTravel etc.Good understanding of Datawarehouse concepts. Cloud Platform - AWS Knowledge.Show moreLast updated: 22 days ago
    • Promoted
    Azure Cloud Data Engineer

    Azure Cloud Data Engineer

    Intelebeehosur, tamil nadu, in
    Data Factory, Synapse, and Databricks.Python, PySpark, and Advanced SQL.Lakehouse / Medallion architecture.Customer Interaction & Technical Documentation. Interact with clients and business stakehol...Show moreLast updated: 14 days ago
    • Promoted
    Data Engineer

    Data Engineer

    thinkbridgehosur, tamil nadu, in
    We are a global digital product development firm that helps growth-stage companies gain the technology sophistication and maturity of leading modern digital businesses. We differentiate ourselves by...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Networth CorpBengaluru, Karnataka, India
    Responsibilities : Working along with architects for planning, designing, integrating, and implementing complex solutions in Azure Cloud computing services End to end development lifecycle of comple...Show moreLast updated: 30+ days ago
    • Promoted
    Big Data Engineer

    Big Data Engineer

    Alef Educationhosur, tamil nadu, in
    Alef Education began with a bold idea : that every learner deserves a personalised and meaningful education experience.What started in 2016 as a small pilot programme in Abu Dhabi has evolved into o...Show moreLast updated: 20 days ago
    • Promoted
    • New!
    Senior Data Engineer

    Senior Data Engineer

    SGS & Cohosur, tamil nadu, in
    Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in. The ideal candidate will play a ke...Show moreLast updated: 19 hours ago
    • Promoted
    • New!
    Lead Data Engineer - Databricks

    Lead Data Engineer - Databricks

    Tredence Inc.Bengaluru, Karnataka, India
    Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI. Headquartered in San Jose, Cali...Show moreLast updated: 21 hours ago
    • Promoted
    Data Engineer

    Data Engineer

    MastekBengaluru, Karnataka, India
    Data Engineer – NiFi / Cloudera / Iceberg / Snowflake / Databricks.We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data fr...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Lead Data Engineer

    Lead Data Engineer

    Cimpresshosur, tamil nadu, in
    Our Team : Enterprise Business Solutions.Vista’s Enterprise Business Solutions (EBS) domain is working to make our company one of the most data-driven organizations to support Finance, Supply Chain,...Show moreLast updated: 20 hours ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Saransh Inchosur, tamil nadu, in
    Data Engineer (Contract-to-Hire | Remote).Full-Time / Fixed-Term Contract (12 months, extendable based on performance).We are seeking a highly skilled. The ideal candidate will be hands-on, detail-o...Show moreLast updated: 22 days ago
    • Promoted
    • New!
    Data Engineer

    Data Engineer

    Dexian Indiahosur, tamil nadu, in
    Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure for efficient ETL processes from vari...Show moreLast updated: 20 hours ago
    • Promoted
    Data Engineer

    Data Engineer

    LTIMindtreehosur, tamil nadu, in
    We are seeking a skilled Data Engineer with hands-on experience in.The ideal candidate will be responsible for developing scalable data pipelines, transforming and ingesting large-scale data, and e...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    SynechronBengaluru, Karnataka, India
    We have immediate opportunity for AWS data engineer.At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innova...Show moreLast updated: 30+ days ago