Talent.com
Senior Data Engineer

Senior Data Engineer

American Inferencedehradun, uttarakhand, in
9 hours ago
Job description

About the Company

We are an AI and Data Consulting Startup transforming how businesses leverage technology through four core service lines :

  • Consulting Services : AI Strategy, Automation, and Digital Transformation for enterprises.
  • SaaS Platform Development : Building a business application suite similar to Odoo and Zoho that is AI-native and user friendly.
  • Data Lakehouse Solutions : Unified data pipelines for aggregation, cleaning, governance, and advanced analytics.
  • Government Contracting : Developing secure, compliant AI solutions for the public sector.

Our tech stack includes Python, TypeScript, React, Next.js, Go, Rust, Azure, Kubernetes, Spark, MLflow, Postgres, graph databases, and vector stores.

We're a small, fast-moving team delivering enterprise-grade solutions with startup agility.

Role Overview

You will design, build, and optimize data lakehouse solutions, enabling analytics, AI, and MLOps across multiple clients.

Key Responsibilities

  • Design lakehouse architectures (Bronze / Silver / Gold) using Azure Databricks, Delta Lake, or Iceberg
  • Build scalable PySpark pipelines for batch and streaming data
  • Implement data quality, validation, and lineage tracking
  • Integrate pipelines with MLflow for experiment tracking
  • Optimize Spark performance and cloud cost
  • Ensure data governance, PII protection, and security compliance
  • Mentor developers and document patterns and runbooks

    • Required Skills

  • Apache Spark (PySpark) and SQL (complex joins, CTEs, optimization)
  • Delta Lake / Iceberg and Azure Data Stack (ADLS, Synapse, Databricks)
  • MLflow integration for MLOps pipelines
  • Kafka / Event Hubs for streaming data
  • Strong Python and data modeling (dimensional, Data Vault)
  • 5+ years hands-on in data engineering with production-grade systems

    • Nice to Have

  • Vector databases, dbt, Airflow, Terraform, FinOps exposure, and Great Expectations data validation
  • Experience with data governance technologies (such as MS Purview or similar)
  • Certifications : Azure Data Engineer / Databricks DE

    • Education

  • UG : B.Tech / B.E. – Computer Science / Information Technology
  • PG : M.Tech / M.E. / Any Postgraduate (preferred)

    • Why Join Us

  • Impact : You'll have significant influence over our data architecture and enable AI / ML capabilities across the organization.
  • Growth : Help shape and build our data-driven culture from the ground up.
  • Learning : Work with the latest in AI, code, and data technologies.
  • Culture : Flat hierarchy, collaborative, and outcome-driven team.
    • Create a job alert for this search

    Senior Data Engineer • dehradun, uttarakhand, in