Talent.com
Senior Data Engineer - Medallion Architecture
Senior Data Engineer - Medallion ArchitectureTalent Socio • Mumbai
Senior Data Engineer - Medallion Architecture

Senior Data Engineer - Medallion Architecture

Talent Socio • Mumbai
30+ days ago
Job description

Description :

We are building a next-generation Customer Data Platform (CDP) powered by the Databricks Lakehouse architecture and Lakehouse Engine framework.

We're looking for a skilled Data Engineer with 4-9 years of experience to help us build metadata-driven pipelines, enable real-time data processing, and support marketing campaign orchestration capabilities at scale.

The core responsibilities for the job include the following :

Lakehouse Engine Implementation :

  • Configure and extend the Lakehouse Engine framework for batch and streaming pipelines.
  • Implement the medallion architecture (Bronze -> Silver -> Gold) using Delta Lake.
  • Develop metadata-driven ingestion patterns from various customer data sources.
  • Build reusable transformers for PII handling, data standardization, and data quality enforcement.

Real-Time CDP Enablement :

  • Build Spark Structured Streaming pipelines for customer behavior and event tracking.
  • Set up Debezium + Kafka for Change Data Capture (CDC) from CRM systems.
  • Design and develop identity resolution logic across both streaming and batch datasets.
  • DataOps and Governance :

  • Use Unity Catalog for managing RBAC, data lineage, and auditability.
  • Integrate Great Expectations or similar tools for continuous data quality monitoring.
  • Set up CI / CD pipelines for deploying Databricks notebooks, jobs, and DLT pipelines.
  • Requirements :

  • 4-9 years of hands-on experience in data engineering.
  • Expertise in Databricks Lakehouse platform, Delta Lake, and Unity Catalog.
  • Advanced PySpark skills, including Structured Streaming.
  • Experience implementing Kafka + Debezium CDC pipelines.
  • Strong in SQL transformations, data modeling, and analytical querying.
  • Familiarity with metadata-driven architecture and parameterized pipelines.
  • Understanding of data governance : PII masking, access controls, and lineage tracking.
  • Proficiency in working with AWS, MongoDB, and PostgreSQL.
  • Nice to Have :

  • Experience working on Customer 360 or Martech CDP platforms.
  • Familiarity with Martech tools like Segment, Braze, or other CDPs.
  • Exposure to ML pipelines for segmentation, scoring, or personalization.
  • Knowledge of CI / CD for data workflows using GitHub Actions, Terraform, or Databricks CLI.
  • (ref : hirist.tech)

    Create a job alert for this search

    Senior Data Engineer • Mumbai

    Related jobs
    Sr. Data Engineer

    Sr. Data Engineer

    HNR Tech • Mumbai, IN
    We are seeking an experienced Senior Data Engineer to design, build, and optimize our modern data ecosystem.The ideal candidate will have deep expertise in data architecture, pipeline orchestration...Show more
    Last updated: 15 days ago • Promoted
    Senior data engineer

    Senior data engineer

    Fornax • Mumbai, Maharashtra, India
    If solving business challenges drives you.Fornax is a team of cross-functional individuals who solve critial business challenges using core concept of analytics, critical thinking.We are seeking a ...Show more
    Last updated: less than 1 hour ago • Promoted • New!
    Senior Oracle Data Engineer

    Senior Oracle Data Engineer

    Strattek Consulting • Mumbai, IN
    This role centers on architecting robust data pipelines across on-prem and cloud Oracle ecosystems, automating ingestion from varied sources including APIs. Exposure to Oracle FDI (Fusion Data Intel...Show more
    Last updated: 8 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Mindlance • Mumbai, IN
    We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show more
    Last updated: 6 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Confidential • Pune, Mumbai
    If you are an extraordinary developer and who loves to push the boundaries to solve.Data Engineer -Azure, you will work in the Technology team that helps deliver our Data.Engineering offerings at l...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer (Middle / Senior) ID42376

    Data Engineer (Middle / Senior) ID42376

    Confidential • Mumbai, India
    Fortune 500 brands and trailblazing startups across 17+ industries.We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Bes...Show more
    Last updated: 26 days ago • Promoted
    Data Engineer - Palantir Foundry

    Data Engineer - Palantir Foundry

    NP Group • Mumbai, IN
    Data Engineer - Palantir Foundry, Workshop, Pyspark & Typescript.Long Term (initially 6 months) full time contract.We have an immediate requirement for an experienced Data Engineer to join the glob...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Donyati • Mumbai, India
    Job Title : Senior Data Engineer.Location : Remote / Willing to Travel.We are seeking a highly skilled Senior Data Engineer to join our team in building a modern data platform on AWS.You will play a...Show more
    Last updated: 3 hours ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    Quantiphi • Mumbai, Maharashtra, India
    Quantiphi is an award-winning Data Science and Machine Learning Software and Services Company focused on helping organizations translate the big promise of Machine Learning technologies into quanti...Show more
    Last updated: 22 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Fornax • Mumbai, India
    If solving business challenges drives you.Fornax is a team of cross-functional individuals who solve critial business challenges using core concept of analytics, critical thinking.We are seeking a ...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Architect- Snowflake

    Senior Data Architect- Snowflake

    USEReady • Mumbai, IN
    USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage. USEReady partners with cloud...Show more
    Last updated: 6 days ago • Promoted
    Data Engineer

    Data Engineer

    Insight Global • Mumbai, IN
    GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts. Proficiency in ETL / ELT pipeline development...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Straive • Mumbai, Maharashtra, India
    Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced.Analytics datasets based on your stakeholders’ requirements. Design data pipelines to curate sourced da...Show more
    Last updated: 30+ days ago • Promoted
    Senior data engineer

    Senior data engineer

    Straive • Mumbai, Maharashtra, India
    Job Description : Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced.Analytics datasets based on your stakeholders’ requirements.Design data pipelines to ...Show more
    Last updated: 30+ days ago • Promoted
    Databricks Data Engineer Lead – Sustainability Project

    Databricks Data Engineer Lead – Sustainability Project

    Blue Cloud Softech Solutions Limited • Mumbai, IN
    BCSS is seeking a Databricks Data Engineer to support its enterprise-wide Sustainability initiative.The engineer will be responsible for building data pipelines and models to support product-level ...Show more
    Last updated: 8 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    VRIZE • Mumbai, IN
    Design, architect, administer, and manage large database systems, particularly on Microsoft platforms such as SQL Server and Azure Database. Develop and optimize complex SQL queries for data extract...Show more
    Last updated: 1 day ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    RapidBrains • Mumbai, IN
    Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show more
    Last updated: 13 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    SAIVA AI • Mumbai, IN
    We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
    Last updated: 30+ days ago • Promoted