Talent.com
Senior Data Platform Engineer

Senior Data Platform Engineer

Black Dog Labsvapi, India
8 days ago
Job description

Position : Senior Data Platform Engineer

Location : Remote (collaboration across time zones), India or LATAM preferred

Engagement Type : Contract (12+ months)

Language : Proficient English communication

Experience : 5+ years in Data Engineering / Backend Engineering / DevOps

About the Role

We’re looking for a hands-on Senior Data Platform Engineer to build and operate production-grade data pipelines and services. You’ll work across ingestion, transformation, orchestration, and APIs; automate environments with Terraform; instrument observability; and write tests to keep things reliable. Azure experience is preferred; strong AWS experience is also acceptable. You should communicate clearly (async chat and live sessions), learn fast, and use AI coding tools (e.g., Claude Code) effectively without relying on them exclusively.

Key Responsibilities

  • Data pipelines : Design, build, and operate reliable batch and change-driven pipelines from multiple sources; schedule / orchestrate jobs; handle schema evolution and failures gracefully.
  • Transformations & modeling : Implement clean, tested merge and transformation pipelines and produce models that are easy to consume for products and analytics; tune SQL for performance.
  • APIs & integration : Build or collaborate on APIs for data access and updates; design request / response contracts; handle auth (OAuth2 / OIDC / JWT), idempotency, validation, and auditing.
  • Infrastructure & DevOps : Provision and manage cloud resources with Terraform; automate build / test / deploy with CI / CD; write solid shell scripts for glue / ops tasks.
  • Observability & reliability : Instrument structured logs, metrics, and traces; define alert policies and on-call runbooks; track SLOs and drive root-cause prevention.
  • Security basics : Apply least-privilege access, secrets management, and encryption in transit / at rest; collaborate with the team on compliance requirements.
  • Quality & tests : Write unit, integration, and data-quality tests (including SQL tests) to keep pipelines and services correct and maintainable.
  • Collaboration & docs : Communicate clearly in writing and in person; produce concise docs (data contracts, mappings, runbooks); work closely with product / engineering / analytics.

Required Skills & Qualifications

  • Strong SQL (query tuning, indexing, query plans) and solid RDBMS fundamentals (e.g., Postgres, SQL Server, MySQL).
  • Python for data work and services; comfortable with shell scripting (bash preferred, powershell proficiency if not bash).
  • Terraform and IaC fundamentals; CI / CD experience (version control workflows, automated tests, environment promotion).
  • Cloud : Azure preferred (e.g., storage, compute, identity, monitoring). If not Azure, very strong AWS background.
  • Data integration : Experience with ingestion (batch + change-driven), orchestration, schema / versioning, and resilient retries / replays.
  • Workflow orchestration & scheduling : Prefect preferred (flows / deployments, retries / backoff, scheduling, observability); Airflow, Dagster, or similar also acceptable.
  • APIs & auth : Practical experience with REST patterns, pagination, validation, rate limiting, and OAuth2 / OIDC / JWT.
  • Observability : Familiar with logs / metrics / traces and setting actionable alerts and dashboards.
  • Testing mindset : Habit of writing tests for code and SQL; comfortable with fixtures / test data and CI.
  • Communication : Clear, concise writing and verbal comms; comfortable in async chat and live sessions.
  • AI tooling : Ready to use Claude Code (and similar assistants) from day one to accelerate work—while exercising judgment and writing your own code / tests.
  • Preferred Qualifications (Nice to Have)

  • Regulatory / compliance awareness (e.g., data protection or public-sector standards) and how it impacts design / operations
  • Analytics experience (dim / fact modeling, BI consumption patterns)
  • Full-stack exposure (can prototype simple UI / ops views when needed)
  • Data platform architecture exposure (storage / layout choices, catalog / lineage, governance concepts)
  • Familiarity with event streaming / message queues, job schedulers / orchestrators, and columnar formats (e.g., Parquet)
  • Experience with cloud monitoring stacks (Azure Monitor / Application Insights, CloudWatch, Datadog, New Relic, etc.)
  • Engagement Details

  • Duration : 12+ months (extension / full-time possible)
  • Schedule : Flexible, with overlap for key meetings across time zones
  • Compensation : Competitive, based on experience
  • Mode : Remote; available for virtual collaboration and on-call windows as agreed
  • How to Apply

    Please share :

  • Resume / CV highlighting data engineering, DevOps / IaC, and / or API / auth work
  • Brief note (one paragraph) about a recent pipeline / service : requirements, your role, tech, validation / tests, deployment / ops, outcome
  • Links to code samples or design docs (if available and shareable)
  • Apply via LinkedIn or by sending an email to

    Create a job alert for this search

    Senior Data Engineer • vapi, India

    Related jobs
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Saransh Incvapi, India
    Data Engineer (Contract-to-Hire | Remote).Full-Time / Fixed-Term Contract (12 months, extendable based on performance).We are seeking a highly skilled. The ideal candidate will be hands-on, detail-o...Show moreLast updated: 8 days ago
    • Promoted
    Senior Data Integration Engineer

    Senior Data Integration Engineer

    Indsafrivapi, India
    Senior Data Integration Engineer.We are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer intell...Show moreLast updated: 8 days ago
    • Promoted
    Staff / Senior Data Engineer

    Staff / Senior Data Engineer

    HG Insightsvapi, India
    At HG Insights we provides AI-powered revenue growth intelligence and technology intelligence data to help B2B companies refine their go-to-market (GTM) strategies by analyzing market size, identif...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Veraxionvapi, India
    Python, Spark, DBT, and AWS-native services.Agile environment to deliver scalable, secure, and high-performance data solutions. Python, DBT, and AWS services (Data Ops Live).Deliver end-to-end data ...Show moreLast updated: 4 days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Tata Consultancy Servicesvapi, gujarat, in
    Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 2 days ago
    • Promoted
    GenAI Lead Engineer (Investment Data Platforms)

    GenAI Lead Engineer (Investment Data Platforms)

    Vichara Technologiesvapi, India
    We are seeking a highly skilled.This role will also involve leading a cross-functional team of.Machine Learning Engineers and UI Developers. Develop custom frameworks using.Optimize LLM usage for in...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    thinkbridgevapi, India
    We are a global digital product development firm that helps growth-stage companies gain the technology sophistication and maturity of leading modern digital businesses. We differentiate ourselves by...Show moreLast updated: 6 days ago
    • Promoted
    Senior Software Engineer

    Senior Software Engineer

    Programmers.iovapi, gujarat, in
    We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team.The ideal candidate will have deep expertise in Microsoft Azure data services, cloud-based data engineeri...Show moreLast updated: 15 days ago
    • Promoted
    Senior Backend Engineer

    Senior Backend Engineer

    Blossom Socialvapi, gujarat, in
    Blossom Social is building the future of social investing.Our platform empowers retail investors by combining the best of social networks with investing tools—enabling users to connect, learn, and ...Show moreLast updated: 4 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Straivevapi, India
    The ideal candidate is a strong software engineer with hands-on experience in Spark (3.You'll be responsible for designing and implementing ETL / ELT solutions, collaborating with teams to deliver da...Show moreLast updated: 8 days ago
    • Promoted
    Senior AWS Data Engineer

    Senior AWS Data Engineer

    CYAN360vapi, India
    Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •. Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 8 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    HealthyMe AIvapi, India
    We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show moreLast updated: 8 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Nityo Infotechvapi, India
    Profile name- Senior Data Infrastructure Engineer - Apache Spark & Kubernetes.Minimum 5 years of experience in Data Engineering or Infrastructure roles. Proven hands-on experience with Apache Spark ...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Sikich Indiavapi, gujarat, in
    Sikich India is seeking an experienced Data Engineer to join our Data & AI practice.You will design, build, and optimize end-to-end data solutions using Microsoft’s data platforms, including Micros...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    USEReadyvapi, gujarat, in
    Designation : Senior Data Engineer.As a part of the Data Warehouse Team, implement technology improvements to Enterprise Data Warehouse environment. In this role, you will be a key contributor to the...Show moreLast updated: 25 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Response Informaticsvapi, gujarat, in
    AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture. CICD - Terraform or CloudFo...Show moreLast updated: 25 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Incept Labsvapi, India
    Incept Labs is an AI research lab based in San Francisco, California.Our small team of scientists, engineers, and builders is passionate about developing domain-specific, next-generation AI solutio...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Talogyvapi, India
    This opportunity is ideal for a determined and proactive individual who has a wide range of skills in a variety of database administration, reporting and dashboarding disciplines.This role requires...Show moreLast updated: 8 days ago