Talent.com
This job offer is not available in your country.
Data Engineering Lead / Architect - Python / Apache Airflow

Data Engineering Lead / Architect - Python / Apache Airflow

SUPERSOURCING TECHNOLOGIES PRIVATE LIMITEDIndia
21 days ago
Job description

Job Title : Lead / Architect Data Engineer

Location : Pune / Bangalore / Hyderabad (Hybrid or Remote flexibility)

Experience : 8+ years (with at least 3+ years in a lead / architect capacity)

Employment Type : Full-time

About the Role :

We are seeking a Lead / Architect Data Engineer to design, architect, and lead implementation of modern data platforms and pipelines. This role requires hands-on expertise in Azure, DBT, Airflow, Python, SQL, and Snowflake, with the ability to provide architectural guidance, mentor team members, and drive best practices in data engineering.

Key Responsibilities :

Solution Architecture & Design :

  • Define and implement scalable data architectures on Azure cloud.
  • Design ELT / ETL pipelines leveraging Airflow and DBT for workflow orchestration and data transformations.
  • Optimize Snowflake usage (warehouses, performance tuning, cost & Strategy :
  • Lead a team of data engineers, ensuring high-quality delivery and adherence to best practices.
  • Partner with product, analytics, and business stakeholders to translate requirements into data engineering solutions.
  • Establish coding, CI / CD, and DevOps standards for data pipelines.
  • Hands-on Development & Optimization
  • Develop robust, reusable, and efficient data pipelines in Python and SQL.
  • Implement monitoring, logging, and alerting for pipeline health.
  • Ensure security, compliance, and governance in data solutions.

Innovation & Scaling :

  • Explore and introduce new tools / technologies to improve data workflows.
  • Contribute to the roadmap for data infrastructure evolution.
  • Must-Have Skills :

  • 8+ years of experience in Data Engineering, with 3+ years in a lead or architect role.
  • Strong expertise in SQL, DBT, Airflow, and Python.
  • Proven experience in Snowflake (data modeling, performance tuning, best practices).
  • Hands-on experience with Azure Data Services (ADLS, ADF, Synapse, Event Hub, etc.).
  • Experience designing scalable and fault-tolerant data architectures.
  • Strong understanding of data warehousing, ELT / ETL patterns, and distributed systems.
  • Good to Have :

  • Experience with Azure Kubernetes Service (AKS) and containerized data solutions.
  • Exposure to real-time / streaming data frameworks (Kafka, Event Hub, etc.).
  • Familiarity with data governance, cataloging, and security best practices.
  • Experience in mentoring teams and conducting architecture reviews.
  • Soft Skills :

  • Excellent problem-solving and critical-thinking abilities.
  • Strong communication and stakeholder management skills.
  • Ability to work independently and in a collaborative team environment.
  • Proven track record of leading end-to-end data engineering initiatives.
  • Why Join Us?

  • Opportunity to lead modern cloud-first data engineering projects.
  • Work with a talented and collaborative team driving data innovation.
  • Freedom to experiment with the latest tools in data engineering and cloud.
  • Competitive compensation and growth-oriented career path.
  • (ref : hirist.tech)

    Create a job alert for this search

    Data Engineering • India

    Related jobs
    • Promoted
    Data Architect

    Data Architect

    InfoBeansNagpur, IN
    Data Architect – Snowflake & Matillion.Snowflake, Matillion, Data Modeling, SQL, Python / Bash, Cloud Platforms (AWS / Azure). Immediate to 15 days preferred.The ideal candidate is proactive, self-direc...Show moreLast updated: 16 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    DeltacubesNagpur, IN
    Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 15 days ago
    • Promoted
    Lead / Architect Data and AI Platform

    Lead / Architect Data and AI Platform

    AkkodisIndia, India
    Deep experience building on-premises or hybrid data systems (e.Spark, Kafka, Trino, Iceberg, Airflow).Strong understanding of LLM architecture, embeddings, vector search, and agentic workflows.Prov...Show moreLast updated: 18 days ago
    • Promoted
    Senior Engineering Manager-Big Data, Generative AI

    Senior Engineering Manager-Big Data, Generative AI

    Extreme NetworksNagpur, IN
    Over 50,000 customers globally trust our end-to-end, cloud-driven networking solutions.They rely on our top-rated services and support to accelerate their digital transformation efforts and deliver...Show moreLast updated: 7 days ago
    • Promoted
    • New!
    Data Architect

    Data Architect

    G10XNagpur, IN
    Senior Data Architect (15+ Years Experience).We are looking for a seasoned Data Architect to lead and shape enterprise-wide data initiatives. This role requires a strategic leader with deep technica...Show moreLast updated: 6 hours ago
    • Promoted
    AI developer / Support

    AI developer / Support

    Dexian IndiaNagpur, IN
    Understanding of the Basics of Machine Learning.Knowledge of Natural Language Processing (NLP) and NLP Algorithms.Familiarity with Vector Databases. Experience with Azure Cloud and Deployments.Profi...Show moreLast updated: 30+ days ago
    • Promoted
    Principal / Senior Data Architect

    Principal / Senior Data Architect

    AaysNagpur, IN
    Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 14 days ago
    • Promoted
    Senior Manager - Data Engineering Lead

    Senior Manager - Data Engineering Lead

    DIAGEO Indianagpur, maharashtra, in
    Senior Manager - Data Engineering Lead.Bachelor’s or master’s degree in computer science, Data Engineering, or related field. Experience in data engineering.Proven experience in cloud platforms (AWS...Show moreLast updated: 26 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data SolutionsNagpur, IN
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 7 days ago
    • Promoted
    Data Engineer

    Data Engineer

    AceolutionNagpur, IN
    We are looking for a freelancer to engage with us for 20-40 hours per week.Kindly find the JD below for your reference.Design, develop, and maintain scalable data pipelines and workflows.Work exten...Show moreLast updated: 7 days ago
    • Promoted
    Senior Full Stack SDE with Data Engineering for Analytics

    Senior Full Stack SDE with Data Engineering for Analytics

    TruckmentumNagpur, IN
    Truckmentum is seeking a Senior Full Stack Software Development Engineer (SDE) with deep data engineering experience to help us build cutting-edge software and data infrastructure for our AI-driven...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Manuh TechnologiesNagpur, IN
    Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Data Engineering Azure databricks

    Data Engineering Azure databricks

    EXLIndia, India
    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders. Work with stakeholders to unders...Show moreLast updated: 6 hours ago
    • Promoted
    • New!
    Data Solutions Architect

    Data Solutions Architect

    BristleconeNagpur, IN
    We are seeking an accomplished Data Solutions Architect with deep experience in designing, implementing, and optimizing modern cloud data platforms. The ideal candidate will expertly leverage Snowfl...Show moreLast updated: 6 hours ago
    • Promoted
    Backend Python / AI Engineer

    Backend Python / AI Engineer

    JuiceLabs AINagpur, IN
    Where creative engineering meets applied AI.At JuiceLabs, we’re building vertical AI-native tools that unlock fresh insights and creative superpowers for our clients in advertising, ecommerce, and ...Show moreLast updated: 7 days ago
    • Promoted
    GIS Engineer

    GIS Engineer

    TAO Digital SolutionsNagpur, IN
    Build ingestion & validation pipelines (Landing → Bronze → Silver) using Spark / Glue / EMR.Standardize schemas, handle drift, quarantine & auditability. Convert to Geo Parquet & load Gold data into Aur...Show moreLast updated: 7 days ago
    • Promoted
    ETL LEAD with IBM DataStage experienced

    ETL LEAD with IBM DataStage experienced

    IntraEdgeNagpur, IN
    ETL Developer – DataStage, AWS, Snowflake.We are looking for a talented and motivated ETL Developer / Senior Developer.You will work on building scalable and efficient data pipelines using.IBM Data...Show moreLast updated: 30+ days ago
    • Promoted
    Jr. Technical Support Engineering (PKI, JAVA & Python)

    Jr. Technical Support Engineering (PKI, JAVA & Python)

    CloudLabs IncNagpur, IN
    CloudLabs Inc was founded in 2014 with the mission to provide exceptional IT & Business consulting services at a competitive price, to help clients realize the best value from their investments.Wit...Show moreLast updated: 17 days ago
    • Promoted
    Engineering Manager - AI Platform

    Engineering Manager - AI Platform

    CoinbaseNagpur, IN
    Ready to be pushed beyond what you think you’re capable of?.At Coinbase, our mission is to increase economic freedom in the world. It’s a massive, ambitious opportunity that demands the best of us, ...Show moreLast updated: 30+ days ago
    • Promoted
    Sr. Fullstack engineer

    Sr. Fullstack engineer

    Relevance LabNagpur, IN
    Design, develop, and maintain our AI Agents platform using Python, FastAPI, React, and TypeScript.Collaborate with cross-functional teams, including product managers, designers, and backend enginee...Show moreLast updated: 30+ days ago