Talent.com
Data Pipeline SDET

Data Pipeline SDET

InfoBeansPune, Republic Of India, IN
3 days ago
Job description

Job Title : SDET – PySpark / Python Data Engineering (Azure Cloud)

Location : Indore | Pune | Bangalore | Chennai

Experience Required : 5+ years

Role Overview :

Seeking a skilled Software Development Engineer in Test (SDET) with expertise in PySpark and Python for data engineering and data pipeline validation on Azure Cloud. You will play a critical role in building automation frameworks, developing and executing test cases to ensure the quality and reliability of large-scale data solutions.

Key Responsibilities :

  • Develop, maintain, and execute automated test scripts for data pipelines and engineering workflows using Python and PySpark.
  • Validate data transformations, data quality, and pipeline performance on Azure Cloud environments (preferably using Databricks).
  • Collaborate with data engineers and developers to understand data workflows and test scenarios.
  • Build and optimize test frameworks that integrate with CI / CD pipelines.
  • Perform thorough regression, performance, and API testing for data applications.
  • Report and track defects, and work closely with engineering teams for resolution.

Skills & Qualifications :

  • Strong programming skills in Python and PySpark.
  • Experience in automated testing for data pipelines and big data platforms.
  • Working knowledge of Azure Cloud services, including Azure Data Factory and Databricks.
  • Familiarity with CI / CD tools and test automation frameworks.
  • Understanding of data validation, ETL testing, and SQL query optimization.
  • Excellent problem-solving and communication skills.
  • Preferred :

  • Hands-on experience with Databricks and related Azure data technologies.
  • Background in software development and test engineering for data-intensive applications.
  • Create a job alert for this search

    Sdet • Pune, Republic Of India, IN

    Related jobs
    • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    iVoyantRepublic Of India, IN
    One of our clients is looking for an experienced Python Data Engineer to join their team.Strong Python Experience (web services, background jobs. .Data processing and reporting using Python (Pandas,...Show moreLast updated: 16 days ago
    • Promoted
    • New!
    AI Data Pipeline Engineer

    AI Data Pipeline Engineer

    Pro5.aiRepublic Of India, IN
    You’ll play a key role in ensuring reliable data flows, integrations, and preprocessing frameworks that power advanced GenAI systems. This is a high-impact opportunity within an.Design, build, and o...Show moreLast updated: 4 hours ago
    • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    HARPRepublic Of India, IN
    Minimum 8 to 11 years of hands-on experience with Databricks platform and Azure Data Factory.Expertise in building and optimizing ELT / ETL pipelines using Databricks and Azure Data Factory.Strong kn...Show moreLast updated: 3 days ago
    • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    True Tech ProfessionalsRepublic Of India, IN
    Job Type : Full-time / Contractual.We’re looking for an experienced Data Engineer / Developer with strong hands-on expertise in Snowflake, Databricks, Python scripting, and REST API integration.The id...Show moreLast updated: 4 days ago
    • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    USTPune, Republic Of India, IN
    The ideal candidate should demonstrate a deep understanding of big data concepts, programming fundamentals, and the ability to solve complex problems related to scalability, failure handling, and o...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    Eucloid Data SolutionsChennai, Republic Of India, IN
    Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure for our cli...Show moreLast updated: 3 days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    IntraEdgeRepublic Of India, IN
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show moreLast updated: 30+ days ago
    • Promoted
    Backend and Data Pipeline Engineer

    Backend and Data Pipeline Engineer

    JRD SystemsNagpur, IN
    Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer (AWS / Databricks)

    Senior Data Engineer (AWS / Databricks)

    Accoladenagpur, maharashtra, in
    The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
    • Promoted
    Sde 1 - Data Platform

    Sde 1 - Data Platform

    sliceRepublic Of India, IN
    We’ve all felt how slow, confusing, and complicated banking can be.We’re building every product from scratch to be fast, transparent, and feel good, because we believe that the best products transc...Show moreLast updated: 6 days ago
    • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    IntraEdgeChennai, Republic Of India, IN
    Onsite / Hybrid (as applicable).We are looking for a highly skilled and motivated.The ideal candidate will have strong expertise in. DBT, Snowflake, CI / CD pipelines.This role requires individuals wi...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Data Engineer – ETL & Pipeline Development (5 to 10 yrs)

    Data Engineer – ETL & Pipeline Development (5 to 10 yrs)

    AIMLEAPIndia, India
    Data Engineer – ETL & Pipeline Development.Remote (Work from Home) / Bangalore / India.Tech / MCA / Computer Science / IT. IT / Data / AI / LegalTech / Enterprise Solutions.ETL workflows — not just ...Show moreLast updated: 22 hours ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    thinkbridgePune, Republic Of India, IN
    We are a global digital product development firm that helps growth-stage companies gain the technology sophistication and maturity of leading modern digital businesses. We differentiate ourselves by...Show moreLast updated: 2 days ago
    • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    HARPRepublic Of India, IN
    The ideal candidate will have strong hands-on experience in.Knowledge of DevOps practices and containerization tools is a plus. Develop and maintain scalable data pipelines for large-scale data proc...Show moreLast updated: 30+ days ago
    • Promoted
    Data Pipeline Engineer

    Data Pipeline Engineer

    LuxoftRepublic Of India, IN
    We are currently hiring for the position of Python Developer at DXC Technology, Mumbai.This role involves working closely with global teams in London and Hong Kong, contributing to high-impact proj...Show moreLast updated: 3 days ago
    • Promoted
    Data Engineering Expert – Financial Data Pipeline Development

    Data Engineering Expert – Financial Data Pipeline Development

    Elongo Global Solutionsnagpur, maharashtra, in
    This role is crucial in enabling seamless integration and real-time flow of financial data, which will later power .If you have hands-on experience in . Azure Functions, Kafka, MS Fabric.You’ll lead...Show moreLast updated: 2 days ago
    • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    DraconXRepublic Of India, IN
    DraconX specializes in creating intelligent, scalable digital solutions that drive growth and innovation for startups and enterprises. As pioneers in AI business automation and SaaS platforms, we ex...Show moreLast updated: 1 day ago
    • Promoted
    SAP Data Migration expert

    SAP Data Migration expert

    Numeric TechnologiesNagpur, IN
    As a Sr Software Engineer (SAP) in Data Migration we expect this developer to participate in project data migration activities to support the successful data conversion from ECC to S / 4HANA (Green f...Show moreLast updated: 30+ days ago