Talent.com
Senior Data Engineer

Senior Data Engineer

SPIROPune (division)
22 hours ago
Job description

Position : Senior Data Engineer

Location : Pune

Experience : 5+ years

Job Summary :

We are seeking an experienced Senior Data Engineer with a strong background in PySpark, Spark, and Big Data technologies. The ideal candidate will have over five years of hands-on experience designing, building, and optimizing large-scale data processing systems and pipelines.

Key Responsibilities :

  • Design, develop, and maintain scalable data pipelines and ETL processes using PySpark and Spark.
  • Perform data wrangling, transformation, and integration across diverse data sources.
  • Optimize the performance and reliability of data processing workflows.
  • Implement data quality frameworks to ensure data accuracy and integrity.
  • Develop and maintain data architectures and data models to support business needs.
  • Collaborate with cross-functional teams including data analysts, software engineers, and business stakeholders to understand data requirements.
  • Stay updated with the latest advancements in Big Data tools, frameworks, and best practices.
  • Mentor and guide junior data engineers in technical and professional growth.

Required Skills and Qualifications :

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 5+ years of experience in data engineering roles.
  • Expertise in PySpark and Spark for large-scale data processing.
  • Strong understanding of Big Data ecosystems and technologies (e.g., Hadoop, Hive).
  • Hands-on experience with ETL development, data modeling, and data warehousing.
  • Proficiency in programming languages such as Python, Scala, or Java.
  • Experience with cloud platforms like AWS, Azure, or Google Cloud.
  • Strong problem-solving skills and a detail-oriented mindset.
  • Excellent communication and collaboration skills.
  • Preferred Qualifications :

  • Experience with containerization and orchestration tools like Docker and Kubernetes.
  • Knowledge of SQL and relational database management systems.
  • Familiarity with data orchestration tools like Airflow, Prefect, or similar.
  • Create a job alert for this search

    Senior Data Engineer • Pune (division)

    Related jobs
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Persistent SystemsPune (division)
    We’re looking for a Lead Technical Engineer who can bridge data engineering depth with FinOps insight.You’ll design and implement scalable data solutions across Databricks and Snowflake, drive perf...Show moreLast updated: 3 days ago
    • Promoted
    Sr.Azure Data Engineer

    Sr.Azure Data Engineer

    Persistent SystemsPune (division)
    We are looking for Sr Azure Data engineer who have experience in Azure data bricks, Data factory, to join our development team. Job Type : Full Time Employment.Design and implement complex, scalable ...Show moreLast updated: 1 day ago
    • Promoted
    Azure API Engineer

    Azure API Engineer

    Persistent SystemsPune (division)
    As an API Engineer, you will be responsible for the end-to-end development and optimization of APIs that connect various systems and services. You will collaborate closely with software engineers, a...Show moreLast updated: 23 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    EXLPune (division)
    Develop high quality, secure and scalable data pipelines using spark, Scala / Python / Java on Hadoop or object storage like MinIO. Leverage technologies and solutions to innovate with increasingly larg...Show moreLast updated: 1 day ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Searce IncPune (division)
    Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’.It signifies our way of working : To improve to the finest degree of excellence, ‘solving for better’ every time.Searcians...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Data Engineer-SAC

    Data Engineer-SAC

    Orbia PunePune (division)
    If you have strong background in.Note : This is a full time role, 5 Days on-site role based at Orbia Global Capability Center in Pune. A minimum of 6 years' experience in data warehousing, data model...Show moreLast updated: 22 hours ago
    • Promoted
    AI and Databricks

    AI and Databricks

    Quantum IntegratorsPune (division)
    Role : AI and Databricks Engineer.Strong experience developing industry grade LLM and Gen-AI applications using OpenAI, Anthropic (Claude), or other major providers. Spark, Delta Lake, MLflow, Databr...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer

    Data Engineer

    Trinity Technology Solutions LLCPune (division)
    Location : Pune preferred / remote.Programming Python, PySpark / Spark.Operating Systems Linux (RedHat).Databases Hive, MSSQL, MySQL, PostgreSQL. Big Data Hadoop ecosystem (HDFS, YARN, Sqoop).Version Co...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Senior Quality Assurance Engineer

    Senior Quality Assurance Engineer

    SourcebaePune (division)
    Job Opportunity : Senior Quality Engineer - Data Pipelines (PySpark / Databricks).Senior Quality Engineer (SDET).Data Validation and Pipeline Testing. This is a heavy-weight testing role that requires ...Show moreLast updated: 22 hours ago
    • Promoted
    Backend Python Engineer - 17852

    Backend Python Engineer - 17852

    TuringPune (division)
    We’re looking for experienced Python engineers to collaborate with one of the world’s top Large Language Model (LLM) companies. Your work will directly help improve how AI models think, reason, and ...Show moreLast updated: 8 days ago
    • Promoted
    Senior QA Engineer - Smartphone

    Senior QA Engineer - Smartphone

    TransPerfectPune (division)
    We are seeking a Senior QA Engineer with extensive experience testing smartphone applications, leading quality strategy, and driving end-to-end testing across complex, cross-device ecosystems.Contr...Show moreLast updated: 3 days ago
    • Promoted
    Generative AI Engineer

    Generative AI Engineer

    CapgeminiPune (division)
    Experience with Natural Language Processing (NLP).Understanding of RAG, neural networks, and other AI / ML concepts.Experience with AI / ML frameworks like TensorFlow, PyTorch, etc.Experience with AI a...Show moreLast updated: 3 days ago
    • Promoted
    Data Architect

    Data Architect

    AccenturePune (division)
    Accenture is a global professional services company with leading capabilities in digital, cloud and security.Combining unmatched experience and specialized skills across more than 40 industries, we...Show moreLast updated: 1 day ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Persistent SystemsPune (division)
    We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services. The ideal candidate will have a strong backgro...Show moreLast updated: 23 days ago
    • Promoted
    • New!
    AWS Data Engineer

    AWS Data Engineer

    Capgemini InventPune (division)
    At Capgemini Invent, we believe difference drives change.As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to...Show moreLast updated: 22 hours ago
    • Promoted
    Gen AI Engineer

    Gen AI Engineer

    Cybage SoftwarePune (division)
    As a leader in technology & product engineering space, Cybage works with some of the world’s largest Independent Software Vendors (ISVs). Our solutions are focused on modern technologies and are ena...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    6221 Azure Data Engineer

    6221 Azure Data Engineer

    EXLPune (division)
    Key Role and Responsibilities : .Design and develop ETL / ELT pipelines using Azure Data Factory, Snowflake, and DBT.Build and maintain data integration workflows from various data sources to Snowflake...Show moreLast updated: 22 hours ago
    • Promoted
    • New!
    GCP data Engineer

    GCP data Engineer

    VOISPune (division)
    Hiring GCP Data Engineer with Vodafone Intelligent Solutions (_VOIS).Top skills for this position is : Google Cloud Platform (Composer, Big Query, Airflow, DataProc, Data Flow, GCS).Hands-on experie...Show moreLast updated: 22 hours ago