Talent.com
5447 | Senior Data Engineer

5447 | Senior Data Engineer

EXLPune (division)
1 day ago
Job description

Title : Senior Data Engineer

Experience : 6+ Years

We are seeking a Senior Data Engineer to build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging. You will have the opportunity to use your expertise in solving big data problems and apply design thinking, coding and analytical skills to develop data pipelines that support marketing data products. We’re looking for talented Data Engineers passionate about building new data-driven solutions with the latest Big Data technology.

Responsibilities for Internal Candidates

  • You will build services that will integrate directly with LendingClub’s products and with external vendors.
  • You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster / Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
  • Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
  • Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
  • Support operations to manage the production environment and help in resolving production issues with RCA
  • Write unit / integration tests, adopt Test-driven development, contribute to engineering wiki, and document design / implementation etc.

Qualifications for Internal Candidates

  • 5+ years of experience and a bachelor’s degree in computer science or a related field, or equivalent work experience
  • In-depth working experience of distributed systems Hadoop, Spark, Hive, DBT and Airflow / Dagster
  • At least 4 years of solid production quality coding experience in data pipeline implementation in Python
  • Experience working with public cloud platforms, preferably AWS
  • Experience working with Databricks and / or Snowflake
  • Experience in Git, JIRA, Jenkins, shell scripting
  • Familiarity with Agile methodology, test-driven development, source control management and test automation
  • Create a job alert for this search

    Senior Data Engineer • Pune (division)

    Related jobs
    • Promoted
    Senior Data Engineer( DWH)

    Senior Data Engineer( DWH)

    MastercardPune (division)
    Mastercard is a global technology company in the payments industry.Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, ...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer

    Data Engineer

    Enterprise Minds, IncPune (division)
    As a data engineer, you will be responsible for delivering data intelligence solutions to our customers all around the globe, based on an innovative product, which provides insights into the perfor...Show moreLast updated: 1 day ago
    • Promoted
    Data Solution Engineer I II and III

    Data Solution Engineer I II and III

    OpenGov Inc.Pune (division)
    OpenGov is the leader in AI and ERP solutions for local and state governments in the U.More than 2,000 cities, counties, state agencies, school districts, and special districts rely on the OpenGov ...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Software Engineer

    Software Engineer

    Alp Consulting Ltd.Pune (division)
    Years of in Big Data & Data related technology experience.Expert level understanding of distributed computing principles. Expert level knowledge and experience in Apache Spark.Hands on programming w...Show moreLast updated: 21 hours ago
    • Promoted
    Data Engineer

    Data Engineer

    Persistent SystemsPune (division)
    We are looking for Strong Data engineers to join the Data Integration team and become part of our successful group.Location : All Persistent Locations. Job Type : Full Time Employment.Design, code, te...Show moreLast updated: 10 days ago
    • Promoted
    Senior Data Engineer- AWS

    Senior Data Engineer- AWS

    InfoCeptsPune (division)
    Key Result Areas and Activities : .Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements. Should be able to design comple...Show moreLast updated: 1 day ago
    • Promoted
    Cloud Data Engineer

    Cloud Data Engineer

    Persistent SystemsPune (division)
    We are seeking cloud data engineers with hands on experience in azure data bricks, data factory, SQL, etc.Job Location : All Persistent Locations. Job Type : Full Time Employment.Collaborate with prod...Show moreLast updated: 10 days ago
    • Promoted
    GCP Data Engineer

    GCP Data Engineer

    Persistent SystemsPune (division)
    We are seeking the GCP Data Engineer with hands on experience in data engineering, python, jaba, etc.Job Type : Full Time Employment. Design, build, and manage data pipelines on Google Cloud Platform...Show moreLast updated: 10 days ago
    • Promoted
    Data Engineer

    Data Engineer

    SynechronPune (division)
    We have immediate opportunity for.Notice Period : Immediate to 30 Days.At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines cre...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer

    Data Engineer

    Bits In GlassPune (division)
    Senior Data Engineer / ETL Engineer.As a Senior Data Engineer / ETL Engineer, you will be instrumental in designing, developing, and optimizing data processing systems that support our organization...Show moreLast updated: 10 days ago
    • Promoted
    GCP Data Engineer @ Pune

    GCP Data Engineer @ Pune

    Quess IT StaffingPune (division)
    Show moreLast updated: 1 day ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Persistent SystemsPune (division)
    We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services. The ideal candidate will have a strong backgro...Show moreLast updated: 10 days ago
    • Promoted
    Data Engineer (PySpark)

    Data Engineer (PySpark)

    RSquareSoft TechnologiesPune (division)
    Design, develop, and maintain scalable.Work with large datasets using.S3, Glue, Lambda, and Redshift.Collaborate with cross-functional teams to define data requirements and optimize data flows.Ensu...Show moreLast updated: 1 day ago
    • Promoted
    5676 - Data Engineer

    5676 - Data Engineer

    EXLPune (division)
    Location - Pune, Bangalore, Noida, Gurgaon, Hyderabad.The ideal candidate will have strong expertise in Snowflake, Hadoop ecosystem, PySpark, and SQL, and will play a key role in enabling data-driv...Show moreLast updated: 1 day ago
    • Promoted
    GenAI Engineer

    GenAI Engineer

    Persistent SystemsPune (division)
    This position focuses on developing and maintaining advanced Generative AI solutions, including model lifecycle management, Natural Language to SQL agents, and chatbot systems.The role involves wor...Show moreLast updated: 1 day ago
    • Promoted
    2917 - Data Engineer

    2917 - Data Engineer

    EXLPune (division)
    The Data Engineer is responsible for designing, building, and optimizing data systems that power analytics, reporting, and machine learning across the organization. The role involves developing scal...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Senior Database Engineer

    Senior Database Engineer

    mthreePune (division)
    Multinational Investment Bank and Financial Services Company.Advanced hands-on experience in configuring, patching, upgrading, and optimizing database software and related products.Advanced experie...Show moreLast updated: 21 hours ago
    • Promoted
    Senior System Engineer

    Senior System Engineer

    Resonating Mindz Pvt LtdPune (division)
    Best Opportunity for Automation Engineers to Build a Rewarding Career in the IIoT Domain!.Are you an Industrial Automation Engineer who wants to take on new challenges in the emerging field of.Indu...Show moreLast updated: 1 day ago