Talent.com
This job offer is not available in your country.
Lead Software Engineer (Lead ETL PySpark Developer)

Lead Software Engineer (Lead ETL PySpark Developer)

ConfidentialBengaluru / Bangalore
19 days ago
Job description

We are looking for a Lead ETL Developer to join our C3 Data team based in Bangalore. This role offers a unique opportunity to work on Clarivate Customer Cloud (C3) and make rich content available across the Clarivate platform to all customers. As the Lead ETL Developer, you will support the data team by loading new content to Clarivate Customer Cloud, and you will have the chance to work with modern, cutting-edge big data technologies. Our team has expertise in Python, PySpark, Spark, Databricks, ECS, AWS, and Airflow and we would love to speak with you if you have experience in Spark, Databricks, or PySpark.

About You - experience, education, skills, and accomplishments

  • At least 5+ years of experience in building  ETL pipelines  and proficiency in  PySpark .
  • At least 2 years of proficient in Python3 and experience with  Databricks  and Apache Spark
  • Strong SQL and non-SQL database skills.
  • Passionate about code and software architecture.

It would be great if you also had

  • AWS services : EC2, ECS, RDS, S3
  • OpenSearch, Snowflake
  • Oracle, PostgreSQL
  • Docker
  • What will you be doing in this role

  • Providing technical leadership and guidance, evaluating various technologies to balance business needs and cost-effectiveness.
  • Collaborating with business and IT teams to design and implement a  data lake platform .
  • Maintaining the overall solution architecture for the data lake platform.
  • Establishing technical best practices for big data management and solutions, from technology selection to architecture and implementation.
  • Ensuring all components of the data lake platform meet SLAs.
  • Leading technical investigations, POCs, and hands-on coding to make technical recommendations.
  • Mentoring and coaching team members to enhance their technical and professional skills.
  • Skills Required

    Oracle, Solution Architecture, Architecture, Coding, Data Management, Big Data, Python

    Create a job alert for this search

    Lead Software Engineer • Bengaluru / Bangalore

    Related jobs
    • Promoted
    ETL Developer

    ETL Developer

    Pinnacle Group, Inc.hosur, tamil nadu, in
    PTR Global is a leader in providing innovative workforce solutions, dedicated to optimizing talent acquisition and management processes. Our commitment to excellence has earned us the trust of busin...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Python Automation Lead

    Python Automation Lead

    Sourcebaehosur, tamil nadu, in
    Python scripting, Pytest, and CI / CD pipelines, preferably with experience in storage or embedded systems.The role involves designing automation frameworks, validating systems, and collaborating wit...Show moreLast updated: 19 hours ago
    • Promoted
    • New!
    Senior Developer – EMDA Analytics (ETL, Biovia Pipeline Pilot)

    Senior Developer – EMDA Analytics (ETL, Biovia Pipeline Pilot)

    Tech One IThosur, tamil nadu, in
    We are seeking a highly skilled.EMDA (Environmental Monitoring Data Analytics) team.The role focuses on developing, deploying, and maintaining ETL workflows for a. Pfizer’s global manufacturing site...Show moreLast updated: 19 hours ago
    • Promoted
    ETL LEAD with IBM DataStage experienced

    ETL LEAD with IBM DataStage experienced

    IntraEdgehosur, tamil nadu, in
    ETL Developer – DataStage, AWS, Snowflake.We are looking for a talented and motivated ETL Developer / Senior Developer.You will work on building scalable and efficient data pipelines using.IBM Data...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Manager - Data Engineering Lead

    Senior Manager - Data Engineering Lead

    DIAGEO Indiahosur, tamil nadu, in
    Senior Manager - Data Engineering Lead.Bachelor’s or master’s degree in computer science, Data Engineering, or related field. Experience in data engineering.Proven experience in cloud platforms (AWS...Show moreLast updated: 27 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Deltacubeshosur, tamil nadu, in
    Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 15 days ago
    • Promoted
    Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location

    Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location

    USThosur, tamil nadu, in
    CTC, notice period, and current location details to.Apache Hadoop, Airflow, Kubernetes, and Containers.Troubleshoot Hadoop log files and work with. Required Skills & Qualifications : .Scala, Spark, Py...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    SnapLogic ETL & Python Lead

    SnapLogic ETL & Python Lead

    Swift Strategic Solutions IncBengaluru, Karnataka, India
    Our technology services client is seeking multiple.These positions offer a strong potential for conversion to full-time employment upon completion of the initial contract period.Below are further d...Show moreLast updated: 22 hours ago
    • Promoted
    • New!
    Data Engineer

    Data Engineer

    TechVeritohosur, tamil nadu, in
    You will play a critical role in designing, building, and optimizing data workflows that enable scalable analytics and real-time insights. The ideal candidate is hands-on, detail-oriented, and passi...Show moreLast updated: 19 hours ago
    • Promoted
    Technical Lead

    Technical Lead

    SigmoidBengaluru, India
    Job Title : Technical Lead- Data Engineer.Years of Experience : 8+ years of experience.Sigmoid works with a variety of clients from start-ups to fortune 500 companies. We are looking for a detailed or...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Lead Software Engineer

    Lead Software Engineer

    Clarivatebangalore, India
    We are looking for a Lead Software Engineer to join our Policy Automation team in India to help us build an end-to-end data automation solution. If you passionate about coding, AI, and have extensiv...Show moreLast updated: 3 hours ago
    • Promoted
    Senior Cloud Engineer AWS

    Senior Cloud Engineer AWS

    Matrix USAhosur, tamil nadu, in
    We are seeking an experienced AWS Developer proficient in Python and PySpark to design, develop, and maintain scalable, serverless data processing and workflow automation solutions on AWS.The ideal...Show moreLast updated: 5 days ago
    • Promoted
    SAP Lead

    SAP Lead

    Ajiledone Technologieshosur, tamil nadu, in
    SAP CPI Consultant (Contract Role).We are seeking an experienced SAP CPI Consultant for a contract position, requiring 5-15 years of overall experience with a minimum of 2-3 end-to-end implementati...Show moreLast updated: 7 days ago
    • Promoted
    GIS Engineer

    GIS Engineer

    TAO Digital Solutionshosur, tamil nadu, in
    Build ingestion & validation pipelines (Landing → Bronze → Silver) using Spark / Glue / EMR.Standardize schemas, handle drift, quarantine & auditability. Convert to Geo Parquet & load Gold data into Aur...Show moreLast updated: 8 days ago
    • Promoted
    • New!
    Lead Engineer

    Lead Engineer

    HCLTechhosur, tamil nadu, in
    Architect efficient and reusable front-end systems to support complex interactions within Meta HW infrastructure.Develop full-stack web applications for internal infrastructure tooling using techno...Show moreLast updated: 19 hours ago
    • Promoted
    Senior ETL Developer

    Senior ETL Developer

    BCE Global TechBengaluru, Karnataka, India
    At BCE Global Tech, immerse yourself in exciting projects that are shaping the future of both consumer and enterprise telecommunications. This involves building innovative mobile apps to enhance use...Show moreLast updated: 24 days ago
    • Promoted
    Data Engineer Team Lead

    Data Engineer Team Lead

    SGIhosur, tamil nadu, in
    To be discussed based on your skills and experience.Strong hands-on data engineering experience with a proven ability to design, build, and optimize scalable data pipelines in .Deep technical exper...Show moreLast updated: 6 days ago
    • Promoted
    Practice Lead

    Practice Lead

    Zensar Technologieshosur, tamil nadu, in
    Technology & Delivery Leadership.NET practice strategy across modern engineering initiatives.NET 6+, Azure PaaS services, microservices, and event-driven patterns. Drive AI-first engineering practic...Show moreLast updated: 26 days ago
    • Promoted
    Data Engineer

    Data Engineer

    ACL Digitalhosur, tamil nadu, in
    Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago
    • Promoted
    Data Integration & LLM Engineer

    Data Integration & LLM Engineer

    Chargebeehosur, tamil nadu, in
    We are seeking a highly motivated.This role is ideal for engineers who enjoy working at the intersection of.APIs, SaaS connectors, and ETL / ELT pipelines to ensure reliable and scalable data flows.B...Show moreLast updated: 8 days ago