Lead Data Pipeline Engineer

About Yubi

Yubi, formerly known as CredAvenue, is re-defining global debt markets by freeing the flow of finance between borrowers, lenders, and investors. We are the world's possibility platform for the discovery, investment, fulfillment, and collection of any debt solution. At Yubi, opportunities are plenty and we equip you with tools to seize it.

In March 2022, we became India's fastest fintech and most impactful startup to join the unicorn club with a Series B fundraising round of $137 million.

In 2020, we began our journey with a vision of transforming and deepening the global institutional debt market through technology. Our two-sided debt marketplace helps institutional and HNI investors find the widest network of corporate borrowers and debt products on one side and helps corporates to discover investors and access debt capital efficiently on the other side. Switching between platforms is easy, which means investors can lend, invest and trade bonds - all in one place. All of our platforms shake up the traditional debt ecosystem and offer new ways of digital finance.

Spocto - Debt recovery & risk mitigation platform

So far, we have on-boarded over 17000+ enterprises, 6200+ investors & lenders and have facilitated debt volumes of over INR 1,40,000 crore.

Backed by marquee investors like Insight Partners, B Capital Group, Dragoneer, Sequoia Capital, LightSpeed and Lightrock, we are the only-of-its-kind debt platform globally, revolutionizing the segment.

At Yubi, People are at the core of the business and our most valuable assets. Yubi is constantly growing, with 1000+ like-minded individuals today, who are changing the way people perceive debt. We are a fun bunch who are highly motivated and driven to create a purposeful impact. Come, join the club to be a part of our epic growth story.

About the Role

As a Senior Data Engineer, you will design, build, and maintain efficient data pipelines that drive our data-driven decision-making. You'll ensure seamless data integration and processing from multiple sources, while optimising performance and reliability. Collaboration with cross-functional teams will be key to meeting business needs in both on-premise and cloud environments.

Role and Responsibilities

• Utilise PySpark and Python to build scalable data pipelines.
• Integrate data from multiple source systems into a unified target.
• Orchestrate and schedule pipelines using Airflow or Temporal for reliable execution.
• Enhance existing pipelines to meet evolving business requirements.
• Optimise pipelines for performance and cost.
• Troubleshoot and resolve pipeline failures quickly.
• Conduct root cause analysis and implement corrective measures.
• Collaborate with stakeholders to address data processing needs.
• Provide support on weekends and shifts as required.
• Travel to client locations for on-prem pipeline development, deployment and support.

Experience & Expertise