Data Pipeline Manager

At INDI, we're passionate about empowering individuals and businesses worldwide. Our cutting-edge recruiters connect leading companies with top talent, fostering a dynamic environment where innovation thrives. Join us in shaping the future of work.

Overview of the role :

The Lead Data Engineer position focuses on serving as core contributors to internal Data Platforms, responsible for building scalable, resilient pipelines that support Business Operations, Analytics, and cutting-edge AI / ML initiatives. This role involves designing, implementing, and optimizing data pipelines using a wide range of Big Data and cloud-native technologies while providing technical leadership, guiding personnel, and ensuring architectural consistency across projects.

Key responsibilities :

• Providing technical leadership and mentorship to other data engineers, helping them develop and succeed in their roles.
• Leading design reviews, enforcing coding standards, and guiding architectural decisions to ensure platform scalability and maintainability.
• Partnering with senior stakeholders and engineering leadership to prioritize and plan technical work that aligns with strategic business goals.
• Acting as technical points of contact for cross-team collaborations, participating in project planning, scoping, and retrospectives.
• Demonstrating curiosity, initiative, and capability to integrate AI-powered tools into day-to-day engineering workflows.
• Supporting AI / GenAI projects by building pipelines that feed LLM apps and recommendation systems.
• Driving adoption of data engineering best practices across organizations, including testing, CI / CD, and observability.
• Designing, building, and maintaining batch and streaming data pipelines using native data platforms.
• Ingesting data from various sources, including REST APIs, cloud services, and enterprise SaaS platforms.
• Implementing robust transformation logic using Python, PySpark, SQL, and Java across structured and semi-structured datasets.
• Ensuring data integrity, lineage, and performance across ingestion, transformation, and delivery layers.

Requirements :

Additional skills preferred :

What to expect from us :

If you are interested in being part of a team composed of the best professionals and working 100% goal-oriented in an innovative environment, do not hesitate to apply!