Talent.com
Lead Data Engineer
Lead Data EngineerINDI Staffing Services • dehradun, uttarakhand, in
Lead Data Engineer

Lead Data Engineer

INDI Staffing Services • dehradun, uttarakhand, in
13 days ago
Job description

At INDI, we're passionate about empowering individuals and businesses worldwide. Our cutting-edge recruiters connect leading companies with top talent, fostering a dynamic environment where innovation thrives. Join us in shaping the future of work.

Overview of the role :

The Lead Data Engineer position focuses on serving as core contributors to internal Data Platforms, responsible for building scalable, resilient pipelines that support Business Operations, Analytics, and cutting-edge AI / ML initiatives. This role involves designing, implementing, and optimizing data pipelines using a wide range of Big Data and cloud-native technologies while providing technical leadership, guiding personnel, and ensuring architectural consistency across projects.

Key responsibilities :

  • Providing technical leadership and mentorship to other data engineers, helping them develop and succeed in their roles.
  • Leading design reviews, enforcing coding standards, and guiding architectural decisions to ensure platform scalability and maintainability.
  • Partnering with senior stakeholders and engineering leadership to prioritize and plan technical work that aligns with strategic business goals.
  • Acting as technical points of contact for cross-team collaborations, participating in project planning, scoping, and retrospectives.
  • Demonstrating curiosity, initiative, and capability to integrate AI-powered tools into day-to-day engineering workflows.
  • Supporting AI / GenAI projects by building pipelines that feed LLM apps and recommendation systems.
  • Driving adoption of data engineering best practices across organizations, including testing, CI / CD, and observability.
  • Designing, building, and maintaining batch and streaming data pipelines using native data platforms.
  • Ingesting data from various sources, including REST APIs, cloud services, and enterprise SaaS platforms.
  • Implementing robust transformation logic using Python, PySpark, SQL, and Java across structured and semi-structured datasets.
  • Ensuring data integrity, lineage, and performance across ingestion, transformation, and delivery layers.

Requirements :

  • Data Engineering Experience : Substantial background (6+ years) as Data Engineer in enterprise-scale data infrastructures.
  • Leadership Skills : Proven leadership in technical project management and personnel mentorship.
  • Architectural Guidance : Track record of guiding architectural conversations in high-throughput data ecosystems.
  • Programming Mastery : Advanced proficiency in SQL and Python programming, with expertise in PySpark or Java.
  • Data Architecture : Comprehensive knowledge of data modeling, transformation methodologies, and data warehouse architectures.
  • Hadoop Ecosystem : Proven expertise with Hadoop ecosystem technologies.
  • Pipeline Development : Successful implementation of resilient, high-performance data pipelines meeting strict service level agreements.
  • Integration Expertise : Deep understanding of API integration, performance optimization, and data schema adaptation.
  • Language Proficiency : Advanced level of English.

    • Additional skills preferred :

  • Workflow Tools : Practical implementation experience with Apache NiFi and Apache Airflow.
  • Development Environment : Expertise in containerized environments, developer tooling, and API frameworks.
  • AI / ML Platforms : Demonstrated capabilities in AI / ML platform support and generative AI pipeline development.
  • Innovation Contributions : Proven contributions to internal technological innovations within data engineering workflows.

    • What to expect from us :

  • Competitive Compensation : Excellent payment in USD or your preferred local currency.
  • Paid Leave : Parental leave, vacation, and national holidays.
  • Dynamic Work Culture : Innovative and multicultural environment.
  • Elite Collaboration : Work with the global top 1% of talent.
  • Growth and Support : Mentorship and career development opportunities.

    • If you are interested in being part of a team composed of the best professionals and working 100% goal-oriented in an innovative environment, do not hesitate to apply!

    Create a job alert for this search

    Lead Data Engineer • dehradun, uttarakhand, in