Lead Data Pipeline Engineer

Data Engineer, Forward Deployed Applications

Location : Hyderabad, India (on-site at client location)

Work Site : On-site

Travel : 0–10%

Role Type : Individual Contributor

Employment Type : Full-time

Job Description

We are hiring a Data Engineer to build reliable data pipelines and infrastructure that support machine learning, analytics, and scientific applications. This role will enable downstream AI engineers and scientists to work efficiently with structured, validated, and high-quality data.

The Role

We are looking for a senior data engineer with 5–7 years of experience to join our Forward Deployed Applications team. This is a customer-facing, project-owning role, you will work directly with engineers and domain experts at client sites to explore raw datasets, design transformation workflows, and deliver production-ready pipelines.

While your primary focus will be data architecture, exploration, and transformation, you will also build Streamlit applications to expose processed data in intuitive, usable ways. You will deliver your own projects end-to-end from discovery to deployment and work independently alongside backend engineers who focus more on application-heavy builds.

Why you will want this job

You will have a direct hand in turning messy, unstructured industrial data into actionable insights that power AI models and decision-making tools. You will be embedded with customer teams, influencing both the data foundation and the applications that depend on it. And you will see your work in production quickly, often within days or weeks.

What you will do

• Partner with customer engineers and domain experts to understand data sources, quality challenges, and integration requirements.
• Design and build ETL / ELT workflows for structured and unstructured datasets, aligning them to analytics, ML, and simulation needs.
• Develop and maintain scalable data pipelines with orchestration tools like Airflow or Prefect.
• Create and maintain data validation, profiling, and observability systems.
• Build and maintain data cataloging, lineage, and metadata tracking to improve discoverability and governance.
• Design schemas and data models that support downstream AI and application workflows.
• Build Streamlit applications for data exploration, visualization, and decision support, with an emphasis on showcasing processed, validated datasets.
• Contribute data-related features and integrations back to AppHub when needed.
• Operate independently on projects while collaborating closely with backend engineers, platform teams, and AI teams.

What we are looking for

Nice to Have