Senior Data Engineer : Joining by 1st Dec 2025

Position : Senior Data Engineer

Experience : 6 9 Years

Role Summary

We are seeking Senior Data Engineer resources to work on the migration of applications from

our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires

strong hands-on development skills in data engineering, with the ability to deliver high-quality

pipelines under guidance from internal leads.

Key Responsibilities

Develop and optimize data pipelines using Spark 3.5 and Python / Scala.

Migrate existing Hive, Spark, and Control-M jobs to Airflow and DBT-based workflows.

Integrate data pipelines with messaging systems (Kafka, Solace) and object stores (S3,

MinIO).

Troubleshoot and optimize distributed jobs running in Kubernetes environments.

Collaborate closely with internal leads and architects to implement best practices.

Design and implement migration / acceleration framework to automate end to end

migration.

Continuous enhancements to the frameworks to ensure the stability, scalability and

support for diverse use cases and scenarios.

Work with various data applications to enable and support the migration process.

Deliver assigned migration tasks within agreed timelines.

Required Skills

6 9 years of hands-on data engineering experience. Strong expertise in Apache Spark (batch + streaming) and Hive.

Proficiency in Python, Scala, or Java. Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation

frameworks (DBT preferred).

Experience working with Kafka, Solace, and object stores (S3, MinIO). Exposure to Docker / Kubernetes for deployment.

Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).

INTERNAL

Engagement Expectations

Vendor resource will be expected to work independently on assigned modules and

deliver production-ready solutions.

Must participate in daily / weekly status calls, reviews, and issue resolution. Should adapt to client processes, documentation standards, and compliance

requirements.