Principal Data Systems Manager

Role Overview :

As Director of Engineering, you will lead the technical direction and execution of our engineering organization. You will be directly involved in system design, architecture

reviews, code quality, and scaling data pipelines. You will work with senior engineers and architects to design resilient, high-throughput systems capable of

processing massive event streams in real time. This role demands someone who can dive deep technically, make critical architectural decisions, and set engineering standards, while also fostering a high-performance engineering culture.

Key Responsibilities :

• Lead architecture and implementation of scalable, distributed data pipelines

(real-time and batch) for ingesting and processing security telemetry at high

volume (5,000–10,000+ events / sec).

(Kafka, Spark, Flink, Elasticsearch, Hadoop ecosystem).

Qualifications :

○ Strong coding background in Java and Python with experience building scalable, cloud-native enterprise applications.

○ Experience designing and implementing microservices based architecture using Java, SpringBoot and relevant technologies.

○ Deep expertise in big data systems : Kafka, Flink, Hadoop, Elasticsearch, etc.

○ Experience designing and scaling real-time and batch data pipelines at enterprise scale.

○ Solid knowledge of distributed systems, concurrency, fault tolerance, and data consistency models.

○ Strong grasp of cloud-native infrastructure (Kubernetes, Docker, Terraform, CI / CD pipelines).

○ Experience with RDBMS and NoSQL databases (e.G., Postgres, MySQL, MongoDB, Redis).

○ Prior experience in deploying workloads at scale in AWS EKS / ECS / Fargate

○ 10+ years in software engineering, with at least 5 years leading high performing technical teams.

○ Experience leading a high-trust, high-performance engineering culture, where teams are empowered to innovate and deliver enterprise-grade solutions at scale.

○ Proven track record building enterprise applications with massive data throughput.

Write to me at rajeshwari.vh@careerxperts.com for more details