Senior Software Engineer - C++/Java/Golang

Description :

We are seeking a Software Engineer to design and build high performance distributed data ingestion systems.

You will work on parallel processing pipelines that integrate with cloud storage, relational databases, and streaming platforms.

This is a core product development role where youll apply best practices in distributed system design, ensuring scalability, reliability, and performance.

Key Responsibilities :

- Design and implement data ingestion pipelines with parallel processing in Java, C++, or Golang.

- Ingest data from Amazon S3, Azure Cloud Storage, Google Cloud Storage, Snowflake, BigQuery, PostgreSQL, files, Kafka, and Iceberg data lakehouse.

- Build high availability (HA) loading systems with cross region replication.

- Develop monitoring and error reporting for ingestion pipelines.

- Integrate with Spark connectors and manage third party systems (Kafka, Kafka Connect).

- Collaborate with cross- functional teams to ensure scalable, reliable, and performant distributed systems.

- Follow Agile development practices and contribute to CI/CD pipelines.

Requirements :

- Strong programming skills in Java, C++, or Golang.

- Hands- on experience with Kafka, Zookeeper, Spark, or stream processing framework.

- Expertise in Kafka Connect, Kafka Streams, Kafka security, and customization.

- Experience with Spark connectors and event- driven architecture.

- Familiarity with Agile development and CI/CD workflow.

Nice to Have :

- Experience with gRPC protocol and multi-threading.

- Exposure to Zookeeper, ETCD, or Consul.

- Understanding of distributed consensus algorithms (Paxos/Raft).

- Knowledge of Docker and Kubernetes for containerized deployment.

(ref:hirist.tech)