Big Data Engineer - Apache Spark

Job Description :

We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.

Core Responsibilities :

• Design, build, and maintain robust data pipelines (batch or streaming) that process and transform data from diverse sources.
• Ensure data quality, reliability, and availability across the pipeline lifecycle.
• Collaborate with product managers, architects, and engineering leads to define technical strategy.
• Participate in code reviews, testing, and deployment processes to maintain high standards.
• Own smaller components of the data platform or pipelines and take end-to- end responsibility.
• Continuously identify and resolve performance bottlenecks in data pipelines.
• Take initiatives, and show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have.

Required Qualifications :

Tooling & Ecosystem :

(ref : hirist.tech)