Specialist - Data Engineering

Job Description

Senior Specialist - Data Engineering

Job Summary :

We are seeking a highly skilled Senior Specialist - Data Engineering with strong Spark Scala knowledge, mandatory for a significant and complex engagement. An AWS background is welcome.

As a Big Data Full Stack Developer, you will work with various underlying tools such as Spark / Scala, Oozie, Postgre, and Couchbase. The role requires collaborating with architects to implement the application solution successfully.

Job Responsibilities :

• Develop Scala / Spark jobs for data transformation and aggregation.
• Modify and enhance the underlying Scala framework, as required.
• Work with Spark Streaming jobs, handle in-memory Spark state, and process messages for near-real-time systems.
• Optimize Spark queries for improved performance.
• Integrate with SQL databases, especially Postgre and Couchbase.
• Collaboratively solve technical problems within specified timelines with minimal guidance.
• Implement various engineering tasks, including gap analysis, data analysis, and impact analysis.
• Independently perform low-level design configuration, code customization, refactoring, and code reviews.
• Demonstrate expertise in design, development, testing, build, and deployment stages of the software lifecycle.
• Ensure thorough documentation and provide reliable engineering support to all stakeholders.

Required Skills :

(ref : hirist.tech)