Spark Scala Applications Development Technology Lead Analyst - Vice President

Description

: Lead Spark Scala Engineer

• Hands on knowledge of SPARK, Py-Spark, scala, Java etc .
• Hands on knowledge of RDBMS like MS-SQL / Oracle
• Knowledge of CI / CD tools like LightSpeed, uDeploy etc

Responsibilities

Development & Optimization : Develop, test, and deploy production-grade Spark applications in Scala, ensuring optimal performance, scalability, and resource utilization.

Technical Leadership : Provide technical leadership and guidance to a team of data engineers, fostering a culture of technical excellence, collaboration, and continuous improvement.

Code Review & Best Practices : Conduct rigorous code reviews, establish coding standards, and enforce best practices for Spark Scala development, data governance, and data quality.

Performance Tuning : Identify and resolve performance bottlenecks in Spark applications through advanced tuning techniques (, shuffling, partitioning, memory management, caching).

Deep Spark Expertise :

Profound understanding of Spark's architecture (RDDs, DataFrames, Datasets), execution model, and optimization techniques.

Extensive experience with Spark SQL, Spark Streaming, and Spark Core.

Proven ability to design and build complex data pipelines (ETL / ELT) .

Scala Proficiency : Expert-level proficiency in Scala programming, including functional programming paradigms and object-oriented design.

Big Data Ecosystem : Strong hands-on experience with the broader Hadoop ecosystem (HDFS, YARN, Hive, Kafka, etc.) and related big data technologies.

Database Knowledge : Solid understanding of relational databases (SQL, schema design) and NoSQL databases .

Communication : Excellent communication, interpersonal, and leadership skills, with the ability to articulate complex technical concepts to diverse audiences.

Problem-Solving : Exceptional analytical and problem-solving abilities, with a meticulous attention to detail.

Education :

Bachelor’s degree / University degree or equivalent experience

Job Family Group : Technology

Job Family :

Applications Development

Time Type : Full time

Most Relevant Skills

Please see the requirements listed above.

Other Relevant Skills

For complementary skills, please see above and / or contact the recruiter.