Sr. / Principle engineer - Bangalore

Strong experience(12+years)inScala programming language and extensive experience with Apache Spark for Big Data processing for design, developing and maintaining scalable on-prem and cloud environments, especially onAWS and as needed with GCP cloud.

Proficiency in performance tuning of Spark jobs, optimizing resource usage, shuffling, partitioning, and caching for maximum efficiency in Big Data environments.

In-depth understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.

Expertise in designing and implementing scalable, fault-tolerant data pipelines with end-to-end monitoring and alerting.

Using Python to develop infrastructure modules. Hence, hands-on experiencewithPython.

Solid grasp of database systems and SQLs for writing efficient SQLs (RDBMS / Warehouse) to handle TBS of data.

Familiarity with design patterns and best practices for efficient data modelling, partitioning strategies, and sharding for distributed systems and experience in building, scheduling and maintaining DAG workflows.

End-to-end ownership withdefinition, development, and documentation of softwares objectives, business requirements, deliverables, and specifications in collaboration withstakeholders.

Experience in working onGIT(or equivalent source control) and solid understanding of Unit and integration test frameworks.

Must have the ability to collaborate with stakeholders / teams to understand requirements and develop a working solution and the ability to work within tight deadlines and effectively prioritize and execute tasks in a high-pressure environment.

Must be able to mentor junior staff.

Skills Required

Hadoop, Scala, Python, Aws