Hadoop Development Lead

Job Title : Hadoop Development Lead

Primary Skill : PySpark , Hadoop, Hive, SQL, must have worked as team lead for minimum 2-3 years.

Location : Hyderabad / Pune / Coimbatore

Mode of Work : Hybrid

Experience : 7-10 years

About the Job :

We are seeking an experienced Hadoop Data Engineer to join our team- a team of high-energy individuals who thrive in a rapid-pace and agile product development environment. The ideal candidate will have extensive experience in Hadoop development, strong knowledge of PySpark programming, and expertise in Hadoop, Hive framework. You will be responsible for writing complex SQL queries.

Know your team :

At ValueMomentum’s Engineering Center , we are a team of passionate engineers who thrive on tackling complex business challenges with innovative solutions while transforming the P&C insurance value chain. We achieve this through strong engineering foundation and continuously refining our processes, methodologies, tools, agile delivery teams, and core engineering archetypes. Our core expertise lies in six key areas : Cloud Engineering, Application Engineering, Data Engineering, Core Engineering, Quality Engineering, and Domain expertise.

Join a team that invests in your growth. Our Infinity Program empowers you to build your career with role-specific skill development leveraging immersive learning platforms. You’ll have the opportunity to showcase your talents by contributing to impactful projects.

Responsibilities :

• Batch Processing Optimization : Analyse and optimize existing batch workflows to ensure efficient data processing.
• Development & Scripting : Write and maintain batch scripts using Python, Hive, and Shell to automate data workflows.
• Data Analysis : Utilize Hadoop and related technologies to gather, analyse, and report on data performance and processing metrics.
• Troubleshooting : Identify and resolve data processing bottlenecks and workflow issues to minimize downtime and ensure smooth operations.
• Collaboration : Work closely with data engineering, data science, and business intelligence teams to understand requirements and improve the data pipeline.
• Documentation : Develop and maintain technical documentation for workflows, troubleshooting guides, and best practices.
• Continuous Improvement : Propose and implement improvements for existing batch processes and pipelines.

Requirements :