Talent.com
This job offer is not available in your country.
Big Data Developer - Scala / Spark

Big Data Developer - Scala / Spark

Kezan ConsultingGurugram
30+ days ago
Job description

Position : Big Data Engineer (Immediate Joiner)

Experience : 5+ Years

Location : (Gurugram / Bangalore)

Joining : Immediate Joiner

Job Summary :

We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala.

The ideal candidate will have expertise in building data pipelines, data warehousing, and data processing applications.

Key Responsibilities :

Data Warehousing :

  • Design, develop, and maintain large-scale data warehouses using Amazon Redshift.
  • Optimize Redshift cluster performance, scalability, and cost-effectiveness.

Data Pipelines :

  • Build and maintain data pipelines using Apache Spark, Scala, and AWS services like S3, Glue, and Lambda.
  • Ensure data quality, integrity, and security across the data pipeline.

    • Data Processing :

  • Develop and optimize data processing applications using Spark, Scala, and AWS services.
  • Work with data scientists and analysts to develop predictive models and perform advanced analytics.

    • AWS Services :

  • Leverage AWS services like S3, Glue, Lambda, and IAM to build scalable and secure data systems.
  • Ensure data systems are highly available, scalable, and and Optimization :
  • Troubleshoot and optimize data pipeline performance issues.
  • Ensure data systems are optimized for cost, performance, and scalability.

    • Requirements :

    Experience : 5+ years of experience in big data engineering or a related field.

    Technical Skills :

  • Proficiency in Amazon Redshift, Apache Spark, and Scala.
  • Experience with AWS services like S3, Glue, Lambda, and IAM.
  • Knowledge of data processing frameworks like Spark and data storage solutions like S3 and Redshift.

    • Data Architecture : Strong understanding of data architecture principles and design patterns.

    Problem-Solving : Excellent problem-solving skills and attention to detail.

    Preferred Qualifications :

    Certifications : AWS Certified Big Data Specialty or similar certifications.

    Machine Learning : Familiarity with machine learning frameworks like Spark MLlib or TensorFlow.

    Agile Methodology : Experience working in agile development environments.

    Data Governance : Experience with data governance, data quality, and data security.

    (ref : hirist.tech)

    Create a job alert for this search

    Big Data Developer • Gurugram