Talent.com
This job offer is not available in your country.
Senior Data Engineer- AWS

Senior Data Engineer- AWS

InfoCeptsPune, Maharashtra, India
11 days ago
Job description

Key Result Areas and Activities :

  • Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements.
  • Develop spark jobs using Scala for new project requirements.
  • Enhance existing spark jobs for any ongoing product enhancement.
  • Performance tuning of spark jobs, stress testing etc.
  • Create new data pipelines for developed / enhanced spark jobs using AWS Lambda or Apache Airflow
  • Responsible for database design process, logical design, physical design, star schema, snowflake schema etc.
  • Analyze data processing, integration, modelling and reporting requirements & Define data loading strategies considering volume, data types, frequency and analytics specifications.
  • Ensure optimal balance between cost and performance.
  • Project documentation, Adheres to Quality guidelines & Schedules.
  • Works hand in hand with PM for successful delivery of project and provide Estimation, scoping, scheduling assistance.
  • Manage build phase and quality assure code to ensure fulfilling requirements and adhering to Cloud architecture. Resolve difficult design and develop issues.

Work and Technical Experience :

Must-Have :

  • Overall, 7-9 years of IT Exp
  • 5+ years of AWS related project
  • Good to have Associate Level and Professional Level AWS Certification
  • In depth knowledge of following AWS Services required : S3, EC2, EMR, Severless, Athena, AWS Glue, Lambda, Step Functions
  • Cloud Databases (Must have) – AWS Aurora, Singlestore, RedShift, Snowflake
  • Big Data (Must have) - Hadoop, Hive, Spark, YARN
  • Programming Language (Must have) – Scala, Python, Shell Scripts, PySpark
  • Operating System (Must have) - Any flavor of Linux, Windows
  • Must have very strong SQL Skills
  • Orchestration Tools (Must have) : Apache Airflow
  • Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.
  • Should have thorough conceptual understanding of AWS VPC, Subnets, Security Groups, & Route Tables
  • Should be a quick and self-learner and be ready to adapt to new AWS Services or new Big Data Technologies as and when required

    • Qualifications :

  • Bachelor’s degree in computer science, engineering, or related field (Master’s degree is a plus)
  • Demonstrated continued learning through one or more technical certifications or related methods
  • Minimum 5 years of experience on Cloud related projects

    • Qualities :

  • Hold strong technical knowledge and experience
  • Should have the capability to deep dive and research in various technical related fields
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm
  • Able to communicate persuasively through speaking, writing, and client presentations
  • Able to consult, write, and present persuasively
  • Able to work in a self-organized and cross-functional team
  • Able to iterate based on new information, peer reviews, and feedback
  • Prior experience of working in a large media company would be added advantage
    • Create a job alert for this search

    Aws Data Engineer • Pune, Maharashtra, India