Talent.com
AWS Lead Big Data Engineer

AWS Lead Big Data Engineer

ConfidentialPune, Noida, Indore
30+ days ago
Job description

Do you love to work on bleeding-edge Big Data technologies, do you want to work with the best minds in the industry, and create high-performance scalable solutions Do you want to be part of the team that is solutioning next-gen data platforms Then this is the place for you.

You want to architect and deliver solutions involving data engineering on a Petabyte scale of data, that solve complex business problems

Impetus is looking for a Big Data Developer that loves solving complex problems, and architects and delivering scalable solutions across a full spectrum of technologies.

  • Experience in providing technical leadership in the Big Data space (Hadoop Stack like Spark, M / R, HDFS, Hive, etc.
  • Should be able to communicate with the customer in the functional and technical aspects
  • Expert-level proficiency in Python / Pyspark
  • Hands-on experience with Shell / Bash Scripting (creating, and modifying scripting files)
  • Control-M, AutoSys, Any job scheduler experience
  • Experience in visualizing and evangelizing next-generation infrastructure in Big Data space (Batch, Near Real-time, Real-time technologies).
  • Should be able to guide the team for any functional and technical issues
  • Strong technical development experience in effectively writing code, code reviews, and best practices code refactoring.
  • Passionate for continuous learning, experimenting, applying and contributing towards cutting-edge open-source technologies and software paradigms
  • Good communication, problem-solving & interpersonal skills. Self-starter & resourceful personality with the ability to manage pressure situations.
  • Capable of providing the design and Architecture for typical business problems.
  • Exposure and awareness of complete PDLC / SDLC.
  • Out of box thinker and not just limited to the work done in the projects.

Must Have

  • Experience with AWS(EMR, Glue, S3, RDS, Redshift, Glue)
  • Cloud Certification
  • valuate and recommend the Big Data technology stack best suited for customer needs.
  • Design / Architect / Implement various solutions arising out of high concurrency systems
  • Responsible for timely and quality deliveries
  • Anticipate on technological evolutions
  • Ensure the technical directions and choices.
  • Develop efficient ETL pipelines through spark or Hive.
  • Drive significant technology initiatives end to end and across multiple layers of architecture
  • Provides strong technical leadership in adopting and contributing to open-source technologies related to Big Data across multiple engagements
  • Designing / architecting complex, highly available, distributed, failsafe compute systems dealing with a considerable amount (GB / TB) of data
  • Identify and work on incorporating Non-functional requirements into the solution (Performance, scalability, monitoring etc.)

    • Role :   Data warehouse Architect / Consultant

    Industry Type :   IT Services & Consulting

    Department :   Engineering - Software & QA

    Employment Type :   Full Time, Permanent

    Role Category :   DBA / Data warehousing

    Education

    UG :   B.Tech / B.E. in Any Specialization

    PG :   M.Tech in Any Specialization, MCA in Computers, M.Com in Commerce

    Skills Required

    Open Source, Architecture, Spark, Big Data, Control-M, Technical Leadership, Python, Sdlc, Monitoring

    Create a job alert for this search

    Big Data Engineer • Pune, Noida, Indore