Talent.com
Senior Data Engineer- AWS
Senior Data Engineer- AWSConfidential • Pune, India
Senior Data Engineer- AWS

Senior Data Engineer- AWS

Confidential • Pune, India
29 days ago
Job description

Key Result Areas and Activities :

  • Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
  • Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect
  • Should be able to assist the architect in reasoning about various architecture choices in terms of cost, performance and durability.
  • Should be able to suggest optimizations in existing designs.
  • Project documentation adheres to quality guidelines & schedules.
  • Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.
  • Articulate design decisions very clearly with stakeholders.
  • Perform Proof-of-Concepts and document all observations before proposing a new solution.
  • Conduct design review sessions with other teams and identify scope of improvement.
  • Suggests automation to improve existing processes.

Work and Technical Experience :

Must-Have :

  • In-depth knowledge of the following AWS services : S3, EC2, EMR, Athena, AWS Glue, Lambda
  • Experience with at least one MPP database : AWS Redshift, Snowflake, SingleStore
  • Proficiency in Big Data technologies : Apache Spark, Databricks
  • Must have strong programming skills in Python
  • Responsible for building data pipelines in AWS And Databricks
  • Experience with Big Data table formats, such as Delta Lake (open source)
  • Must have very strong SQL skills
  • Experience with orchestration tools like Apache Airflow
  • Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.
  • Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required
  • Strong understanding of data warehousing concepts
  • Good to have :

  • Cloud Databases – Snowflake, AWS Aurora
  • Big Data – Hadoop, Hive
  • Cloud Databases– AWS Aurora
  • Associate Level or Professional Level AWS Certification or Databricks Certification
  • Qualifications :

  • Overall, 7+ years of IT Exp
  • 5+ years of relevant experience in AWS related project
  • Bachelor's degree in computer science, engineering, or related field (master's degree is a plus)
  • Demonstrated continued learning through one or more technical certifications or related methods.
  • Qualities :

  • Hold strong technical knowledge and experience
  • Should have the capability to deep dive and research in various technical related fields
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm
  • Able to communicate persuasively through speaking, writing, and client presentations
  • Able to consult, write, and present persuasively
  • Able to work in a self-organized and cross-functional team
  • Able to iterate based on new information, peer reviews, and feedback
  • Prior experience of working in a large media company would be added advantage
  • Skills Required

    Apache Airflow, Aws Redshift, snowflake , AWS Glue, Apache Spark, Databricks, Sql, Python

    Create a job alert for this search

    Senior Data Engineer • Pune, India

    Related jobs
    Senior Data Engineer

    Senior Data Engineer

    RapidBrains • Pune, IN
    Job Title : Senior Data Engineer.We are looking for a Senior Data Engineer with deep expertise in Azure Data Engineering to design, build, and optimize large-scale data pipelines.The ideal candidate...Show more
    Last updated: 16 days ago • Promoted
    AWS Cloud Engineer

    AWS Cloud Engineer

    Volkswagen Group Digital Solutions [India] • Pune, Maharashtra, India
    Join to Grow, Great work will eventually happen! #ShapingFutureTechnologists is our POV.If technology is your passion and innovation is your fuel, then this is your workplace.More than a Great Pl...Show more
    Last updated: 8 days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Capgemini Invent • Pune, Maharashtra, India
    At Capgemini Invent, we believe difference drives change.As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to...Show more
    Last updated: 10 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    RiDiK (a Subsidiary of CLPS. Nasdaq : CLPS) • Pune, India
    Role : Data Engineering Practice Leader.Availability : Immediate to 30 days.We're hiring a Data engineering Practice Leader to drive our data and AI initiatives. This senior role demands over 20 ...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    vConstruct Private Limited • Pune, Maharashtra, India
    Construct, a Pune-based Construction Technology company is seeking a Senior Data Engineer for its Data Science and Analytics team, a close-knit group of analysts and engineers supporting all data a...Show more
    Last updated: 30+ days ago • Promoted
    AWS Data Platform Engineer

    AWS Data Platform Engineer

    Persistent Systems • Pune, Maharashtra, India
    We’re looking for an AWS Data Platform Engineer to help automate and scale our cloud-based analytics environment.You’ll work with our BI and Data Engineering teams to build secure, automated, and r...Show more
    Last updated: 18 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Persistent Systems • Pune, Maharashtra, India
    Data Engineer with extensive experience in leading large-scale data migration projects, specializing in PySpark and Databricks. Proven ability to design and optimize high-performance data pipelines,...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    Tata Consultancy Services • Pune, Maharashtra, India
    TCS is hiring for AWS Data engineer, please find the below JD.Experience range – 6 to 8 years.Location - Hyderabad / Pune / Indore. Skills Required - AWS, Pyspark, EMR, Glue.AMI Rehydration, Cloud for...Show more
    Last updated: 30+ days ago • Promoted
    Senior AWS Data Engineer

    Senior AWS Data Engineer

    Epergne Solutions • Pune, Maharashtra, India
    Quick Apply
    Immediate / 10 Days To 15 days NP.The ideal candidate should have a solid background in data engineering, strong programming skills, and deep experience working with AWS cloud services.Optimize and...Show more
    Last updated: 30+ days ago
    Senior Data Engineer

    Senior Data Engineer

    SAIVA AI • Pune, IN
    We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
    Last updated: 30+ days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Tata Consultancy Services • Pune, Maharashtra, India
    Required Technical Skill Set AWS Redshift, Glue, PySpark.Desired Experience Range 4-10 Years.Location of Requirement Chennai / Pune. Desired Competencies (Technical / Behavioral Competency).Must-Have • ...Show more
    Last updated: 30+ days ago • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Vista Applied Solutions Group Inc • Pune, IN
    Job Summary for AWS Data Engineer : .Job Qualification and Responsibilities for AWS Data Engineer : .Advanced SQL (performance tuning, joins, windows functions). Hands-on with AWS : Glue, Lambda, Redshif...Show more
    Last updated: 11 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    EXL • Pune, Maharashtra, India
    Develop high quality, secure and scalable data pipelines using spark, Scala / Python / Java on Hadoop or object storage like MinIO. Leverage technologies and solutions to innovate with increasingly larg...Show more
    Last updated: 25 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    eClerx • Pune, Maharashtra, India
    We are looking for an experienced Senior Data Engineer of hands-on experience in designing, building, and managing complex data architectures and BI solutions. The ideal candidate will have deep exp...Show more
    Last updated: 16 days ago • Promoted
    Senior Data Engineer ( Palantir)

    Senior Data Engineer ( Palantir)

    Lear Corporation • Pune, Maharashtra, India
    Lear, a global automotive technology leader in Seating and E-Systems, is Making every drive better by delivering intelligent in-vehicle experiences for customers around the world.With over 100 year...Show more
    Last updated: 22 days ago • Promoted
    AWS / Azure Data Engineer

    AWS / Azure Data Engineer

    Axiom Software Solutions Limited • Pune, MH, IN
    Quick Apply
    Design, implement, and manage scalable ETL / ELT pipelines using AWS services and Databricks.Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data ...Show more
    Last updated: 30+ days ago
    AWS Data Engineer

    AWS Data Engineer

    Epergne Solutions • Pune, Maharashtra, India
    Quick Apply
    Job Roles & Responsibilities.Design, build, and optimize data pipelines and scalable data assets.Develop and maintain high-performance code using PySpark / Python with best practices.Perform code...Show more
    Last updated: 30+ days ago
    Senior Data Engineer

    Senior Data Engineer

    VRIZE • Pune, IN
    Design, architect, administer, and manage large database systems, particularly on Microsoft platforms such as SQL Server and Azure Database. Develop and optimize complex SQL queries for data extract...Show more
    Last updated: 4 days ago • Promoted