This job offer is not available in your country.

Senior Data Engineer- AWS

InfoCeptspune, maharashtra, in

10 days ago

Job description

Key Result Areas and Activities :

Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements.
Develop spark jobs using Scala for new project requirements.
Enhance existing spark jobs for any ongoing product enhancement.
Performance tuning of spark jobs, stress testing etc.
Create new data pipelines for developed / enhanced spark jobs using AWS Lambda or Apache Airflow
Responsible for database design process, logical design, physical design, star schema, snowflake schema etc.
Analyze data processing, integration, modelling and reporting requirements & Define data loading strategies considering volume, data types, frequency and analytics specifications.
Ensure optimal balance between cost and performance.
Project documentation, Adheres to Quality guidelines & Schedules.
Works hand in hand with PM for successful delivery of project and provide Estimation, scoping, scheduling assistance.
Manage build phase and quality assure code to ensure fulfilling requirements and adhering to Cloud architecture. Resolve difficult design and develop issues.

Work and Technical Experience :

Must-Have :

Overall, 7-9 years of IT Exp

5+ years of AWS related project

Good to have Associate Level and Professional Level AWS Certification

In depth knowledge of following AWS Services required : S3, EC2, EMR, Severless, Athena, AWS Glue, Lambda, Step Functions

Cloud Databases (Must have) – AWS Aurora, Singlestore, RedShift, Snowflake

Big Data (Must have) - Hadoop, Hive, Spark, YARN

Programming Language (Must have) – Scala, Python, Shell Scripts, PySpark

Operating System (Must have) - Any flavor of Linux, Windows

Must have very strong SQL Skills

Orchestration Tools (Must have) : Apache Airflow

Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.

Should have thorough conceptual understanding of AWS VPC, Subnets, Security Groups, & Route Tables

Should be a quick and self-learner and be ready to adapt to new AWS Services or new Big Data Technologies as and when required

Qualifications :

Bachelor’s degree in computer science, engineering, or related field (Master’s degree is a plus)

Demonstrated continued learning through one or more technical certifications or related methods

Minimum 5 years of experience on Cloud related projects

Qualities :

Hold strong technical knowledge and experience

Should have the capability to deep dive and research in various technical related fields

Self-motivated and focused on delivering outcomes for a fast-growing team and firm

Able to communicate persuasively through speaking, writing, and client presentations

Able to consult, write, and present persuasively

Able to work in a self-organized and cross-functional team

Able to iterate based on new information, peer reviews, and feedback

Prior experience of working in a large media company would be added advantage

Create a job alert for this search

Aws Data Engineer • pune, maharashtra, in

Related jobs

Promoted

Senior Data Engineer

Recropune, maharashtra, in

Responsible and accountable for development and maintenance of data pipelines.US for gathering requirements and provide efficient solutions to business requests by using. Design, build, and maintain...Show moreLast updated: 30+ days ago

Promoted

Senior Cloud Engineer AWS

Matrix USAPune, IN

We are seeking an experienced AWS Developer proficient in Python and PySpark to design, develop, and maintain scalable, serverless data processing and workflow automation solutions on AWS.The ideal...Show moreLast updated: 4 days ago

Promoted

Senior Data Engineer

DeltacubesPune, IN

Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 14 days ago

Promoted

Senior Data Engineer

Otomeyt AIpune, maharashtra, in

We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 17 days ago

Promoted

TCS Hiring for AWS Senior Data Engineer with Pyspark, AWS, Glue_Pune

Tata Consultancy ServicesPune, Maharashtra, India

Job Title : AWS Senior Data Engineer with Pyspark, AWS, Glue_Pune.PySpark, AWS[ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow] ,Qlik Replicate, Data Testing. Hadoop, Teradata Background, IaC[Clo...Show moreLast updated: 6 days ago

Promoted

Senior Cloud Platform Engineer -AWS-Salary 70LPA

The BigCjobs.comPune, IN

We are looking for a Senior Cloud Platform Engineer to lead the automation, reliability, and performance of our AWS-based infrastructure. You will architect, optimize, and scale mission-critical sys...Show moreLast updated: 5 days ago

Promoted

Senior Data Engineer (AWS cloud ecosystem) | Green Card holders or valid H1B | Jersey City, New Jersey,USA

Trupti Solutions - Indiapune, maharashtra, in

We are seeking Senior Data Engineers with 8–12 years of experience to join our client's team in Jersey City.This is a hybrid role that requires both strong technical expertise in cloud-based data e...Show moreLast updated: 4 days ago

Promoted

Senior Data Engineer

Infogainpune, maharashtra, in

Big Data Engineer (Lead) : As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers. You will work closely with clients and team members to understand their req...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer (AWS cloud ecosystem) | Green Card holders or valid H1B | Jersey City, New Jer

Trupti Solutions - IndiaPune, India

Promoted

Senior Big Data Engineer

VeltrisPune, IN

Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-gene...Show moreLast updated: 14 days ago

Promoted

Senior Data Engineer

CESPune, IN

As part of our data engineering team, you’ll build and maintain.This is a hands-on role where you’ll design and optimize modern data infrastructure, ensuring reliability, scalability, and performan...Show moreLast updated: 5 days ago

Promoted

Senior Data Engineer

Saras Analyticspune, maharashtra, in

We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value. Our suite of work spans extraction, transformatio...Show moreLast updated: 23 days ago

Promoted

Senior Data Engineer

SAIVA AIPune, IN

We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 7 days ago

Promoted

Senior Data Engineer

Manuh TechnologiesPune, IN

Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

Prudent Technologies and Consulting, Inc.pune, maharashtra, in

Snowflake Instance Management : Set up, configure, and manage between three to four new Snowflake data warehouse instances per quarter, tailored to support emerging products and analytics requiremen...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

BrightEdgePune, IN

BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of da...Show moreLast updated: 30+ days ago

Promoted

Senior Data Engineer

EXLPune, IN

We are seeking a highly skilled.You will work closely with cross-functional teams to provide thought leadership deliver robust, secure, and high-performance data solutions that empower analytics an...Show moreLast updated: 25 days ago

Promoted

Senior Data Engineer

Tredence Inc.pune, maharashtra, in

Design, build, and maintain scalable data pipelines using DBT and Airflow.Develop and optimize SQL queries and data models in Snowflake. Implement ETL / ELT workflows, ensuring data quality, performan...Show moreLast updated: 30+ days ago