Senior data engineer- awsInfoCepts • Pune, Maharashtra, India
No longer accepting applications
Senior data engineer- aws
InfoCepts • Pune, Maharashtra, India
30+ days ago
Job description
Key Result Areas and Activities :
Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect
Should be able to assist the architect in reasoning about various architecture choices in terms of cost, performance and durability.
Should be able to suggest optimizations in existing designs.
Project documentation adheres to quality guidelines & schedules.
Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.
Articulate design decisions very clearly with stakeholders.
Perform Proof-of-Concepts and document all observations before proposing a new solution.
Conduct design review sessions with other teams and identify scope of improvement.
Suggests automation to improve existing processes.
Work and Technical Experience :
Must-Have :
In-depth knowledge of the following AWS services : S3, EC2, EMR, Athena, AWS Glue, Lambda
Experience with at least one MPP database : AWS Redshift, Snowflake, Single Store
Proficiency in Big Data technologies : Apache Spark, Databricks
Must have strong programming skills in Python
Responsible for building data pipelines in AWS And Databricks
Experience with Big Data table formats, such as Delta Lake (open source)
Must have very strong SQL skills
Experience with orchestration tools like Apache Airflow
Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.
Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required
Strong understanding of data warehousing concepts
Good to have :
Cloud Databases – Snowflake, AWS Aurora
Big Data – Hadoop, Hive
Cloud Databases– AWS Aurora
Associate Level or Professional Level AWS Certification or Databricks Certification
Qualifications :
Overall, 7+ years of IT Exp
5+ years of relevant experience in AWS related project
Bachelor’s degree in computer science, engineering, or related field (master's degree is a plus)
Demonstrated continued learning through one or more technical certifications or related methods.
Qualities :
Hold strong technical knowledge and experience
Should have the capability to deep dive and research in various technical related fields
Self-motivated and focused on delivering outcomes for a fast-growing team and firm
Able to communicate persuasively through speaking, writing, and client presentations
Able to consult, write, and present persuasively
Able to work in a self-organized and cross-functional team
Able to iterate based on new information, peer reviews, and feedback
Prior experience of working in a large media company would be added advantage
Create a job alert for this search
Data Engineer Aws • Pune, Maharashtra, India
Related jobs
Data Engineer
Quess IT Staffing • Pune, IN
Lead the design and implementation of scalable data pipelines for ingesting, transforming, and activating customer data.Develop and orchestrate workflows using Apache Airflow and Spark.S3, Glue, At...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
IntraEdge • Pune, IN
We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team.
You will be responsible for building scalable and reli...Show more
Last updated: 30+ days ago • Promoted
AWS Data Engineer
Capgemini Invent • Pune, Maharashtra, India
At Capgemini Invent, we believe difference drives change.As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to...Show more
Last updated: 14 days ago • Promoted
Senior Data Engineer
Syngenta Group • Pune, Maharashtra, India
The Senior Data Engineer plays a pivotal role in driving the design development and optimization of robust scalable and high-performance data products and pipelines on Databricks (Spark / Delta Lake / ...Show more
Last updated: 28 days ago • Promoted
Data Engineer
TerraGiG • Pune, IN
Lead the design, development, and implementation of data solutions using AWS and Snowflake.Collaborate with cross-functional teams to understand business requirements and translate them into techni...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Live Connections • Pune, IN
Hiring : Data Engineer | 9-12 Years Experience | Remote (India).Python, SQL, Big Data, Cloud, Snowflake.We are looking for an experienced.
The ideal candidate will have deep experience in.Python, SQL...Show more
Last updated: 30+ days ago • Promoted
Data Engineer
Tata Consultancy Services • Pune, IN
TCS has been a great pioneer in feeding the fire of Techies like you.We are a global leader in the technology arena and there’s nothing that can stop us from growing together.Your role is of key im...Show more
Last updated: 30+ days ago • Promoted
Data Analytics Engineer
Cube Consultancy Services • Pune, IN
We are seeking a skilled and motivated Data Analytics Engineer to join our growing data.This role is ideal for someone with a strong foundation in SQL, experience using Sigma.Computing, and a deep ...Show more
Last updated: 7 hours ago • Promoted • New!
AWS Engineer
Spryc Systems • Pune, IN
We are seeking an experienced AWS Engineer to design, implement, and maintain AWS infrastructure and services in a managed service environment.
The ideal candidate will possess deep expertise in AWS...Show more
Last updated: 30+ days ago • Promoted
Senior AWS Data Engineer
Epergne Solutions • Pune, Maharashtra, India
Quick Apply
Immediate / 10 Days To 15 days NP.The ideal candidate should have a solid background in data engineering, strong programming skills, and deep experience working with AWS cloud services.Optimize and...Show more
Last updated: 30+ days ago
Senior Data Engineer
Globant • Pune, Maharashtra, India
At Globant, we are working to make the world a better place, one step at a time.We enhance business development and enterprise solutions to prepare them for a digital future.With a diverse and tale...Show more
Last updated: 2 days ago • Promoted
AWS Data Architect
ACL Digital • Pune, IN
AWS (S3, Redshift, Glue, Lake Formation, IAM).Proficient in data modeling, performance tuning, and security best practices.
.AWS Certified Solutions Architect preferred.Show more
Last updated: 20 days ago • Promoted
Senior Data Engineer
EXL • Pune, Maharashtra, India
Develop high quality, secure and scalable data pipelines using spark, Scala / Python / Java on Hadoop or object storage like MinIO.
Leverage technologies and solutions to innovate with increasingly larg...Show more
Last updated: 29 days ago • Promoted
Senior Data Engineer
SAIVA AI • Pune, Maharashtra, India
We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
Persistent Systems • Pune, Maharashtra, India
About Position : We are looking for an experienced Azure Databricks and Snowflake Data Engineer to lead and execute complex user migrations from legacy systems to modern cloud-based solutions.This ...Show more
Last updated: 30+ days ago • Promoted
AWS / Azure Data Engineer
Axiom Software Solutions Limited • Pune, MH, IN
Quick Apply
Design, implement, and manage scalable ETL / ELT pipelines using AWS services and Databricks.Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data ...Show more
Last updated: 30+ days ago
AWS Data Engineer
Epergne Solutions • Pune, Maharashtra, India
Quick Apply
Job Roles & Responsibilities.Design, build, and optimize data pipelines and scalable data assets.Develop and maintain high-performance code using PySpark / Python with best practices.Perform code...Show more
Last updated: 30+ days ago
Data Engineer - Fully Remote (Global Data Platform & Analytics Projects)
SkillsCapital • Pune, IN
Remote
These fully remote, long-term freelance roles are ideal for engineers who can build scalable data pipelines, work with modern cloud-native data stacks, and support large-scale enterprise data initi...Show more