Data Engineer

Job title: Senior Data Engineer

Experience:3+yrs

Location: Hyderabad

Work type: Hybrid

Roles & Responsibilities:

• At least 3+yrs years of experience in application design, development and analysis

• Experience in AWS Cloud Solutions & GCP. Retail Industry experience preferred.

• Implement data transformations and data structures for data warehouse and lake/repository.

• Manage cloud and/or on-premises solutions for data transfer and storage.

• Process data using spark (PySpark)

• Manage cloud and/or on-premises solutions for data transfer and storage.

• Collaborate and work with data analysts in various functions to ensure that data meets their reporting and analysis needs.

• Experience in creating Various ETL Frameworks in processing or extracting the data from cloud databases by using AWS Services by leveraging Lambda, Glue, PySpark, Step Functions, SNS, SQS and Batch.

• Proven ability to be a strategic thinker to drive the necessary ownership and data governance within the organization

• Ability to evaluate risks and provide business recommendations / solutions in a timely manner

• Ability to remain calm in high stress situations while navigating and leading during times of conflict

• Should have sound knowledge in various AWS Services

• Able to understand the existing Frameworks built in AWS PySpark and Glue

• Able to scan ETL Frameworks and propose optimizations and cost savings.

• Able to work independently and able to take key decisions.

• Comfortable interacting with stakeholders at all levels through Business and IT.

• Ability to express complex concepts effectively, both verbally and in writing to IT & Business partners

• Understanding of and experience working in an Agile environment

• Knowledge on Talend Cloud ETL,Kafka, Snowflake Cloud and PowerBI is addon

• Independent. Strong critical thinking, decision making, troubleshooting and problem-solving skills.

Skills Required
glue , Spark, Kafka, Powerbi, Gcp, Sns, Pyspark, Lambda, Batch, Sqs, Aws Services, Step Functions