Responsibilities :
As a Data Engineer, you will be responsible implementing complex data pipelines and analytics
solutions to support key decision-making business processes in our client s domain. You will
gain exposure to a project that is leveraging cutting edge AWS technology that applies Big Data
and Machine Learning to solve new and emerging problems for our clients. You will gain a
added advantage of working very closely with AWS Professional Services teams directly
executing within AWS Services and Technologies to solve complex and challenging business
problems for Enterprises.
Key responsibilities include :
- Work closely with Product Owners and AWS Professional Service Architects to understand
requirements, formulate solutions, and implement them.
Implement scalable data transformation pipelines as per designImplement Data model and Data Architecture as per laid out design.Evaluate new capabilities of AWS analytics services, develop prototypes, and assist in drawingPOVs , participate in design discussionsRequirements
Minimum 3 years experience implementing transformation and loading of data from awide variety of traditional and non-traditional sources such as structured, unstructured,and semi structured using SQL, NoSQL and data pipelines for real-time, streaming,batch and on-demand workloadsAt least 2 years implementing solutions using AWS services such as Lambda, AWSAthena and Glue AWS S3, Redshift, Kinesis, Lambda, Apache Spark,Experience working with data warehousing data lakes or Lakehouse concepts on AWSExperience implementing batch processing using AWS Glue / Lake formation, DataPipeline
Experience in EMR / MSKExperience or Exposure to AWS Dynamo DB will be a plusDevelop object-oriented code using Python, besides PySpark, SQL and one otherlanguages (Java or Scala would be preferred)Experience on Streaming technologies both OnPrem / Cloud such as consuming andproducing from Kafka, KinesisExperience building pipelines and orchestration of workflows in an enterpriseenvironment using Apache Airflow / Control MExperience implementing Redshift on AWS or any one of Databricks on AWS, or Snowflake on AWSGood understanding of Dimensional Data Modelling will be a plus.Ability to multi-task and prioritize deadlines as needed to deliver resultsAbility to work independently or as part of a teamExcellent verbal and written communication skills with great attention to detail and accuracyExperience working in an Agile / Scrum environmentSkills Required
Nosql, snowflake , Py Spark, Databricks, LAMDA, Sql, Python, Aws