AWS Data Engineer

Description

Roles and responsibilities

Design AWS architectures based on business requirements.

Create architectural diagrams and documentation.

Present cloud solutions to stakeholders.

Experience : 5-10 yrs

Skills and Qualifications :

Design, develop, and maintain scalable ETL / ELT pipelines using AWS services like Glue, Lambda, and Step Functions.

Work with batch and real-time data processing using AWS Glue, Kinesis, Kafka, or Apache Spark.

Optimize data pipelines for performance, scalability, and cost-effectiveness.

Identify bottlenecks and optimize query performance on Redshift, Athena, and Glue.

Strong knowledge of AWS services : EC2, S3, RDS, Lambda, IAM, VPC, CloudFormation, CloudWatch, etc.

Experience with serverless architectures (AWS Lambda, API Gateway, Step Functions).

Experience of AWS networking (VPC, Route 53, ELB, Security Groups, etc.).

Experience with AWS CloudFormation for automating infrastructure.

Proficiency in scripting languages such as Python or Bash.

Experience with automation tools (AWS Systems Manager, AWS Lambda)

Experience of containerization (Docker, Kubernetes, AWS ECS, EKS, Fargate).

Experience with AWS CloudWatch, AWS X-Ray, ELK Stack, or third-party monitoring tools.

Experience with AWS database services (RDS, DynamoDB, Aurora, Redshift).

Experience of storage solutions (S3, EBS, EFS, Glacier).

Experience of AWS Direct Connect, Transit Gateway, and VPN solutions.