Cloud Consultant

We need an experienced AWS Data Engineer / Performance Optimization Consultant to help us analyze, tune, and optimize our existing AWS Glue ETL, Step Functions workflows and Amazon Redshift data warehouse. The goal is to reduce job runtimes, improve query performance, and lower AWS operational costs.

Key Responsibilities :

Glue Job Optimization :

• Review and analyze existing AWS Glue jobs, including global / local variables, job parameters, and configurations.
• Evaluate Glue job scripts, reference framework Python modules, and orchestration via Step Functions.
• Consolidate multi-layered Glue jobs where feasible to improve efficiency and maintainability.
• Identify and resolve performance bottlenecks across the ETL pipeline.
• Optimize job configurations such as worker types, parallelism, partitioning strategies, Spark tuning, and incremental load logic.
• Review Spark UI and job metrics to identify opportunities for runtime and cost improvements.

Redshift Performance Tuning :

Data Pipeline & Cost Optimization :