Key Responsibilities
- Work extensively with AWS services including S3, Lambda, Glue, API Gateway, and SQS.
- Design and implement data engineering solutions on AWS, utilizing Python, PySpark, and SQL for processing and transformation.
- Manage batch job scheduling and handle data dependencies to ensure timely and accurate data delivery.
- Utilize data processing tools such as Spark and Airflow for efficient workflow orchestration.
- Automate repetitive tasks and build reusable frameworks to enhance operational efficiency.
- Provide Run / DevOps support and manage the ongoing operation and maintenance of data services.
- Collaborate with cross-functional teams to support data needs and improve overall data infrastructure.
Skills Required
Aws S3, Aws Lambda, AWS Glue, Api Gateway, AWS SQS, Python