Key Responsibilities :
- Work with AWS services including S3, Lambda, Glue, API Gateway, and SQS to build and maintain data pipelines
- Develop and manage scalable data workflows using Python, PySpark, and SQL
- Handle batch job scheduling and manage data dependencies effectively
- Use tools like Apache Spark and Airflow for data orchestration and transformation
- Automate repetitive tasks and build reusable frameworks to improve operational efficiency
- Provide Run / DevOps support for ongoing data services and ensure high availability and performance
- Collaborate with cross-functional teams to deliver data solutions in a timely and efficient manner
Skills Required
Aws S3, AWS Glue, Api Gateway, AWS SQS, Python, Pyspark