Apps Associates - AWS Data Engineer - ETL

Key Responsibilities :

• ETL Development & Support : Lead enhancements, new development, defect resolution, and provide production support for ETL development using AWS native services.
• Data Integration : Design and implement robust data integration solutions using AWS services such as Glue and Lambda functions.
• Data Orchestration & Alerting : Utilize AWS SNS to send critical emails and alerts related to data processes and system health.
• ETL Process Authoring : Author, optimize, and maintain complex ETL processes using Python and PySpark.
• Monitoring & Validation : Implement and manage ETL process monitoring using CloudWatch events. Connect with different data sources like S3 and validate data using Athena.
• SQL Expertise : Apply extensive working experience with Advanced SQL and a complex understanding of SQL for data manipulation and analysis.
• DevOps Integration : Contribute to CI / CD practices using GitHub Actions for automated deployments of data pipelines.
• Agile Collaboration : Work proficiently within an Agile methodology framework, collaborating with cross-functional & Competencies :
• Deep technical skills in AWS Glue (Crawler, Data Catalog) : 5 years of hands-on experience.
• Hands-on experience with Python and PySpark : 3 years of experience.
• PL / SQL experience : 3 years of experience.
• CloudFormation and Terraform : 2 years of experience with Infrastructure as Code.
• CI / CD GitHub Actions : 1 year of experience.
• Experience with BI systems (PowerBI, Tableau) : 1 year of experience is beneficial.
• Good understanding of AWS services like S3, SNS, Secret Manager, Athena, and Lambda : 2 years of experience.
• Extensive working experience with Advanced SQL and a complex understanding of SQL.
• Proficiency in Agile methodology.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.

ref : hirist.tech)