Position Summary
We are seeking a highly skilled Senior Cloud Migration Engineer to lead the transformation of our EMR workloads from EC2-based infrastructure to EMR Serverless. This strategic role focuses on enhancing scalability, reducing operational overhead, and improving cost efficiency across our cloud-native data processing platform.
______________
Key Responsibilities
Migration Planning & Assessment
- Analyze existing EC2-based EMR jobs to assess their suitability for serverless migration
- Prioritize workloads based on business impact, complexity, and readiness
- Conduct deep code discovery and ensure alignment with production standards
- Audit and synchronize code repositories with live production environments
Technical Implementation
Transition Hadoop / HDFS workloads to Amazon S3-based architecturesRefactor data processing pipelines to leverage Amazon AthenaUpgrade and modernize runtime environments (e.g., Python versions)Ensure full dependency compatibility across target serverless stacksInfrastructure Modernization
Integrate EMR Serverless logging with AWS CloudWatchConfigure centralized logging via AWS Log GroupsOptimize workflows for observability and performance in a serverless modelMigrate legacy scheduling (e.g., Control-M, cron) to AWS Step FunctionsDevOps - Git Hub and Workflow ExperienceQuality Assurance & Deployment
Design and implement robust testing strategies across staging and QA environmentsDocument test plans, validation procedures, and migration outcomesCollaborate with the AMI and QA teams for pre-deployment sign-offsExecute controlled production rollouts with post-deployment monitoring______________
Required Qualifications
5+ years of experience with Amazon EMR and big data platformsDeep expertise in Apache Spark, Hadoop ecosystem, and AWS cloud servicesProficient in Python and version managementHands-on experience with AWS Step Functions, CloudWatch, and serverless solutionsFamiliarity with job schedulers like Control-M or cronStrong knowledge of Amazon S3, Athena, and data lake architectures______________
Preferred Qualifications
AWS Certifications (Solutions Architect, DevOps Engineer, or Data Analytics)Experience with Infrastructure as Code (IaC) tools such as CloudFormation or TerraformProven success in enterprise-scale cloud / data migration projectsExperience working in Agile environments using JIRA or similar tools______________
Success Metrics
Full migration of selected EMR jobs to EMR ServerlessMeet or exceed existing EC2-based performance benchmarksDemonstrable cost savings post-migrationZero data loss throughout the migration processSuccessful completion of quality checks and stakeholder approvalsSkills Required
Cron, Hadoop, Cloudformation, Amazon S3, Apache Spark, AWS CloudWatch, Jira, amazon emr , Terraform, Control-M, Python