Senior Engineer - Cloud Migration

Nextpathway | The Automated Cloud Migration Company

Next Pathway is a technology services company providing clients a pathway from existing to emerging technologies. Our automation technology helps our customers accelerate the migration of complex applications and workloads to the cloud.

About the Role :

Next Pathway is looking for a highly skilled and experienced Senior Engineer to join our team, specializing in large-scale cloud migration projects. The ideal candidate will have a minimum of 10 years of experience in data engineering and ETL, with a strong background in migrating on-premise data systems to the cloud. This role requires a hands-on professional who can design, develop, and implement complex data pipelines, working independently and as a key contributor within a project team.

Key Responsibilities :

Development and Implementation : Design, develop, and deploy robust and scalable data pipelines for migrating data from various on-premise sources to cloud platforms.

Technical Expertise : Serve as a subject matter expert in the project's core technologies, including Datastage, Azure Data Factory (ADF), Databricks, and PySpark.

Solution Design : Collaborate with architects and technical leads to design efficient and effective cloud migration strategies and data transformation solutions.

Code Quality : Ensure the delivery of high-quality, well-documented, and maintainable code. Participate in code reviews and provide constructive feedback to peers.

Performance Optimization : Analyze and optimize the performance of data pipelines and applications to ensure efficiency and scalability.

Problem-Solving : Troubleshoot and resolve complex technical issues related to data migration, data quality, and system performance.

Collaboration : Work closely with cross-functional teams, including business analysts, data scientists, and other engineers, to understand requirements and deliver successful outcomes.

Mentorship : Provide guidance and technical support to junior engineers, helping them grow their skills and contribute effectively to the team.

Required Skills and Qualifications :

Minimum of 10 years of experience in the IT industry, with a focus on data engineering and ETL development.

Extensive hands-on experience with Datastage for building and managing ETL jobs.

Proven experience in Cloud Migration projects , with a strong understanding of the challenges and best practices involved in moving data to the cloud.

In-depth knowledge and hands-on experience with Azure Data Factory (ADF) for building and orchestrating data workflows.

Strong proficiency in Databricks and its ecosystem, including notebooks, jobs, and Delta Lake.

Expertise in PySpark for developing data processing applications on a distributed computing framework.

Solid understanding of cloud platforms, preferably Microsoft Azure .

Experience with SQL and database concepts.

Excellent analytical, problem-solving, and communication skills.

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.