Senior Data Engineers - AWS

Description GSPANN is hiring Senior Data Engineers with expertise in Amazon Web Services (AWS) to design, build, and optimize scalable data pipelines and architectures on cloud platforms. Candidates must be proficient in Python, Hive, and Airflow, with strong problem-solving and data engineering expertise.

Role and Responsibilities

• Actively contribute to all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support.
• Solve complex business problems by applying a disciplined development methodology.
• Build scalable, flexible, efficient, and supportable solutions using appropriate technologies.
• Analyze source and target system data, and map transformations that meet business requirements.
• Collaborate with clients and onsite coordinators throughout project phases.
• Design and implement product features in partnership with business and technology stakeholders.
• Anticipate, identify, and resolve issues related to data management to improve data quality.
• Clean, prepare, and optimize large-scale data for ingestion and consumption.
• Support new data management initiatives and restructure existing data architectures as needed.
• Implement automated workflows and routines using workflow scheduling tools such as Apache Airflow.
• Apply continuous integration, test-driven development, and production deployment frameworks.
• Review and contribute to design, code, test plans, and dataset implementations by other data engineers, ensuring adherence to standards.
• Analyze and profile data to design scalable and reliable solutions.
• Troubleshoot data issues, perform root cause analysis, and proactively resolve product-related problems.

Skills and Experience