Big Data Engineer

About the Company

Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.

Position : Data Engineer

Experience : 6+ Years

Location : Hyderabad

Employment Type : Full-time

About the Role

We are looking for a highly skilled Data Engineer with 6+ years of experience in designing and building scalable data pipelines, big data processing solutions, and cloud-based data platforms. The ideal candidate will have strong expertise in Big Data technologies, PySpark, Python, AWS, and Snowflake , and will work closely with data architects, analysts, and business stakeholders to deliver efficient, reliable, and high-performance data solutions.

Key Responsibilities

• Design, develop, and maintain scalable and high-performance ETL / ELT pipelines using PySpark, Python, and SQL.
• Build and optimize Big Data processing workflows using Spark, Hive, and other distributed data frameworks.
• Develop ETL / ELT frameworks and workflows on AWS cloud using services such as S3, Glue, EMR, Lambda, Step Functions, and Redshift.
• Implement and maintain enterprise-grade Snowflake data models, warehouses, data sharing, and performance tuning.
• Collaborate with data architects to design end-to-end data architecture, ensuring scalability, security, and reliability.
• Perform data ingestion, transformation, quality validation , and metadata management across various structured and unstructured data sources.
• Build CI / CD pipelines for data applications using tools like Git, Jenkins, or AWS CodePipeline.
• Monitor and optimize data pipelines for cost efficiency, performance, and reliability.
• Troubleshoot production issues and implement solutions for long-term stability and scalability.
• Ensure compliance with data governance, security best practices, and industry standards.

Required Skills & Qualifications

Nice-to-Have Skills