Big Data Engineer

About the CompanyOur client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.Title : Senior Data EngineerLocation : Hyderabad,ChennaiExperience : 6 - 12 yearsEmployment Type : PermanentNotice Period : Immediate JoinersKey Responsibilities : Big Data Processing : Design, develop, and maintain scalable ETL / ELT pipelines using PySpark, Scala, or Java.Implement data transformation workflows to ensure clean and reliable datasets.Cloud Integration : Work extensively with AWS services such as EMR, S3, Glue, Lambda, Redshift, DynamoDB, and RDS to build and manage data solutions.Data Modeling & Optimization : Develop and optimize data models in Hadoop and Spark environments.Ensure performance tuning and cost optimization of cloud-based data platforms.Data Ingestion & Integration : Ingest data from various sources including APIs, RDBMS, and NoSQL databases.Implement data pipelines for batch and real-time processing.Collaboration & Documentation : Collaborate with cross-functional teams to understand data requirements and deliver solutions.Document data models, transformation logic, and technical specifications.Automation & Monitoring : Automate data workflows and ensure data pipelines are reliable, scalable, and error-free.Implement data monitoring, logging, and alerting to ensure pipeline reliability.Required Skills & Qualifications : Technical Expertise : 6–12 years of experience in Big Data engineering with strong hands-on experience in AWS, PySpark, Scala, or Java.Proficiency in SQL and experience with Hadoop ecosystem components like Hive, HDFS, and Kafka.Cloud Services : Extensive experience with AWS services such as EMR, S3, Glue, Lambda, Redshift, DynamoDB, and RDS.Data Engineering Tools : Familiarity with data orchestration tools like Apache Airflow or Oozie.Experience with containerization technologies like Docker and Kubernetes is a plus.Soft Skills : Strong analytical and problem-solving skills.Excellent communication and collaboration abilities.Ability to work independently and in a team-oriented environment.Preferred Skills (Good to Have) : Certifications : AWS Certified Solutions Architect, AWS Certified Big Data – Specialty, or Cloudera Certified Associate (CCA).Experience with CI / CD pipelines using tools like Jenkins or GitLab.Knowledge of data warehousing concepts and experience with platforms like Redshift or Snowflake.Familiarity with streaming technologies such as Apache Kafka or Kinesis.