No longer accepting applications

Data Engineer (AWS MSK Specialist)

BPMLinksHyderabad, India

29 days ago

Job description

Job Title : Data Engineer - AWS MSK Specialist Job type : Full time hire

On-site work from office - Hitech City

Work timings : 3 PM / 4 PM - 11 PM / 12 AM (Flexible as per work requirement, according to US EST Timings)

Job Summary :

We're looking for a skilled Data Engineer with expertise in AWS Managed Streaming for Apache Kafka (MSK) to design, build, and maintain high-performance data pipelines. You'll work closely with data architects and software engineers to ensure seamless data integration and streaming capabilities using AWS MSK.

Responsibilities :

Design, develop, and manage real-time data pipelines using AWS MSK
Integrate AWS MSK with various data sources and consumers
Monitor and optimize AWS MSK cluster performance and costs
Collaborate with DevOps to ensure optimal performance and uptime
Implement security best practices for data streaming and storage
Document technical processes and best practices
Work with AWS services like S3, Glue, and Lambda for data processing and integration

Requirements :

Bachelor’s degree in computer science, Engineering, or a related field

3-5 years of experience working with AWS MSK or Apache Kafka in a production environment

Strong understanding of distributed systems, real-time data processing, and cloud-based data streaming

Proficiency in programming languages such as Python

Experience with AWS services like S3, Glue, and Lambda

Excellent problem-solving and debugging skills

Nice-to-Have Skills :

Experience with containerization technologies like Docker and Kubernetes

Familiarity with other messaging systems like RabbitMQ or Apache Pulsar

Experience with data serialization formats like Avro, Protobuf, or JSON

Knowledge of Snowflake or other cloud-based data warehousing solutions

Create a job alert for this search

Aws Data Engineer • Hyderabad, India