Data Engineer - ETL

Description :

Key Responsibilities :

• Design, build, and optimize data pipelines and ETL processes using AWS Glue, Lambda, RDS, and S3.
• Develop scalable batch and streaming data processing workflows using Apache Spark, Spark Streaming, and Kafka.
• Work with SQL and NoSQL databases (MySQL, PostgreSQL, Elasticsearch) to design schemas and optimize queries for high performance.
• Write efficient and optimized SQL for data extraction, transformation, and analytics.
• Build and maintain data ingestion, transformation, and integration solutions across multiple systems.
• Collaborate with cross-functional teams (data science, analytics, engineering) to provide reliable, high-quality data.
• Use Scala, Java, and Python for data processing and automation scripts.
• Implement monitoring, logging, and alerting for data pipelines using AWS services (CloudWatch, CloudTrail).
• Ensure data security, compliance, and governance using AWS best practices.

Required Skills & Experience :

(ref : hirist.tech)