Senior Data Pipeline Engineer

Job Title : Senior Data Engineer

Job Summary

We are looking for a highly skilled Senior Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines and integration frameworks. The ideal candidate will have strong experience in batch and near real-time data processing within the AWS ecosystem , with hands-on exposure to Databricks, Spark, Flink, and Kafka . This role requires strong programming skills, advanced SQL knowledge, and the ability to architect scalable, high-performance data solutions to support analytics, AI / ML, and business intelligence initiatives.

Key Responsibilities

• Design, build, and maintain scalable data pipelines for batch and near real-time data processing.
• Develop and optimize ETL / ELT workflows using Spark, Flink, and Databricks .
• Work extensively within the AWS ecosystem (S3, Glue, EMR, Redshift, Lambda, etc.) to build reliable data solutions.
• Implement real-time data streaming and messaging solutions using Kafka .
• Write efficient, reusable, and scalable code in Scala and Python .
• Design, optimize, and maintain complex queries in SQL and SparkSQL .
• Collaborate with data scientists, analysts, and other stakeholders to ensure data availability, quality, and governance.
• Monitor, troubleshoot, and optimize data workflows for performance and cost-efficiency.
• Drive best practices in data engineering, automation, and DevOps for data platforms.

Required Skills & Qualifications

Preferred Skills