The Data Engineer is responsible for overseeing data engineering activities and aiding in building the business’ data collection systems and processing pipelines.
Responsible for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department.
Requirements :
Minimum 2 years of Data Engineering experience.
Proven expertise in Spark, Python, SQL, Kafka & Spark Streaming.
Have worked on multiple TBs / PBs of data volume from ingestion to consumption.
Experience with the complete software development life cycle including requirement analysis, design, development, deployment, and support.