Senior Data Engineer

Location : Bengaluru, Karnataka, India

About The Role

We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.

What You'll Do

• Design and build efficient data pipelines using Spark / PySpark / Scala.
• Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs).
• Clean, transform, and prepare data for analysis.
• Use Python for data tasks, automation, and building tools.
• Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure.
• Collaborate closely with the Analytics team to understand what data they need and provide solutions.
• Help develop and maintain our Node.js backend, using Typescript, for data services.
• Use YAML to manage the settings for our data tools.
• Set up and manage automated deployment processes (CI / CD) using GitHub Actions.
• Monitor and fix problems in our data pipelines to keep them running smoothly.
• Implement checks to ensure our data is accurate and consistent.
• Help design and build data warehouses and data lakes.
• Use SQL extensively to query and work with data in different systems.
• Work with streaming data using technologies like Kafka for real-time data processing.
• Stay updated on the latest data engineering technologies.
• Guide and mentor junior data engineers.
• Help create data management rules and procedures.

What You'll Need

Bonus Points