Senior Data Engineer - Data Warehousing

Job Description :

As a Data Engineer in the Data Warehouse team you will be relied on to independently develop and deliver high-quality features and finish tasks to a high standard. Youll be part of an emerging team, focussing on building and improving our innovative BNL Data Warehouse offerings applying marketing suites.

What we are looking for :

For this position the ideal candidate has 6+ years of experience in software development, and with data structures / algorithms with a focus on backend and data engineering.

Level of skills and experience :

• 5 years of hands-on experience in using Python, Pyspark and Shell Scripting with production experience.
• Should have experience in streaming (FLink / NIFI), data governance, security, quality checks.
• 3 years of proven experience on building cloud native data intensive applications, Amazon Web Service experience is a must.
• Experience on Data Warehouse design and development per the business nature using any of the modern data warehouses.
• Having experience in Databricks and dbt experience is required.
• Highly proficient in architecting and implementing workflow management systems, Airflow is a must.
• Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).
• Hands-on experience of building and maintaining Spark applications.
• Fundamental understanding of Parquet, Delta Lake and other OTFs file formats.
• Proficiency on an IaC tool such as Terraform, CDK or CloudFormation is a must.
• Experience with observability systems (such as Datadog or Prometheus / Grafana)

ref : hirist.tech)