As a Principal Software Engineer in our Data Platform infrastructure team, you'll have a key role in building and designing the strategy of our Enterprise Data Engineering group.
Responsibilities :
- Design and build a highly scalable data platform to support data pipelines for diversified and complex data flows.
- Track and identify relevant new technologies in the market and push their implementation into our pipelines through research and POC activities.
- Deliver scalable, reliable, and reusable data solutions.
- Leading, building, and continuously improving our data gathering, modeling, reporting capabilities, and self-service data platforms.
- Working closely with Data Engineers, Data Analysts, Data Scientists, Product Owners, and Domain Experts to identify data needs.
- Develop processes and tools to monitor, analyze, maintain, and improve data operations, performance, and usability.
Requirements :
Relevant Bachelor's degree or other equivalent Software Engineering background.12+ years of experience as an infrastructure / data platform / big data software engineer.Experience with AWS / GCP cloud services such as GCS / S3 Lambda / Cloud Function,EMR / Dataproc, Glue / Dataflow, Athena.
IaC design and hands-on experience.Familiarity with designing CI / CD pipelines with Jenkins, Github Actions, or similar tools.Experience in designing, building, and maintaining enterprise systems in a big data environment on the public cloud.Strong SQL abilities and hands-on experience with SQL, performing analysis and performance optimizations.Hands-on experience in Python or an equivalent programming language.Experience with administering data warehouse solutions (like BigQuery / Redshift / Snowflake).Experience with data modeling, data catalog concepts, data formats, data pipelines / ETLdesign, implementation, and maintenance.
Experience with Airflow and DBT - advantageExperience with Kubernetes using GKE or EKS - advantage.Experience with development practices - Agile, TDD - an advantage.Mandate Skills : SQL, CICD, Terraform, IAC, Python, GCP / AWS, Jenkins, Airflow.(ref : hirist.tech)