Refine to Perfection : Data Engineer Lead
We're on a mission to excel in data innovation, driven by intelligent, impactful and futuristic business outcomes using cutting-edge technology. Join our pursuit of excellence by driving solutions that empower better decision making.
The Lead Data Engineer will collaborate with cross-functional teams to build scalable and optimized data solutions, leveraging cloud technologies like EMR, Lambda, Cloud Storage, BigQuery etc. We're seeking an expert who can translate business requirements into data services to solve complex problems.
- Translate business needs into actionable data strategies
- Develop and manage data pipelines (ETL / ELT jobs) and retrieve datasets for specific use cases using cloud platforms and tools
- Explore new technologies to design complex data modeling scenarios and provide optimal data engineering solutions
- Build integration layers to connect heterogeneous sources using various approaches
- Understand data and metadata to support consistency of information retrieval, combination, analysis and reporting
- Troubleshoot and monitor data pipelines for high availability of the reporting layer
- Collaborate with engineering and business teams to propose ways to improve platform and tools
- Effectively manage workstreams for self and 2-3 analysts to support delivery
Requirements
Bachelor's degree in Computer Science or related field7-10 years of experience building data pipelines or data ingestion for Batch / Streaming data from different sources to a data warehouse / data lakeExperience leading and delivering data warehousing and analytics projects using cloud technologiesHands-on experience / knowledge with SQL / Python / Java / Scala programming, understanding of SQL is essentialExperience with any cloud computing platforms like AWS / GCP / Azure etc.Knowledge of version control software like Git and experience in working with relevant hosting servicesStrong experience working with Python & R packages and ecosystemsExtensive experience in developing complex stored procedures and transformationsExperience / knowledge with Big Data tools like Hadoop / Hive / Spark / Presto