Website- https : / / intraedge.com /
Job Title : Data Engineer
Experience : 5+Years
Location : [Remote]
Senior Data Engineer
The Senior Data Engineer will design, develop, monitor and maintain a robust and scalable data platform used by other data analyst and engineering teams to deliver powerful insights to both internal and external stakeholders
Responsibilities
- Design, build and maintain data infrastructure that powers both batch and realtime processing of billions of records a day.
- Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection
- Design, build and maintain a central data cataloging system to ease integration and discovery of datasets
- Develop data pipelines that provide fast, optimized, and robust end-to-end analytical solutions
- Automate manual processes and create a platform in favor of self-service data consumption
- Deploy and configure components to production environments.
- Participate in on-call schedule to provide emergency incident support
- Mentor and train teammates on design and operation of data platform
- Stay current with industry trends, making recommendations as needed to help the company excel
- Other job-related duties as assigned Skills,
Experience and Qualifications
Bachelor’s Degree in Computer Science or Engineering a plus5+ years of relevant industry experience in Data Engineering working with large scale data driven systemsDeep knowledge of dimensional modeling and designing schemas and data sets optimized for OLAP environmentExperience fine-tuning queries around large, complex data setsExtensive experience working with an MPP data platform (ie Snowflake, BigQuery, or Databricks)Experience designing and implementing workflows on a modern orchestration framework (ie Airflow, Prefect)Advanced Data Build Tool (DBT) experience (ie macro design, generalizing tests, using different incremental strategies)Deep understanding of SQL and data warehouse systems, especially Snowflake (SnowPro certification preferred)Experience working with and administering a BI tool (Tableau preferred)Expertise in object-oriented and / or functional programming languages (Python preferred)Strong programming skills, able to write modular, maintainable codeUnderstanding of DevOps principles such as automating of CI / CD pipelines and Infrastructure as codeUnderstanding of polyglot data persistence (relational, key / value, document, column)Excellent problem-solving skills and the ability to proactively solve issuesExcellent communication and organizational skills and proven ability to complete tasks and meet deadlinesAbility to be flexible with working in tandem with a team of engineers or alone, as required