Job Description
We are seeking a skilled Ops Data Engineer to maintain robust data infrastructure and pipelines that support our operational analytics and business intelligence needs.
Candidates will bridge the gap between data engineering and operations, ensuring reliable, scalable, and efficient data systems that enable data-driven decision making across the organization.
Primary Skills :
- Strong proficiency in Spark SQL
- Hands-on experience with Apache Kafka, Apache Flink, and other relevant streaming technologies
- Databases : Strong knowledge of relational databases (Oracle, MySQL) and NoSQL systems
- Proficiency with Version Control Git, CI / CD practices and collaborative development workflows
- Good understanding of orchestrating complex computational workflows and data processing pipelines - UC4, Apache Airflow or equivalent.
- Agile development to have :
- Proficiency in programming language
- Python / Java or equivalent
Key Responsibilities :
Design, develop, and maintain data pipelines and infrastructure to support operational analytics and BI needsImplement and optimize streaming data processing using technologies like Apache Kafka and Apache FlinkDevelop and maintain complex SQL queries and data transformations using Spark SQLManage and optimize data storage across relational databases (Oracle, MySQL) and NoSQL systemsUtilize version control systems like Git, and apply CI / CD best practices for deployment and collaborationOrchestrate and automate complex workflows and data processing pipelines using tools such as UC4, Apache Airflow, or Collaborate with cross-functional teams in an Agile development environment to deliver high-quality data solutions(ref : hirist.tech)