Data Engineer

We are looking for a Data Engineer with strong experience in CI / CD practices, Databricks (Spark), Python, Github and SQL. The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and working with modern DevOps / configuration management tools.

Key Responsibilities

• Design, implement, and manage CI / CD pipelines using GitHub Actions or equivalent tools.
• Configure and maintain GitHub branch protection rules and workflows to ensure smooth code integration and deployment.
• Build, optimize, and debug Spark jobs in Databricks for large-scale data processing.
• Automate Databricks deployments using Databricks Asset Bundles (DABs) or similar frameworks.
• Develop, test, and maintain robust data pipelines using Python and SQL .
• Implement unit testing, validation, and quality checks for data workflows.
• Manage multi-environment deployments (Dev, Stage, Prod) ensuring reliability and consistency.
• Write and maintain configuration files (YAML, JSON) for workflows, pipelines, and automation.
• Troubleshoot and resolve configuration or syntax issues in YAML and related files.
• Keep critical Databricks notebooks running reliably to ensure business continuity.
• Migrate legacy reports into modern, self-serve dashboards (Snowflake, Tableau, Power BI).
• Partner with cross-functional teams to understand needs and deliver data models, insights, and reporting they can trust.
• Put guardrails in place with data contracts, quality checks, and documentation to ensure long-term reliability.

What we’re looking for :