Optimum InfoSystem - Azure Databricks Engineer - CDC Implementation

Job Summary :

We are seeking a skilled and experienced Azure Databricks Engineer to join our growing data engineering team. The ideal candidate will have deep hands-on expertise in building scalable data pipelines and streaming architectures using Azure-native technologies. Prior experience in the banking or financial services domain is highly desirable, as you will be working with critical data assets and supporting regulatory, risk, and operational reporting use cases.

Key Responsibilities :

• Design, develop, and optimize data pipelines using Databricks (PySpark) for batch and real-time data processing.
• Implement CDC (Change Data Capture) and Delta Live Tables / Autoloader to support near-real-time ingestion.
• Integrate various structured and semi-structured data sources using ADF, ADLS, and Kafka (Confluent).
• Develop CI / CD pipelines for data engineering workflows using GitHub Actions or Azure DevOps.
• Write efficient and reusable SQL and Python code for data transformations and validations.
• Ensure data quality, lineage, governance, and security across all ingestion and transformation layers.
• Collaborate closely with business analysts, data scientists, and data stewards to support use cases in risk, finance, compliance, and operations.
• Participate in code reviews, architectural discussions, and documentation efforts.

Required Skills & Qualifications :

Preferred Qualifications :

(ref : hirist.tech)