Data Engineer - Python / SQL

Job Title : Data Engineer

About the Role :

We are seeking a highly skilled Data Engineer to join our team. The ideal candidate will have expertise in Databricks, Python, and SQL.

Key Responsibilities :

• Design, develop, and maintain scalable ETL / ELT data pipelines using Databricks (PySpark) on Azure / AWS / GCP.
• Develop clean, reusable, and performant Python code for data ingestion, transformation, and quality checks.
• Write efficient and optimized SQL queries for querying structured and semi-structured data.
• Work with stakeholders to understand data requirements and implement end-to-end data workflows.
• Perform data profiling, validation, and ensure data quality and integrity.
• Optimize data pipelines for performance, reliability, and integrate data from various sources APIs, flat files, databases, cloud storage e.g., S3, ADLS.
• Build and maintain delta tables using Delta Lake format for ACID-compliant streaming and batch pipelines.
• Work with Databricks Workflows to orchestrate pipelines and scheduled jobs.
• Collaborate with DevOps and cloud teams to ensure secure, scalable, and compliant infrastructure.

Technical Skills Required :

Core Technologies :

ETL & Orchestration :

Cloud Platforms Any one or more :

Data Modeling & Storage :

Monitoring & Version Control :

Optional / Preferred :

Soft Skills :

(ref : hirist.tech)