We are looking for a skilled Data Engineer with hands-on experience in Databricks to build and manage scalable, high-performance data pipelines. The ideal candidate will have strong expertise in big data technologies, cloud platforms, and workflow orchestration to support data science and analytics initiatives.
Responsibilities
- Design, develop, and optimize data pipelines on Databricks for ingestion, transformation, and delivery.
- Work with large datasets using Databricks, Spark, Hadoop, Hive, and SQL .
- Automate data workflows and orchestrations using Airflow .
- Ensure high data quality, reliability, and governance across platforms.
- Collaborate with data scientists, analysts, and software engineers to enable ML and analytics use cases.
- Monitor, troubleshoot, and improve data infrastructure performance.
Required Skills
4 -8 years of experience in Data Engineering .Mandatory hands-on experience with Databricks in production environments.Proficiency in Python, Spark, Hadoop, Hive, SQL .Experience with Airflow or similar orchestration tools.Strong knowledge of ETL / ELT pipelines , data modeling, and performance optimization.Cloud experience (AWS / Azure / GCP) with data engineering services.