Description :
We are seeking a Databricks Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions on the Databricks Lakehouse Platform. You will be responsible for transforming raw data into high-value datasets that power analytics, AI / ML models, and business insights. We're looking for a dynamic, self-motivated, and proactive leader who thrives in a fast-paced environment and can foster a high-performing :
- Develop scalable ETL / ELT pipelines using PySpark, SQL, and Delta Lake.
- Build and maintain batch and stream data pipelines using Databricks Workflows.
- Design and implement Lakehouse architecture and optimize Delta Lake tables.
- Integrate data from multiple sources, including APIs, databases, cloud storage, and third-
party platforms.
Ensure data quality and reliability using Delta Live Tables, expectations, and monitoring tools.Work with CI / CD pipelines using Git, Databricks Repos, and DevOps tools.Implement governance, security, and compliance standards (Unity Catalog, lineage, RBAC).Requirements :
Strong knowledge of data modeling, data warehousing concepts, and Lakehouse Strong experience with Databricks, PySpark, Apache Spark, and SQL.Hands-on with Delta Lake, Databricks Workflows, Unity Catalog.Experience building scalable pipelines on Azure, AWS, or GCP.Solid understanding of distributed systems and performance optimization.Experience with streaming frameworks (Structured Streaming, Kafka, Auto Loader).Passion for problem-solving, both technical and User-centric mindset focused on building delightful software experiencesLead by example : implement best practices and maintain high-quality code standardsNice to have skills :
Databricks certification (Data Engineer Associate / Professional).Experience with ML workflows and feature engineering.Hands-on with Databricks Photon, serverless SQL, and optimization techniques.Familiarity with BI tools (Power BI, Tableau, Looker).(ref : hirist.tech)