We are seeking a proactive Senior Databricks Engineer to design, implement, and optimize data processing pipelines on the Databricks platform. You will be responsible for architecting scalable data solutions, collaborating with cross-functional teams to migrate existing solutions, and ensuring the reliability and performance of our data infrastructure. This role requires deep expertise in Databricks and strong programming skills in Python, SQL, or Scala.
Roles & Responsibilities :
- Design and develop data processing pipelines and analytics solutions using Databricks .
- Architect scalable and efficient data models and storage solutions on the Databricks platform.
- Collaborate with architects and other teams to migrate current solutions to use Databricks.
- Optimize the performance and reliability of Databricks clusters and jobs to meet business requirements.
- Use best practices for data governance, security, and compliance on the Databricks platform.
- Mentor junior engineers and provide technical guidance.
- Stay current with emerging technologies and trends in data engineering and analytics to drive continuous improvement.
Skills Required :
Proficiency in programming languages such as Python, Scala, or SQL .Strong understanding of distributed computing principles and experience with big data technologies like Apache Spark .Experience with cloud platforms such as AWS, Azure, or GCP , and their associated data services.Proven track record of delivering scalable and reliable data solutions in a fast-paced environment.Excellent problem-solving skills and attention to detail.Strong communication and collaboration skills with the ability to work effectively in cross-functional teams.Experience with containerization technologies like Docker and Kubernetes is a plus.Knowledge of DevOps practices for automated deployment and monitoring of data pipelines is a plus.QUALIFICATION :
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.Skills Required
Python, Scala, Sql, Apache Spark, Aws, Azure, Docker