Databricks Engineer

Roles & Responsibilities

• Bachelors or Master's degree with 2-6 years of professional experience.
• Proven experience in implementing data solutions on the Databricks platform.
• Solid understanding of Databricks fundamentals and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse).
• Configure, set up, and manage Databricks clusters, workspaces, and notebooks to ensure optimal performance, scalability, and resource allocation.
• Implement data pipelines for ingesting, cleansing, and transforming data from various sources into Databricks, ensuring data quality, consistency, and reliability. Develop ETL (Extract, Transform, Load) processes as needed.
• Develop, optimize, and maintain Spark-based ETL workflows and data pipelines for data preparation and transformation within Databricks.
• Knowledge of Performance Optimization, Monitoring and Automation would be a plus.
• Understanding of data governance, compliance, and security best practices.
• Strong Proficiency in Pyspark, Databricks Notebooks, SQL, Python, Scala, or Java.
• Proficiency in SQL for data querying and manipulation.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Strong problem-solving and troubleshooting skills.
• Excellent communication and collaboration skills.
• Certifications in Databricks would be beneficial.

Role :   Data Engineer

Industry Type :   IT Services & Consulting

Department :   Engineering - Software & QA

Employment Type :   Full Time, Permanent

Role Category :   Software Development

Education

UG :   B.Tech / B.E. in Any Specialization

• PG :   M.Tech in Any Specialization, MCA in Any Specialization

Skills Required

Azure Data Factory, Pyspark, Azure Databricks, Sql