Pyspark + Databricks

Key Responsibilities

• Lead the design and implementation of scalable data solutions using Databricks Workflows .
• Oversee integration and utilization of Databricks Unity Catalog to ensure data governance and security.
• Provide technical expertise in Databricks to optimize performance and support advanced analytics.
• Collaborate with cross-functional teams to deliver solutions aligned with business objectives.
• Develop and maintain documentation for data workflows and processes.
• Mentor and guide junior team members to foster continuous learning and improvement.
• Conduct regular code reviews to maintain coding standards and best practices.
• Analyze complex datasets to identify trends and insights supporting strategic decisions.
• Implement robust testing frameworks to ensure reliability and accuracy of data solutions.
• Coordinate with stakeholders to gather requirements and translate them into technical specifications.
• Drive adoption of new tools and technologies to improve team efficiency and capabilities.
• Ensure compliance with industry standards to maintain data integrity and privacy .
• Monitor system performance and troubleshoot issues to maintain optimal platform operation.

Skills Required

Pyspark, Databricks, data engineering , Data Analytics