Data Observability Specialist

Data Quality (DQ) Observability

• Familiarity with cloud platforms and containerization technologies (Data Bricks).
• Proficiency in scripting and automation using languages such as Python,
• Excellent problem-solving skills with the ability to analyse complex issues and provide efficient solutions.
• Good communication skills and the ability to collaborate effectively across teams.
• Understanding of Agile and DevOps and GIT principles and their application in observability and monitoring contexts.
• Solid experience with instrumentation practices, including metrics, logging, and distributed tracing.
• E2E implementation. Right from requirement gathering and DQ design engaging with the Data Asset owners, development, and reporting.
• Development DQ rules with the help of Databricks
• Presentation of DQ results to the Data Asset Owners and other partners and work with them to build an action plan to improve the DQ results.
• Putting in place specific Data Improvement Plans
• Scale up the DQ framework to all the data assets and more.
• Ensure systems run efficiently, monitoring system performance and handle support in demanding environment.
• Handle the configuration of existing toolset for data quality, KPI system and Reporting.
• 
  

Proficiency in implementing and maintaining observability tools is a plus

Qualifications And Experience Required

• Degree level education, preferably with overall 6-8 years of experience in SQL, Data warehousing and hands on Data Science
• Must be proficient and have experience in Data Science related technologies.
• Strong Python/ PySpark and SQL development skills
• Experience in Azure Databricks
• Agile development methodology using DevOps and GIT repos.
• Experience with Big Data tools and Cloud environment, especially Azure is a big plus.
• Expertise in converting Functional requirements to technical solutions and present in the business context.
• Experience in working with Hub and Spoke model data lakes, preferably with Diageo context is a big plus.
• Keeping pace with IT trends, innovation, and new developments in technology space
• Good Communication Skills required

Skills Required
Devops, Azure Databricks, Big Data Testing, Pyspark, Sql Data Base, Data Analysis, Python