Key Responsibilities :
- ETL Testing & Validation : Lead the quality assurance process for our ETL (Extract, Transform, Load) pipelines.
You will be responsible for validating data accuracy and integrity between various sources and targets.
SQL & Data Warehouse Testing : Use your expertise in SQL Server Management Studio (SSMS) to write complex queries and validate data in relational databases and data warehouses.Azure Data Stack Testing : Actively test and validate data processes built on the Azure Data Factory, Azure Databricks, and Azure Synapse platforms.Pyspark Scripting & Review : Use Pyspark notebooks and scripts to analyze and validate data within Databricks, ensuring data transformations are working as expected.Collaboration & Agile Methodology : Work closely with data engineers and business stakeholders in an Agile environment.Use Jira for managing test cases, tracking defects, and reporting on progress.
Process Understanding : Leverage a strong understanding of ETL processes and transformations to design comprehensive test cases that cover all data flow scenarios.Source & Target Expertise : Demonstrate experience working with a variety of data sources and targets, ensuring seamless and accurate data Skills & Experience :Strong knowledge of and experience with SSIS (SQL Server Integration Services).Proficiency in SQL and hands-on experience with SQL Server Management Studio (SSMS).Hands-on testing experience with the Azure Data Factory and Azure Synapse.Experience with Azure Databricks and working with Pyspark notebooks.Strong understanding of Pyspark for data validation.Practical experience working in an Agile environment.Proficiency with Jira or similar project management tools.A deep understanding of ETL processes and transformations.Proven experience in validating data between various sources and targets.Experience working with multiple data sources and targets(ref : hirist.tech)