Position : Big Data Tester
Experience : 5-9 Years
Location : Pune, India
Job Summary :
We are seeking an experienced Big Data Tester with 5-9 years of experience in software quality assurance, with a dedicated focus on data. The ideal candidate will have a strong background in Python, PySpark, and SQL, along with a proven track record of implementing best practices for data validation and ETL testing. You will be responsible for ensuring the accuracy, integrity, and reliability of our large-scale data systems. This role is perfect for a detail-oriented professional with a strong technical skillset and a passion for data quality.
Key Responsibilities :
Data Validation and Testing :
- Design, develop, and execute comprehensive test plans and test cases for Big Data applications.
- Perform data validation and integrity checks across various data sources and systems.
- Use Python and PySpark to write scripts and automate testing for ETL processes and data pipelines.
SQL & Data Querying :
Write and optimize complex SQL queries to validate data transformations and business logic.Perform data analysis and use querying skills to identify discrepancies and anomalies.Troubleshooting and Collaboration :
Work closely with data engineers and developers to troubleshoot and resolve data quality issues.Participate in the entire software development lifecycle, ensuring quality is integrated from the start.Required Skills & Qualifications
Core Experience : 5-9 years of experience in a testing or quality assurance role.
Mandatory Technical Skills :
Expertise in Python and PySpark for data processing and automation.Extensive hands-on experience with SQL, including writing complex queries.Strong knowledge of ETL processes and best practices for data validation.Professional Attributes :
A strong, hands-on approach to testing and a meticulous attention to detail.Excellent analytical and problem-solving skills.Strong communication and collaboration skills to work effectively with cross-functional teams.Preferred Skills :
Experience with Unix environments.Hands-on experience with Big Data technologies such as Hadoop, Spark, Kafka, and NoSQL databases (MongoDB, Cassandra, Hive, etc.).Experience with both traditional and modern Data Warehousing solutions (Oracle, Teradata, SQL Server, Amazon Redshift, Google BigQuery, Snowflake).Prior development experience in AWS.(ref : hirist.tech)