Job Title : Sr. ETL Test Automation Engineer
Experience : 510 Years
Job Summary :
We are seeking a highly skilled Sr. ETL Test Automation Engineer with expertise in testing large-scale data integration, pipelines, and data warehouse ecosystems. The role requires advanced knowledge of ETL testing, automation frameworks, Kafka, and AWS data platforms, along with strong problem-solving skills to ensure data accuracy, consistency, and reliability across complex workflows.
Key Responsibilities :
Test Automation Design & Execution :
- Design and implement automated testing strategies for validating ETL pipelines, streaming data flows, and batch processes.
- Build and maintain automation frameworks to test data ingestion, transformation, storage, and delivery across heterogeneous platforms.
ETL / Data Validation :
Perform end-to-end validation of data pipelines between DB2 source systems and downstream targets (e.g., S3, Redshift, Glue).Develop SQL queries for validating large datasets, transformations, and aggregations.Conduct data quality checks (duplicate detection, null handling, reconciliation, referential integrity).Kafka Streaming Validation :
Test and validate Kafka topics for schema evolution, offset tracking, and producer / consumer data consistency.Ensure real-time streaming data adheres to defined contracts.Automation & CI / CD :
Develop reusable automated test scripts using frameworks like Pytest, JUnit, or TestNG with Python / Java.Integrate test suites with CI / CD pipelines (Jenkins, GitHub Actions) to support continuous quality.Collaborate with DevOps teams to ensure scalable automated test execution.Test Management & Reporting :
Define test coverage and quality metrics in collaboration with developers and business analysts.Perform functional, regression, performance, and integration testing.Analyze test failures, perform root cause analysis, and document defects with reproducible steps.Collaboration & Stakeholder Engagement :
Work closely with ETL developers, data engineers, QA engineers, and product owners.Provide clear reporting on data quality and testing outcomes to technical and business stakeholders.Required Skills & Experience
6+ years of QA / Test Engineering experience with strong ETL & Data Warehouse testing expertise.Proven hands-on experience with :
Apache Kafka (message validation, schema registry, consumer offsets).AWS ecosystem (S3, Redshift, Glue, Lambda, EMR).DB2 or other relational databases for large-scale data validation.Strong SQL scripting skills (complex queries, joins, data reconciliation).Proficiency in test automation tools / frameworks (Pytest, JUnit, TestNG, Selenium, or Python scripting).Familiarity with CI / CD tools (Jenkins, GitHub Actions) and version control (Git).Solid understanding of data quality, lineage, and metadata validation.Strong analytical, debugging, and communication skills.Nice to Have :
Experience with cloud-native data lakes and ETL orchestration.Exposure to Docker / Kubernetes for creating test environments.Experience in API testing (REST / microservices) supporting data workflows.Familiarity with big data tools (Spark, Hive, Hadoop).(ref : hirist.tech)
