ETL Data Test Automation Engineer

We are seeking experienced ETL Data Test Automation Engineers with Databricks expertise to validate complex cloud data pipelines. Work onsite to ensure data quality, integrity, and performance across ETL/ELT processes using Python automation, PySpark testing, and advanced SQL validations.

Key Responsibilities

• Design/execute ETL/ELT test automation frameworks for cloud data pipelines
• Validate Databricks pipelines (Notebooks, Jobs, Delta Lake, Unity Catalog)
• Write Python test automation using PyTest/UnitTest + Pandas/PySpark
• Perform complex SQL validations (joins, window functions, aggregations)
• Test data lineage, metadata management, SCD transformations
• Automate API/CLI testing for data platform components
• Collaborate with data engineers for test-driven development
  
  

Mandatory Technical Skills

• ETL Fundamentals: Data modeling (Star/Snowflake), SCDs, lineage
• Databricks: Notebooks, Jobs, Delta Lake, Unity Catalog, SQL Warehouse
• Python: PyTest/UnitTest, Pandas, PySpark for test data manipulation
• SQL: Complex joins, window functions, performance validation
• Cloud Testing: Data/ETL testing in AWS/Azure/GCP environments
• Experience: 6-8 years QA (3+ years data testing)
  
  

Skills: testing,sql,python,automation,cloud,etl,data
Skills Required
Etl, Pandas, Pyspark, Cloud Testing, Pytest, Sql, Databricks, Python