Develop and maintain comprehensive automated test suites for generative AI models, including language models,and multimodal systemsCreate innovative testing strategies to validate the quality, coherence, and safety of AI-generated contentDesign and implement automation frameworks to test generative AI products across multiple platforms and use casesCollaborate closely with AI researchers, data scientists, and product developers to understand product requirements and unique testing challenges of generative systemsDevelop tools and scripts to generate diverse test data sets and simulate real-world generative AI scenariosImplement automated tests for prompt engineering, fine-tuning processes, and model versioningContinuously improve testing methodologies to keep pace with constantly evolving generative AI technologiesAnalyze test results and provide detailed reports on model performance, potential biases, and areas for improvementSkills Required
Computer Science, Automation, Data Validation, Test Strategy, Test Tools