Problem/Motivation

Currently this module is targetting Drupal CMS and comes with some test groups aimed specifically at Drupal CMS itself. However if a module isn't in Drupal CMS where should these tests go?

  • We don't want all modules to depend on this module.
  • Maybe we have a folder for the tests? You can export the yaml file of the tests and test groups for your agent and then put it in a specific consistent folder within your module.
  • If you have the AI Agent Test module installed it either automatically imports the tests from all modules that have anything in that folder or there is a button for importing (Can we use core configuration management?)

Another issue is which version of Drupal CMS and AI should these tests target?

  • Tests and Test groups should store the version of everything they are on when they were created. The Test Group Detailed information should show this.
  • There should be a free text place to write the versions that this supports (For example Drupal CMS 1.1). The test group creator can put in the information they think is relevant (It's not true that a new version of the AI module necessarily invalidates the tests.
  • All the tests are likely to change when we have our new Tools API and update Drupal CMS AI Agents to use them. Need to plan for this.
  • Maybe we should have Drupal CMS AI Agent Tests as its own module? It can have versions that mimic Drupal CMS' versioning?
  • We need to figure out where the Drupal CMS XB AI Agent tests go as thats for version 2.0 of Drupal CMS?

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Comments

yautja_cetanu created an issue. See original summary.

yautja_cetanu’s picture

Issue summary: View changes
shashikanth171’s picture

Modules can have test group YAML files, saved in a folder with specified naming convention.

AI Agent Test module should automatically parse such folders from enabled modules and load the test groups.

Maybe we can have certain keys for AI features in the module's info file.

With modules shipping the tools along with the AI Agent tests, the task #3541338: Create a Central Store of tests such as on github. can be stripped down to "Tests Browser and Import" feature of the AI Agents Tests module. We can implement new UX for Tests list as Tests Browser along with import feature.

shashikanth171’s picture