Problem/Motivation

When doing automated tests or just trying different models and not using Tests to help with prompt engineering its good to just fire off all the same tests you want and get the overall results.

Steps to reproduce

Proposed resolution

  • Maybe make views bulk operation style UI so you can click a checkbox next to the test groups you want to run and also run all.
  • Or we could have a Tag on test groups and run all of a specific tag? Maybe this is better if we are going to store the collection of test results together? Maybe its Category, or Test Group Group! Test Collection?
  • Can we have the results page show all the test results in one place, so we see the test groups but they appear one above each other and then have overall summaries for the test collection.

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#9 Group batch process monitor.png382.53 KBlbesenyei

Issue fork ai_agents_test-3541324

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

yautja_cetanu created an issue. See original summary.

yautja_cetanu’s picture

yautja_cetanu commented
Priority: Normal » Major
yautja_cetanu’s picture

yautja_cetanu commented
Title: Run more than one test group in Bulk » Run more than one test group in Bulk - Test Collections?
yautja_cetanu’s picture

yautja_cetanu commented
Issue tags: +priority
marcus_johansson’s picture

marcus_johansson commented
Issue tags: +AI Initiative Sprint
arianraeesi’s picture

arianraeesi commented
Issue tags: +AI Product Development
lbesenyei’s picture

lbesenyei commented
Assigned: Unassigned » lbesenyei

lbesenyei opened merge request !11

lbesenyei’s picture

lbesenyei commented
Assigned: lbesenyei » Unassigned
Status: Active » Needs review
StatusFileSize
new Group batch process monitor.png382.53 KB

I have drafted a solution for running test groups by adding a new views bulk action.
Users can select which tests they wan to run by checking the checkbox on the test groups lister, then select the action to run them and submit.

On the confirmation page I've reused the model selection elements.

On the batch processing page I though we need more information so I've injected test monitoring similar to the current group progress page, with collapsible panels.
I'm interested on thought if this implementation, is there a better way to extend the batch progress page with useful information?
Batch monitor
The result page will contain all group results.

I wonder if there should be a checkbox to "Stop on first failure".