Problem/Motivation

Over in the drupalci testbot issue queue we have this issue: #2569585: Split incoming patches into psr0/psr4 tests and code and run just new tests first.

The point of that issue is to provide a way to run changed tests first and potentially allow them to fail out the test process.

In order to keep a separation of concerns, we should have core provide a list of test files which can be compared to a list of modified files by the testbot.

This way, modified tests can be prioritized without turning the testbot into a horribly opinionated tool.

We could also use this new option to provide a list of tests which have been modified, and/or deleted.

Proposed resolution

Add run-tests.sh --list-files and run-tests.sh --list-files-json.

These will be similar to run-tests.sh --list, except they will provide either the file list, or a structured JSON classmap.

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#9 interdiff.txt538 bytesMile23
#9 2841948_9.patch2.04 KBMile23
#7 interdiff.txt538 bytesMile23
#7 2821387_17.patch72.56 KBMile23
#5 interdiff.txt349 bytesMile23
#5 2841948_5.patch1.87 KBMile23
#2 2841948_2.patch1.86 KBMile23
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Mile23 created an issue. See original summary.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
FileSize
2841948_2.patch1.86 KB

Patch.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, 2: 2841948_2.patch, failed testing.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Needs work » Needs review
FileSize
2841948_5.patch1.87 KB
interdiff.txt349 bytes

Typo.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Chapter Three commented 
+++ b/core/scripts/run-tests.sh
@@ -89,6 +89,33 @@
+if ($args['list-files'] || $args['list-files-json']) {

Should we document what this might be used for on the testbot side? Basically copy what is in the issue summary ...

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
FileSize
2821387_17.patch72.56 KB
interdiff.txt538 bytes

Added.

Status: Needs review » Needs work

The last submitted patch, 7: 2821387_17.patch, failed testing.

Mile23’s picture

Mile23
Primary language English
Location Seattle, WA
CreditAttribution: Mile23 as a volunteer commented
Status: Needs work » Needs review
FileSize
2841948_9.patch2.04 KB
interdiff.txt538 bytes

Note to self: Don't work on two things at the same time.

The last submitted patch, 7: 2821387_17.patch, failed testing.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner at Chapter Three commented
Status: Needs review » Reviewed & tested by the community

Great!

One day we might be even able to provide test coverage for those things.

I tested both the non JSON and the JSON variant. We have a good amount of tests :)

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
CreditAttribution: alexpott at Chapter Three commented
Category: Feature request » Task
Status: Reviewed & tested by the community » Fixed

Committed and pushed fa4c50a to 8.3.x and fba8750 to 8.2.x. Thanks!

I've committed this to both 8.2.x and 8.3.x since supporting DrupalCI is important and run-tests.sh is not runtime code and this does not make any risky changes.

  • alexpott committed fa4c50a on 8.3.x 
    Issue #2841948 by Mile23, dawehner: Modify run-tests.sh to show file...

  • alexpott committed fba8750 on 8.2.x 
    Issue #2841948 by Mile23, dawehner: Modify run-tests.sh to show file...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.