Problem/Motivation

The transition to Drupal 9 sees us shuffling around some of the testing infrastructure.

We might keep the Testing UI: #3028663: Split up simpletest into simpletest and testing_ui to enable easier decisions for Drupal 9

We might not need the Simpletest framework at all: #2807237: PHPUnit initiative

We might need to maintain some of the Simpletest framework: #2866082: [Plan] Roadmap for Simpletest

Some core devs want to abandon run-tests.sh in favor of, for instance, paratest, though there is no issue about this conversion.

Since it's impossible to search for 'run-tests.sh' on D.O, here is a meta about it: #2626986: [meta] Improvements to run-tests.sh

Background

A good chunk of the test runner code that run-tests.sh uses is within the simpletest module. That's why the script has a dependency on that module.

run-tests.sh is itself largely untestable and brittle.

It is not a good fit for our actual testing needs, having been accrued over many years of short-term fixes.

Much of the work to refactor this part of the testing system has been done but not reviewed or committed. See child issues for #2866082: [Plan] Roadmap for Simpletest

Proposed resolution

Decide the fate of run-tests.sh. Will we support it and make it a better fit for our needs? Or will we gut it and use some other, as yet unproven test runner?

Remaining tasks

Do all the things.

User interface changes

API changes

Data model changes

Release notes snippet

Comments

Mile23 created an issue. See original summary.

mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
Issue summary: View changes
mondrake’s picture

mondrake
Location 🇮🇹
commented
Related issues: +#2781123: [experiment] Explore paratest to run our phpunit tests in parallel

Adding related

Version: 9.x-dev » 9.0.x-dev

The 9.0.x branch will open for development soon, and the placeholder 9.x branch should no longer be used. Only issues that require a new major version should be filed against 9.0.x (for example, removing deprecated code or updating dependency major versions). New developments and disruptive changes that are allowed in a minor version should be filed against 8.9.x, and significant new features will be moved to 9.1.x at committer discretion. For more information see the Allowed changes during the Drupal 8 and 9 release cycles and the Drupal 9.0.0 release plan.

xjm’s picture

xjm
she/her
Primary language English
commented
Version: 9.0.x-dev » 9.1.x-dev

These would be a minor-only changes if not major-only ones. Since 8.9.x and 9.0.x are now in beta, I'm moving this to 9.1.x. Thanks!

joachim’s picture

joachim commented

> Some core devs want to abandon run-tests.sh in favor of, for instance, paratest, though there is no issue about this conversion.

Why do we need something more than vendor/bin/phpunit?

lendude’s picture

lendude
Primary language Dutch
Location Amsterdam
commented

@joachim concurrency. PHPUnit doesn't do it, the testbot needs it.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.