Problem/Motivation

As we know, the phpunit <-> simpletest integration is problematic.
Let's start to plan how we can use the phpunit test runner only.

Proposed resolution

First step: Don't run the tests in the UI but show just a code you can use in your shell to run it.

Remaining tasks

User interface changes

API changes

Data model changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner created an issue. See original summary.

jibran’s picture

dawehner’s picture

Status: Active » Needs review
FileSize
74.25 KB
3.8 KB

See screenshot.

Status: Needs review » Needs work

The last submitted patch, 3: 2566767-3.patch, failed testing.

The last submitted patch, 3: 2566767-3.patch, failed testing.

MattA’s picture

That command seems unlikely to work on every supported system (especially for those that use Windows). Maybe just provide a link to a help page on d.o?

dawehner’s picture

@MattA
Oh well, feel free to create a follow up to produce a different output for windows machines. I don't care much too be honest. There are ways to run phpunit tests on windows, this is for sure.

Status: Needs work » Needs review

dawehner queued 3: 2566767-3.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 3: 2566767-3.patch, failed testing.

MattA’s picture

Yeah, Windows is irrelevant to the point. I'm saying that a link would probably be better at providing tailored, relevant information for those interested in running tests, rather than a specific command that may not even work. Even if it were simply to the testing chapter of the Drupal developer guide that would provide:

  • Install instructions and specific commands for specific system configurations.
  • The specific guides for SimpleTests and PHPUnit tests.
  • How to write tests that actually test what you want.
  • For those looking to help contribute to issues, how to test and review patches.
  • How to debug tests and/or test modules.
neclimdul’s picture

I wouldn't say irrelevant. A good number of people use it for different reasons. We could probably do windows people a favor and replace in DIRECTORY_SEPARATOR
http://php.net/manual/en/dir.constants.php

dawehner’s picture

Well, I would never argue its irrelevant, I just don't care about it. Feel free to work on it, seems pretty tricky anyway :)

david_garcia’s picture

As we know, the phpunit <-> simpletest integration is problematic.
Let's start to plan how we can use the phpunit test runner only.

Can someone expand on this? I really appreciate being able to run tests from the UI.

david_garcia’s picture

cilefen’s picture

Title: Don't allow to run phpunit based tests via the UI » Don't allow running phpunit-based tests via the UI
Mile23’s picture

As we know, the phpunit <-> simpletest integration is problematic.

We could fix that. :-) #2641632: Refactor simpletest's *_phpunit_*() (and junit) functions etc. to a class, deprecate

Mile23’s picture

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jonathan1055’s picture

Now that #2575725: Run PHPunit tests one at a time via UI to avoid exception when selecting too many has been completed, and the test results are displayed in a better way, maybe this issue is no longer relevant?

dawehner’s picture

@jonathan1055
I still think that running phpunit via phpunit is the way to go, so IMHO its better to encourage people to learn the tools properly.

Mile23’s picture

@dawehner: If we allow running tests at all through the UI, then we should allow running all tests. Then we have two useful tools instead of one.

klausi’s picture

dawehner’s picture

@mile23
Everytime you don't use batch you potentially have an much easier time to figure out why something fails. This is for me mostly about teaching good practises and well this takes time.

Mile23’s picture

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

Project: Drupal core » SimpleTest
Version: 8.9.x-dev » 8.x-3.x-dev
Component: simpletest.module » Code

Triaging issues in simpletest.module as part of the Bug Smash Initiative to determine if they should be in the Simpletest Project or core.

This looks like it belongs in the Simpletest project.