Problem/Motivation

Allow the command line test runner to run all tests but not phpunit tests.

Proposed resolution

Add a new --all-simpletest command argument.

Remaining tasks

None.

User interface changes

Users are able to run only simpletest tests by adding --all-simpletest argument to the command line.

API changes

None.

Data model changes

None.

CommentFileSizeAuthor
#23 2567029-reroll-21.patch1.11 KBgnuget
#21 interdiff.txt846 bytesclaudiu.cristea
#21 2567029-21.patch1.16 KBclaudiu.cristea
#14 interdiff.txt2.19 KBdawehner
#14 2567029-14.patch1.16 KBdawehner
#12 2567029-12.patch1.29 KBMixologic
#11 2567029-11.patch1.05 KBdawehner
#8 2567029-2.patch1.07 KBdawehner
#2 2567029-2.patch1.07 KBdawehner
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner created an issue. See original summary.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Tag1 Consulting for Drupal Association commented
Status: Active » Needs review
FileSize
2567029-2.patch1.07 KB

At that point I'm actually not 100% sure whether we need that. Wouldn't it be enough to run phpunit tests twice, given how fast they are?

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented

They're fast, but not that fast - about 4-5 minutes parallelized which adds up to real money spent on ec2. Plus there'd be the issue of double output, number of failing tests reported, etc. (of the ~13000 tests, phpunit is ~8000 of them). This would be valuable to us.

Fabianx’s picture

Fabianx CreditAttribution: Fabianx as a volunteer commented
Status: Needs review » Reviewed & tested by the community

+1 to this change, drupal_ti users also had talked about this change.

As phpunit (in testsuite unit) never asks the DB I think this also means that we could run phpunit tests in a process and then simpletest in another.

But that means that for Postgres, SQL lite, etc. we don't need to re-run all the unit tests, which feels like a win to me :).

jibran’s picture

jibran
he/him
Primary language Urdu
Location Sydney, Australia
CreditAttribution: jibran as a volunteer commented

Is this true for Functional and Kernel group as well?

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Tag1 Consulting for Drupal Association commented

As phpunit (in testsuite unit) never asks the DB I think this also means that we could run phpunit tests in a process and then simpletest in another.

But that means that for Postgres, SQL lite, etc. we don't need to re-run all the unit tests, which feels like a win to me :).

Well, technically running the unit tests for all databases is maybe a tiny tiny bit of the actual execution time of all tests. But sure, not the worst idea.

Is this true for Functional and Kernel group as well?

Nope, those need the DB, of course, but still, they can be executed via phpunit itself already.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 2: 2567029-2.patch, failed testing.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Tag1 Consulting for Drupal Association commented
Status: Needs work » Needs review
FileSize
2567029-2.patch1.07 KB

Let's see what happens when the file is reuploaded

Status: Needs review » Needs work

The last submitted patch, 8: 2567029-2.patch, failed testing.

The last submitted patch, 8: 2567029-2.patch, failed testing.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer commented
Status: Needs work » Needs review
FileSize
2567029-11.patch1.05 KB

Resolved the conflict.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
FileSize
2567029-12.patch1.29 KB

This causes a "unknown argument --skip-phpunit" when its added to an command line invocation. Hopefully this is all thats needed to fix it.

Status: Needs review » Needs work

The last submitted patch, 12: 2567029-12.patch, failed testing.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer commented
Status: Needs work » Needs review
FileSize
2567029-14.patch1.16 KB
interdiff.txt2.19 KB

Alexpott asked for a positive option.

Status: Needs review » Needs work

The last submitted patch, 14: 2567029-14.patch, failed testing.

The last submitted patch, 14: 2567029-14.patch, failed testing.

The last submitted patch, 14: 2567029-14.patch, failed testing.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented

I added that second test and ran it with the --all-simpletests option, and again, like before, drupalci wasnt finding any results in the database, primarily because I was overwriting the default --keep-results option. Re-running it now with that there. we should see the thinner results this time.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented

That second test has now completed (https://www.drupal.org/pift-ci-job/50741). However It seems like the PHPUnit tests are still being ran.

Fabianx’s picture

Fabianx CreditAttribution: Fabianx as a volunteer commented 
+++ b/core/scripts/run-tests.sh
@@ -163,6 +163,10 @@ function simpletest_script_help() {
+  --all-simpletest

@@ -254,6 +258,7 @@ function simpletest_script_parse_args() {
+    'all-simpletests' => FALSE,

@@ -795,6 +800,11 @@ function simpletest_script_get_test_list() {
+      if ($args['all-simpletests']) {

-simpletest vs. -simpletests

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
CreditAttribution: claudiu.cristea at Webikon commented
Title: Be able to opt out of phpunit running from run-tests.sh » Be able to run only simpletest tests, but no phpunit, running from run-tests.sh
Issue summary: View changes
Status: Needs work » Needs review
FileSize
2567029-21.patch1.16 KB
interdiff.txt846 bytes

I would say all-simpletest because they are not simpletests, they are tests of type "simpletest" :-)

Updated IS and title per #14.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer commented

Agreed with using singular, its too confusing to use the other alternative.

gnuget’s picture

gnuget
Primary language Spanish
CreditAttribution: gnuget as a volunteer commented
FileSize
2567029-reroll-21.patch1.11 KB

This patch needed a reroll, I just did it.

There weren't conflicts, when I rebase it I got a Auto-merging core/scripts/run-tests.sh.

Reroll attached.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer commented
Status: Needs review » Fixed

We can use 

  --types
              Runs just tests from the specified test type, for example
              (i.e. --types "Simpletest,PHPUnit-Functional")

now

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.