The testbot is not working, we need to fix it.

CommentFileSizeAuthor
#16 2833282--fix-testbot--16.patch887 bytese0ipso
#5 fix_the_testbot-2833282-5.patch630 bytesMixologic
#2 2833282--fix-testbot--2.patch2.58 KBe0ipso

Comments

e0ipso created an issue. See original summary.

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented
StatusFileSize
new 2833282--fix-testbot--2.patch2.58 KB

First attempt.

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented
Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, 2: 2833282--fix-testbot--2.patch, failed testing.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
StatusFileSize
new fix_the_testbot-2833282-5.patch630 bytes

I've been working on getting the testbot codebase ready to deploy and was trying to use this module as an example of one with tests and composer dependencies.

I got as far as not being able to run the tests due to there being no test group:

https://staging-dispatcher.drupalci.org/job/default/104298/console

So this patch adds that group. It will still fail here, but will allow me to continue working.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 5: fix_the_testbot-2833282-5.patch, failed testing.

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented

@Mixologic that is a base class that is used in the actual tests. It should not be annotated as part of a group since it does not contain any tests. Aren't the other 3 test classes good for these tests? Maybe I'm missing something here.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented

All tests have to be annotated otherwise run-tests.sh --list will fail with a missing @group exception.

The testbots do not run phpunit on their own, they run run-tests.sh which runs simpletest.

I was able to get this to build, and could get the tests to pass if I only ran phpunit, however, under run-tests.sh they would not pass. This is indicative of a number of possibilities:

run-tests.sh executes the tests in parallel, if there is any leakage between the tests, they could cause each other to fail.
The testbots run with drupal in a subdirectory, so if there is any expectation that drupal is the root of the web directory that could also break the tests.

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented

My point is that there are no tests in that class that need to be executed. It's just a regular class inside of the tests directory. Does that need a group annotation as well?

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented

In order to run this on the d.o. testbots, you need to be able to run ./core/scripts/run_tests.sh --list - which finds all test classes, and looks for an @group on those classes. But basically, the only way I can get it to work is to put an @group on that. That doesnt explain why run-tests.sh then fails to run when running at phpunit on its own does not.

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented
Status: Needs work » Closed (outdated)

This was already fixed by the DrupalCI team.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
Status: Closed (outdated) » Active

We didnt fix this. What we did was fix the ability to use composer to install your dependencies. You must still put an @group annotation on your tests, and you must still figure out why your tests fail when running under run-tests.sh, and not just via phpunit.

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented

I'm very confused.

When I said

My point is that there are no tests in that class that need to be executed. It's just a regular class inside of the tests directory.

I meant that TokenBearerFunctionalTestBase is not a test class, but a class other tests classes inherit from for code reuse.

In order to run this on the d.o. testbots, you need to be able to run ./core/scripts/run_tests.sh --list - which finds all test classes

seems to indicate that we only need to add the @group to the actual test classes.

Adding a @group to the base class, results in a CI error reporting no tests are found:

Time: 146 ms, Memory: 4.00Mb

There was 1 failure:

1) Warning
No tests found in class "Drupal\Tests\simple_oauth\Functional\TokenBearerFunctionalTestBase".

Am I missing anything?

e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented
Status: Active » Needs review
StatusFileSize
new 2833282--fix-testbot--16.patch887 bytes

Declaring the base class as abstract will do the trick.

  • e0ipso committed b6f2c6d on 8.x-2.x 
    test(Functional) Fix the testbot (#2833282 by e0ipso, Mixologic)
e0ipso’s picture

e0ipso
Primary language Catalan
Location Can Picafort
commented
Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.