Problem/Motivation

\Drupal\Tests\automatic_updates\Build\CoreUpdateTest::testCron Test fails on local
my configs are Core: 9.5.x and automatic updates: 8.x-2.x
after applying https://www.drupal.org/project/automatic_updates/issues/3319045#comment-... patch for better results I get

screeenshot of bug
on line where assertStageEventsLogged() is called.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#11 3319497-10.patch1000 bytestedbow
#10 3319497-10.patch0 bytestedbow
#10 interdiff-6-10.diff934 bytestedbow
#6 3319045-build-events-assert-3319497-6.patch13.39 KBtedbow
#6 3319497-6-simple-fix.patch776 bytestedbow
ss.png98.23 KByash.rode

Comments

yash.rode created an issue. See original summary.

yash.rode’s picture

yash.rode commented
Issue summary: View changes
yash.rode’s picture

yash.rode commented
Issue summary: View changes
tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented

@yash.rode can run the full test suite locally and see what fails.

I think from our discussion more build tests are failing that just CoreUpdateTest::testCron
we have a script composer test that will run all the tests.
This will take a while but can you run this and post the results here.

thanks

After that I have an idea what might give us more information on the build tests.

you said \Drupal\Tests\automatic_updates\Build\CoreUpdateTest::testUi is failing at

$assert_session->pageTextContains('Ready to update');

This could be the same error as in cron.

Before this assert line can you save the output of the page to a file and upload it here? I think it will show us that particular error.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented

@yash.rode thanks for filing the issue. Can you give some more context?

How did you set up the dev environment? Which version of PHP? Which version of composer do you have installed?
Are other tests in CoreUpdateTest failing or this the only one?

Also can search the project directory you are working with and test if there are any symlinks present? find . -type l -ls works for this

Please provide as much information as you can, thanks.

Thanks

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Status: Active » Needs review
StatusFileSize
new 3319497-6-simple-fix.patch776 bytes
new 3319045-build-events-assert-3319497-6.patch13.39 KB

My findings

I was able to re-produce a locally failing build tests in \Drupal\Tests\automatic_updates\Build\CoreUpdateTest.

I created a new dev environment with scripts/setup_local_dev.sh

I ran the tests with

php core/scripts/run-tests.sh --sqlite /tmp/test.sqlite --url http://au-contrib.test --module automatic_updates --types PHPUnit-Build --verbose

The tests failed.

I think the problem is because in \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::createVendorRepository which is used to make a the contents of the vendor.json file it will put information about 'drupal/core-vendor-hardening', 'drupal/core-project-message' core packages. It will only do this if you do not have these projects symlinked. Because of this line

if (is_dir($path) && !is_link($path)) {

So in my local I had 2 environments, 1 that I had set up a long time ago with the projects 'drupal/core-vendor-hardening', 'drupal/core-project-message' symlinked and other that I made today with scripts/setup_local_dev.sh in which these packages were in vendor but not symlinked.

The problem is if 'drupal/core-vendor-hardening', 'drupal/core-project-message' are not symlinked in your local project then \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::createVendorRepository will end up putting them in vendor.json with version 9.5.x-dev(or whatever your local branch of core is). This will cause build tests to fail because composer create-project command inside \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::createTestProject will fail. We should never have reference you actual core clone version in our build. They currently simulate updating from 9.8.0 to 9.8.1. We picked this version because we assumed they would never actually be released.

testing to see if you have the same problem

I think @yash.rode and @travis.carden had failing test after a fresh environment. I think @wim leers had local pasing tests.

If yall could try this out and report back. I would be interested to know the result for those who have failing build tests locally and those that passing build tests.

  1. checkout 8.x-2.x
  2. from the base of your clone run find . -type l . did this find any symlinks?
  3. Add protected $destroyBuild = FALSE; to \Drupal\Tests\automatic_updates\Build\CoreUpdateTest this will make it so that build test does not destroy the workspace when the test is over
  4. Run the tests with php core/scripts/run-tests.sh --sqlite /tmp/test.sqlite --url http://auto_updates_dev.test --module automatic_updates --types PHPUnit-Build --verbose replacing http://auto_updates_dev.tes the URL for your local site. . Paste back the results of this into a comment on this issue, with answers to the other questions in this list
  5. After the tests fail or pass look for the work space directories in your temp directory. For instance 1 of mine was /private/tmp/build_workspace_20e87df060f4016dfc26eda49609ef54GdEBBH it should have build_workspace_. in the name and then a random number. It should have been created very shortly after started the test. Make sure you get the recent one
  6. In the base of that folder should be a vendor.json file. for instance mine is at /private/tmp/build_workspace_20e87df060f4016dfc26eda49609ef54GdEBBH/vendor.json
  7. Open the vendor.json in a code editor
  8. Does that file have either drupal/core-project-message" or drupal/core-vendor-hardening? If so what "version" key is under those packages?

The fix

I will upload a patch, 3319497-6-simple-fix.patch, for what fixed it for me. I also pushed my worked to https://github.com/tedbow/automatic_updates/tree/3319497-simple-fix

The solution for me was in \Drupal\Tests\package_manager\Build\TemplateProjectTestBase::createVendorRepository to explicitly skip 'drupal/core-vendor-hardening', 'drupal/core-project-message' packages. Right now I hard coded them but we can probably get these dynamically. We don't need them in vendor.json as core itself will have information information about these packages.

Please try this patch and then report again with the steps from "testing to see you have the same problem"

My fix not working for you?

If 3319497-6-simple-fix.patch does not fix your local tests I am also uploading
3319045-build-events-assert-3319497-6.patch. This is the branch I was working on when I figured out the fix above. It has more changes that I removed because didn't think they were needed and the work from #3319045: Assert that all expected package manager events are fired during build tests.. They stuff I removed wasn't needed to fix my tests but if 3319497-6-simple-fix.patch still leaves you with failing build test, I would like to see if 3319045-build-events-assert-3319497-6.patch would fix them.

if you tried out this patch please report again with the steps from "testing to see you have the same problem"

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Title: Build test failure on local in 8.x-2.x » Build test failure on local in 8.x-2.x due to setup_local_dev.sh
Priority: Normal » Critical
Status: Needs review » Reviewed & tested by the community
Issue tags: +DX (Developer Experience)

If yall could try this out and report back. I would be interested to know the result for those who have failing build tests locally and those that passing build tests.

I did this.

I can confirm that this caused the problem.

In other words: the build tests fail because of setup_local_dev.sh 🙈 it indeed resulted in 

$ find . -type l | wc -l
       0

and that indeed caused this failure:

$ php core/scripts/run-tests.sh --sqlite /tmp/test.sqlite --url http://core.test --module automatic_updates --types PHPUnit-Build --verbose

Drupal test run
---------------

Tests to be run:
  - Drupal\Tests\automatic_updates\Build\CoreUpdateTest

Test run started:
  Tuesday, November 8, 2022 - 10:18

Test summary
------------

Drupal\Tests\automatic_updates\Build\CoreUpdateTest            0 passes             1 exceptions             
FATAL Drupal\Tests\automatic_updates\Build\CoreUpdateTest: test runner returned a non-zero error code (2).
Drupal\Tests\automatic_updates\Build\CoreUpdateTest            0 passes   1 fails                            

Test run duration: 3 min 42 sec

Detailed test results
---------------------


---- Drupal\Tests\automatic_updates\Build\CoreUpdateTest ----


Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Exception Other      phpunit-185.xml      0 Drupal\Tests\automatic_updates\Buil
    PHPUnit Test failed to complete; Error: PHPUnit 9.5.26 by Sebastian
    Bergmann and contributors.
    
    Warning:       Your XML configuration validates against a deprecated
    schema.
    Suggestion:    Migrate your XML configuration using
    "--migrate-configuration"!
    
    Testing Drupal\Tests\automatic_updates\Build\CoreUpdateTest
    EEEE                                                                4 / 4
    (100%)
    
    Time: 03:41.621, Memory: 26.00 MB
    
    There were 4 errors:
    
    1) Drupal\Tests\automatic_updates\Build\CoreUpdateTest::testApi
    Behat\Mink\Exception\ExpectationException: Current response status code is
    500, but 200 expected.
    
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:794
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:130
    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:81
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:661
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:144
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:97
    
    2) Drupal\Tests\automatic_updates\Build\CoreUpdateTest::testUi
    Behat\Mink\Exception\ResponseTextException: The text "Ready to
    update" was not found anywhere in the text of the current page.
    
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:811
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:262
    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:114
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:661
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:144
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:97
    
    3) Drupal\Tests\automatic_updates\Build\CoreUpdateTest::testCron with data
    set "RecommendedProject" ('RecommendedProject')
    Behat\Mink\Exception\ResponseTextException: The text "No update
    available" was not found anywhere in the text of the current page.
    
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:811
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:262
    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:259
    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:156
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:661
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:144
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:97
    
    4) Drupal\Tests\automatic_updates\Build\CoreUpdateTest::testCron with data
    set "LegacyProject" ('LegacyProject')
    Behat\Mink\Exception\ResponseTextException: The text "No update
    available" was not found anywhere in the text of the current page.
    
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:811
    /private/tmp/auto_updates_dev/vendor/behat/mink/src/WebAssert.php:262
    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:259
    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:156
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/Framework/TestSuite.php:673
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:661
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:144
    /private/tmp/auto_updates_dev/vendor/phpunit/phpunit/src/TextUI/Command.php:97
    
    ERRORS!
    Tests: 4, Assertions: 389, Errors: 4.
Fail      run-tests. Unknown              0 Unknown                            
    FATAL Drupal\Tests\automatic_updates\Build\CoreUpdateTest: test runner
    returned a non-zero error code (2).
wim.leers at MacBookPro-WimLeers in /tmp/auto_updates_dev on 9.5.x*

👆 Note that this indeed fails on

    /private/tmp/auto_updates_dev/modules/contrib/automatic_updates/tests/src/Build/CoreUpdateTest.php:156

… which is what @yash.rode has been reporting for over a week now! 😬

So, applying the fix and running it again;

$ cd modules/contrib/automatic_updates/ && curl https://www.drupal.org/files/issues/2022-11-07/3319497-6-simple-fix.patch | git apply -3v && cd ../../..
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   776  100   776    0     0   1583      0 --:--:-- --:--:-- --:--:--  1606
Checking patch package_manager/tests/src/Build/TemplateProjectTestBase.php...
Applied patch to 'package_manager/tests/src/Build/TemplateProjectTestBase.php' cleanly.
Applied patch package_manager/tests/src/Build/TemplateProjectTestBase.php cleanly.
wim.leers at MacBookPro-WimLeers in /tmp/auto_updates_dev on 9.5.x*
$ php core/scripts/run-tests.sh --sqlite /tmp/test.sqlite --url http://core.test --module automatic_updates --types PHPUnit-Build --verbose

Drupal test run
---------------

Tests to be run:
  - Drupal\Tests\automatic_updates\Build\CoreUpdateTest

Test run started:
  Tuesday, November 8, 2022 - 10:25

Test summary
------------

Drupal\Tests\automatic_updates\Build\CoreUpdateTest            4 passes                                      

Test run duration: 5 min 7 sec

Detailed test results
---------------------


---- Drupal\Tests\automatic_updates\Build\CoreUpdateTest ----


Status    Group      Filename          Line Function                            
--------------------------------------------------------------------------------
Pass      Other      CoreUpdateTest.ph   64 Drupal\Tests\automatic_updates\Buil
    
Pass      Other      CoreUpdateTest.ph   94 Drupal\Tests\automatic_updates\Buil
    
Pass      Other      CoreUpdateTest.ph  130 Drupal\Tests\automatic_updates\Buil
    
Pass      Other      CoreUpdateTest.ph  130 Drupal\Tests\automatic_updates\Buil

That worked!

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Related issues: +#3319679: Assert known preconditions for test runs and fail early if unmet

Given the painful process it required to get to #6, I think we ought to prevent this from ever happening again in the future, so created #3319679: Assert known preconditions for test runs and fail early if unmet.

IMHO we should add a @see that points to that issue — and for now, perhaps a @todo.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Title: Build test failure on local in 8.x-2.x due to setup_local_dev.sh » Build test failure on local in 8.x-2.x due to drupal core packages being added to vendor.json

Actually the problem is that these packages that ship with Drupal core should never be added to vendor.json. Can explain in more detail later

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Status: Reviewed & tested by the community » Needs review
StatusFileSize
new interdiff-6-10.diff934 bytes
new 3319497-10.patch0 bytes

Added the todo to https://drupal.org/i/3319679. if tests pass will merge

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
StatusFileSize
new 3319497-10.patch1000 bytes

ugghh not remembering how to do patch files

  • 0d934d2 committed on 8.x-2.x 
    Issue #3319497 by tedbow, yash.rode, Wim Leers: Build test failure on...
tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Issue tags: +core-mvp