Problem/Motivation

If you move the example ./composer.json out of the webroot, Unit Tests fail:

Drupal\Tests\ComposerIntegrationTest::testComposerJson
file_get_contents(docroot/composer.json): failed to open stream: No such file or directory

docroot/core/tests/Drupal/Tests/ComposerIntegrationTest.php:62

Proposed resolution

Since ./composer.json is an example file (like example.gitignore), then it should be excluded from testing since it has no function to Drupal what-so-ever.

Remaining tasks

  1. Wirte Patch

User interface changes

None

API changes

None

Data model changes

None

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

timmillwood’s picture

Status: Active » Needs review
FileSize
1.01 KB

Here's an initial patch, it just does a simple file_exists, then fails if the file is missing, unless it's ./composer.json.

I was thinking we should return some sort of message if ./composer.json is missing, but not fail the test, not sure how useful that would be though.

martin107’s picture

I am not sure how to do this ... maybe someone else could comment I would be really grateful

If the thrust of this issue is expanded to :-

"Make sure the file exists AND it has no errors in it" then we would solve the testing required for this issue #2545430: Remove "UnexpectedValueException" from composer.json

currently the only way I know to add this is through the

"drush composer diagnose" command

webflo’s picture

Issue tags: +Composer
webflo’s picture

+++ b/core/tests/Drupal/Tests/ComposerIntegrationTest.php
@@ -60,11 +60,17 @@ protected function getPaths() {
+      if (file_exists($file)) {

I think we should not introduce the file_exists for all composer.json because this fails silent if we remove one of files accidentally. I think it makes more sense to add a special case for root composer.json because this files may does not exists in a project managed with composer.

dawehner’s picture

+++ b/core/tests/Drupal/Tests/ComposerIntegrationTest.php
@@ -60,15 +59,27 @@ protected function getPaths() {
+    if (file_exists($this->root . '/composer.json')) {
+      $this->validateComposerJson($this->root . '/composer.json');
+    }

It would be great to document this file_exists() call. The reasons given by webflo make total sense.

webflo’s picture

tstoeckler’s picture

Status: Needs review » Reviewed & tested by the community

Awesome, thanks!

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

I'm not entirely convinced by this change since this test is for testing core so that file should be there - there are many tests that would fail if tested not on a clean checkout of drupal core. Also there is an issue suggesting making this the main composer json.

webflo’s picture

Status: Needs work » Closed (won't fix)

Thanks correct. Lets focus our efforts on #2380389: Use a single vendor directory in the root. We have to get it in before RC1.

davidwbarratt’s picture

Status: Closed (won't fix) » Active

This is still a problem if you choose to move your composer.json file out of the webroot. :(

timmillwood’s picture

Status: Active » Needs review

Agree with @davidwbarratt.

Queued the patch in #6 for retesting. Think it's still valid.

davidwbarratt’s picture

Status: Needs review » Reviewed & tested by the community

Works for me!

$ ./vendor/bin/phpunit docroot/core/tests/Drupal/Tests/ComposerIntegrationTest.php
PHPUnit 4.8.16 by Sebastian Bergmann and contributors.

..

Time: 213 ms, Memory: 11.75Mb

OK (2 tests, 6 assertions)
alexpott’s picture

Status: Reviewed & tested by the community » Needs review

I think this is wrong thinking - tests are designed to run against the checkout - if you move things around they break - that is how it is supposed to be. There is no response to my point in #8.

davidwbarratt’s picture

#13,

I think the test is faulty to begin with. It's like testing to see if .htaccess or web.config is there, when technically it's not needed and depending on your configuration (i.e. if you have Drupal in a subfolder, etc.) it might mess things up. If ./composer.json is intended to be modified, I don't think we should have a test ensuring it's in a proper format, or even if it's there at all.

Currently this is the only test that fails in a live environment, so the choice is either to fix it, or skip the entire file.

Mile23’s picture

Since ./composer.json is an example file (like example.gitignore), then ...

The root composer.json file isn't an example file.

If you need to run all the unit tests against an installation where you've moved the composer.json file around, then you can use ./vendor/bin/phpunit -c core --exclude-group Composer We might also make a feature request for run-tests.sh to be able to exclude groups.

Alternately, if there were some way to determine whether the test were being run to test core or test something else, then it might be fair to mark the test as skipped under that circumstance. But that seems a long way to go for an edge case. The file_exists() call negates part of the purpose of the test otherwise.

bradjones1’s picture

Agreed with the point in #14... there's nothing to say Drupal couldn't be included inside another project, with a top-level composer.json.

Regardless, though, I've come to include a composer.json file in the webroot with a simple contents of {} to appease this test, but also composer_manager, which last time I checked also expected to find this file as valid JSON. Just in case you're looking for a quick, non-patch fix.

Mile23’s picture

I thought: Hey, let's add the collected wisdom of this issue to the documentation of ComposerIntegrationTest.

Then I saw getPaths() and thought: That should be a data provider.

Lo the yak was shaved, and I ended up adding a getRoot() static method to UnitTestCase.

Presented here: The full code-changes patch, and also the docs-only patch.

dawehner’s picture

+++ b/core/tests/Drupal/Tests/ComposerIntegrationTest.php
@@ -42,30 +52,35 @@ protected function getErrorMessages() {
+    // Data provider functions are run before setUp(), so we have to determine
+    // DRUPAL_ROOT here.
+    $root = self::getRoot();

Even is the provider is executed before setup() we still don't need a static method, IMHO

Mile23’s picture

Even is the provider is executed before setup() we still don't need a static method

Well, then it won't be useful for setupBeforeClass() either.

But we can just do the docs change in patch 2545344_17-docs-only.patch if that's intractable or out of scope.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

bradjones1’s picture

Status: Needs review » Needs work

Since this was last working, Composer use has basically become required for installing Drupal core from source, and there's also a new test method to compute the hash of the lock file. See http://cgit.drupalcode.org/drupal/commit/core/tests/Drupal/Tests/Compose...

So, including an empty composer.json file is no longer enough to stave off this failure.

I think it would be worthwhile at this point to see if a core maintainer thinks it's worth supporting non-out-of-the-box installation methods that cause this to be an issue, or to simply document this and require the dev to exclude this test suite?

dawehner’s picture

Here is a general question. Why does one run the core tests in their custom projects? It is not the job of each individual site to prove that core works. Maybe though I misunderstand the question. Beside of that the patch here certainly improves the quality of the patch though.

Mile23’s picture

@dawehner: If you run tests using the core tools, the core tests will be discovered and possibly run.

The solution to this problem is to mark the test as incomplete if the file is not present, probably in setUp().

dawehner’s picture

I still think its not the responsibility for you to run those tests and its not the responsibility for Drupal to support arbitrary paths of stuff in its test.
Its also not that our tests are smoke tests for your environment.

Just an idea: We could maybe separate unit from unit-core in our phpunit.xml file and so exclude all those tests easily.

webflo’s picture

I run all core unit tests on https://github.com/drupal-composer/drupal-project. We caught a few bugs around autoloading that way, because the project changes the location of the vendor folder.

Mile23’s picture

Looks like drupal-project does this in travis-ci:

- ./../vendor/bin/phpunit -c core --testsuite unit --exclude-group Composer

@group Composer only applies to this test.

So that's either a workaround for a problem in core or it's a solution to the policy in core. Which do we think it is?

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

kyvour’s picture

Why do not use drupal-finder package?

It's already used in the drush and drupal console for the searching of the composer and drupal root, so seems like it's a stable package which works fine. I don't see any reasons why we can't use it here.

There is a patch for ComposerIntegrationTest. The idea was taken from #17 but code was rewritten to use DrupalFinder class.

kyvour’s picture

Status: Needs work » Needs review

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

borisson_’s picture

I still think its not the responsibility for you to run those tests and its not the responsibility for Drupal to support arbitrary paths of stuff in its test.

I agree with this, it's not for every site to run all tests.

I run all core unit tests on https://github.com/drupal-composer/drupal-project. We caught a few bugs around autoloading that way, because the project changes the location of the vendor folder.

I'm pretty sure that this project is not just another project, excluding this test-group is a good idea for that.

The docs-only patch in #17 is a good idea, otherwise I think we should just close this issue.

bradjones1’s picture

I'd +1 a docs-only change... if you are sophisticated enough to get WTF'd by this you will be able to do some investigating and the proposed docs patch is clear enough to give you some sensible options, while not changing anything functionally on core's end.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
FileSize
132 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture