Problem/Motivation
A little while ago, @tedbow, @TravisCarden, and I talked about how some of our test fixtures have grown out-of-control. In particular, tests which rely on separate "active" and "staged" fixtures are very hard to understand, since you have to know what each fixture represents, and how they relate to the test(s) they're coupled with.
To solve this, we added three new methods to FixtureUtilityTrait: addPackage(), removePackage(), and modifyPackage(). (See #3317385: Remove staged fixtures from StagedProjectsValidatorTest). Then, we used these new methods to make StagedProjectsValidatorTest completely fixture-less in that issue, and in #3317796: Make StagedProjectsValidatorTest fixture-less.
Proposed resolution
Currently we are only able to remove the composer fixture files, not the module/theme fixture files.
Let's do the same thing for other tests which follow a similar pattern. In particular, I think we could convert the following, using the new StagedProjectsValidatorTest as a model:
\Drupal\Tests\package_manager\Kernel\OverwriteExistingPackagesValidatorTest\Drupal\Tests\automatic_updates_extensions\Functional\UpdaterFormTest. In particular, the following fixtures probably do not need to exist at all:no_project: This simulates an active directory with no non-core projects installed; why can't we use Package Manager'sfake_site-- the default basis for any virtual project -- for this?one_project: Could we replace this with a single call toFixtureUtilityTrait::addPackage()?stage_composer/two_projects: This also looks suspiciously like it could be replaced with a couple of calls toaddPackage().
For each of these test classes, we should probably open a new child issue and refactor it to use the new stuff in FixtureUtilityTrait, if we can. The more of our fixtures we can remove, the better off we'll be.
We should first try to convert the kernel tests because the functional tests will need more work get the stage directory call functions in FixtureUtilityTrait
End goal
- Our fixtures should have our
tests/fixturesfolders should have no installed.json, installed.phpThe exceptions to this are
package_manager/tests/fixtures/fake_siteandautomatic_updates_extensions/tests/fixtures/fake-sitewhich are our starting point fixtures - Our tests should not have any *.info.yml.hide files.(we had to use the .hide prefix because drupalci complains if info.yml files have the
projectkey set#3321236: Add actual project folders in \Drupal\Tests\package_manager\Traits\FixtureUtilityTrait::addPackage will have to completed to enable this
-
The only exception to the 2 points above may be if we need to create fixtures that are set up incorrectly specifically to test validation that detects that incorrect setup. It may make more sense to leave a few fixture files than to make our methods in
FixtureUtilityTraitflexible enough create fixtures that we don't consider "correct". An example of this would be if we needed to create installed.json and installed.php files for single use case where the packages in these 2 files did match. In a valid composer setup these packages would be the same in these 2 files.
Issue fork automatic_updates-3317815
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
phenaproximaComment #3
phenaproximaComment #4
phenaproximaComment #5
kunal.sachdev commentedComment #6
tedbowPut a not in the summary about doing the kernel test first the functional test are a bit tricker because of getting the stage directory
Comment #7
tedbowComment #8
tedbowWe have 1 child issue left #3321904: Remove automatic_updates_extensions/tests/fixtures/stage_composer in favor of using StageFixtureManipulator, tagging as
core-mvpbecause that issue is tagged. Then we can close this issueComment #9
tedbowSomeone please, check how much work is left on this. If there specific tests still with fixtures please make issue for them(if there isn't one).
also if there tests where we will always need a fixture, say produce an edge error please note in the summary so we don't have to do this search again.
So for every test that still uses a fixture folder either
Comment #10
omkar.podey commentedComment #11
omkar.podey commentedThe two cases where we need to have fixtures.
package_manager/tests/src/Build/PackageUpdateTest.php=> has a fixture (alpha/updated_module) required for mocking pulling from composer. **I'm Wondering what we are copying to a temp directory could be simulated but I'm not sure.**package_manager/tests/src/Kernel/FixtureManipulatorTest.php=> has a fixture FixtureUtilityTraitTest which is also required to prove that Manipulator works on existing fixturesComment #12
omkar.podey commentedI could not find any more unused/can be removed fixture files anywhere.
Comment #13
wim leers\Drupal\Tests\package_manager\Kernel\FixtureManipulatorTest::testModifyPackage()'s reliance on theexisting_correct_fixturefixture by using thefake_sitefixture? 🤔\Drupal\Tests\package_manager\Build\PackageUpdateTest::testPackageUpdate()'s reliance on thealphaandupdated_modulefixtures by using the manipulator to create those circumstances? 🤔If the answer is "yes" twice, then we could end up with a drastically simpler state: a single fixture.
Assigning to @tedbow for telling me why this is not possible, but optimistically retitling because that'd make this issue actionable and no longer a meta! 🤓
Comment #14
tedbowupdated_moduleis special case where we have 2 version of modules with 2 different code and we test updating between the 2 and make sure call the correct version of code\Drupal\updated_module\PostApplySubscriber. The fixture manipulator isn't made to do this fact that you can inspect the 2 versions of the module I think make understanding it easier.for
alphawe don't need manipulation of the installed.php or installed.json. We could use\Drupal\fixture_manipulator\FixtureManipulator::addProjectAtPathbut the 2 versions need a composer.json with different version values which the FixtureManipulator doesn't handle. so I would say these can't be replaced either.Maybe we should move these modules under a new folder
package_manager/tests/fixtures/build_test_projectsor something we could but a README.md explain how this fixtures are used because they are different than all the other fixtures.So here is what I think we should do
automatic_updates_extensions/tests/fixtures/fake-sitethis should be taggedcontrib-onlyand does not need thesprint, not high prioritypackage_manager/tests/fixtures/FixtureUtilityTraitTest/existing_correct_fixturemove
updated_module,alphainto a sub-folder for build test related repos(this could be done in different follow-up)Unassigning so anyone could take this on
Comment #15
omkar.podey commentedSo from what i understand under
So here is what I think we should do1 is not core-mvp and 2 is what's not clear is it core-mvp or not ?Comment #16
tedbowre #15
Yes #14.1 is not core-mvp. We will need a new issue for that and tag it as
contrib-onlybecause it deals withautomatic_updates_extensions#14.2 is
core-mvpand should be done in this issue.Comment #18
omkar.podey commentedComment #19
wim leersThis is ready AFAICT, and the documentation helps a lot. But I think the documentation could be far clearer still.
I left 4 suggestions: 2 of which are trivial Markdown tweaks, the other 2 are providing more context to the documentation that was already being added), and one also contains a question that may require the creation of a follow-up issue.
Assigning @tedbow for accepting and merging my suggestions, but this is definitely in the RTBC realm :)
Comment #21
tedbowMerged because the last change from gitlab was just to readme. So don't need to wait on tests 🤞🏻(will check 8.x-2.x)
thanks!
Comment #22
wim leersYou did not respond to:
→ that is what could merit a follow-up 😊
Comment #23
tedbow@Wim Leers sorry. Yes I agree with the comment that
(Active|Stage)FixtureManipulatorshould not be used in build tests.The only case I could see were we might use it is if used `\Drupal\fixture_manipulator\FixtureManipulator::addProjectAtPath` to the active directory to simulate having a project that is not known to composer. But we haven't had a need for that yet.
I follow up to make the comment more specific or to somehow forbid this
AFAICT we not using
(Active|Stage)FixtureManipulatorin build tests right now. Did you find a case where we are?Comment #24
wim leersNo, I did not. But we were contemplating it in this very issue — that is literally what the docs we just committed (and you wrote the key parts of it in #14) are saying: "we can't use the manipulator for reasons X and Y" — which wouldn't even be a consideration if build tests could not use the manipulators!
Comment #25
tedbowI created #3338392: Document that StageFixtureManipulator should not be used in build tests as a
core-post-mvplets move discussion there and close thisComment #26
tedbow