Problem/Motivation
Drupal\Core\Composer\Composer::vendorTestCodeCleanup() needs to be able to run on the whole install and not just one package at a time. Refactoring that to allow for post-create-project hooks would be useful.
This is to support #2990257: Determine how to handle composer scripts in drupal/drupal and be able to perform the vendor test cleanup once the library which supports it is available, which in turn supports creating the drupal/drupal-project-legacy project template in #2982680: Add composer-ready project templates to Drupal core
Regardless of how #2990257: Determine how to handle composer scripts in drupal/drupal is implemented, we will likely need to be able to clean up the vendor directory in one step at the end of the installation process, since the library will have become available and autoloadable by that point.
Proposed resolution
Remaining tasks
User interface changes
API changes
Data model changes
| Comment | File | Size | Author |
|---|---|---|---|
| #12 | interdiff.txt | 1.76 KB | Mixologic |
| #12 | 2998829_12.patch | 5.73 KB | Mixologic |
| #10 | 2998829_10.patch | 5.79 KB | mile23 |
| #8 | interdiff.txt | 3.3 KB | mile23 |
| #6 | interdiff.txt | 5.37 KB | mile23 |
Comments
Comment #2
mile23The reason that
vendorTestCodeCleanup()runs per-package is because that's a Composer limitation.Composer event scripts receive different event classes based on what event is being handled. Different event classes allow for discovery of different information. A post-create-project event does not include easy access to a list of packages that were installed; it has to be deduced from the lock file.
I managed to implement this as part of drupal-project-legacy, right about here: https://github.com/paul-m/drupal-project-legacy/blob/f3f806eb8de548d8513...
That's the do-it-once-at-the-end solution, but it's fragile because it uses the lock file to determine all the packages to clean up.
A better solution is to do it the way we do it in drupal/drupal, per package during install, which is what that branch of drupal-project-legacy ended up looking like: https://github.com/paul-m/drupal-project-legacy/blob/passing/composer.js...
As it turns out this also meets our needs better in terms of generating .htaccess files and so forth for drupal-project-legacy, because it's located in the same
Composerclass.Comment #3
mile23Updating IS to reflect conversations w/ @mixologic.
Comment #4
mile23Comment #5
mile23This patch turns
vendorTestCodeCleanup()into a method that can run during post-install-cmd, post-update-cmd, and post-create-package-cmd.You can use it locally to see it work with
composer install -vv.You can see that it has the same equivalent functionality in https://travis-ci.org/paul-m/drupal-project-legacy which is a test of https://github.com/paul-m/drupal-project-legacy/tree/passing
That test is related to #2982680: Add composer-ready project templates to Drupal core, trying to make a Composer-based template that will install core to be exactly the same as the current tarball.
The patch might also break a few things related to case-sensitive file systems. Being on a non-sensitive file system it's difficult to know.
Comment #6
mile23Chatted with @mixologic about this in slack and we realized that we need BC for anyone who's already installed using drupal/drupal.
Comment #7
MixologicI think we should leave these as is, and not switch these to commands in drupal/drupal.
We can't return the result of doTestCodeCleanup, because it doesnt return anything.
I dont think we need to add a return value as the original function doesnt have a return value.
Do we need to collapse variables like this? I find code that has been 'made more efficient' like this is harder to parse, harder to debug (with xdebug). Also, we should name this var $package_path to be consistent with the function call and usage in the other method.
This call is hard to understand without looking at what we're passing to doTestCodeCleanup. Can we leave in the shorthand variables ($vendor_dir/$io) and add one more for static::$packageToCleanup[$package_key], (like $cleanup_paths).
Seems like we should preserve this behavior ?.
Otherwise I think this should work.
Comment #8
mile23Addressing #7.
Comment #9
MixologicI think that might be the wrong patch, but everything else looks great, except theres still some returning the return of a void function
Comment #10
mile23Woops. Forgot to rebase.
This patch is what should be in #8.
Comment #11
phenaproximaI think this patch makes sense. Can't find much to complain about.
The type hint in the method did not change to match this.
What if $package_name is not set in $packageToCleanup?
Can $cleanup_paths be type hinted as an array?
Comment #12
MixologicHere's a patch addressing #9, #11-1, and #11-3.
#11-2 is already handled by the check surrounding the call to $packageToCleanup:
I have this patch applied to a 'fork' of drupal/core over at https://github.com/ryanaslett/core/blob/8.6.x/lib/Drupal/Core/Composer/C..., and I am using it in the build process for a travis build that utilizes it.
It works exactly how we'd like it to, at least for
https://travis-ci.org/drupal/drupal-project-legacy/builds/443325811#L558...
So. one problem I see is that if we have the
post-create-project-cmdas a script, then this runs fine the first time we do acomposer create-project, but we also want to ensure that those test dirs are stripped out whenever we update to newer versions of those vendor libs.So, if we add the
post-package-installandpost-package-updatescripts to our template's composer.json, then we end up with a bunch oftype of error messages on our create project, because that script does not exist until drupal/core gets installed.
Which is making me start to think that this shouldnt be in core's composer.php class at all, and instead should be part of the drupal-scaffold plugin, and be part of the composer plugin framework, instead of being executed as a script. Nonwithstanding the fact that drupal-scaffold's responsibility is to 'add files to the filesystem to setup the product' - i.e. the corrollary of "remove files from the filesystem that we dont want" fits right into its role.
I think I might see about migrating this cleanup code to the drupal scaffold plugin to see if that works better.
Comment #13
mile23+1 on #12. I was thinking of making it another library altogether, but it could live in drupal-scaffold.
Comment #14
MixologicAs an experiment I moved vendorTestCodeCleanup and vendorTestCodeCleanupCommand to drupal/drupal-scaffold, and it works great. We no longer get the
post-package-install scripterrors.In fact, the post-create-project-cmd ends up doing essentially nothing now that the post-package-install hook works again. So, Im wondering if maybe we don't need to introduce the "make it work on both command and package events" thing.
I also got rid of the script execution in composer.json and moved the functionality into the plugin.
Im working on incorporating all the feedback from https://www.drupal.org/project/drupal/issues/2982684#comment-12769425 and https://www.drupal.org/project/drupal/issues/2982684#comment-12769839, and I think I'll post on that thread.
Comment #15
andypostCould be useful for https://github.com/drupal-composer/drupal-project/issues/323
Comment #17
mile23Tentatively closing this as duplicate because we're implementing vendorTestCodeCleanup() as part of the drupal-scaffold plugin here: #2982684: Add a composer scaffolding plugin to core
Please re-open if we decide on another strategy there.