Problem/Motivation
As a developer, I am concerned about eco-friendly and good development practices.
Trying to have a meaningful impact on our world, I think we should try to minimize the digital footprint of all Drupal projects we deploy to the world wide web.
Steps to reproduce
- Build a standard Drupal project
https://github.com/drupal-composer/drupal-project
It takes ~230Mb of disk space - Remove
tests/directories and otherREADME*files
Codebase weights 30% less
Disk usage : ~160Mb
Proposed resolution
Add a script to automatically delete unnecessary directories and files in the whole project - including webroot and vendors.
Example: https://github.com/MatthieuScarset/drupal-template/blob/10.x/scripts/dru...
# Delete Drupal test directories.
# https://unix.stackexchange.com/a/89937
find . -type d -name tests -exec rm -rf {} +
# Delete README files.
find . -type f -name 'README*' -delete
find . -type f -name 'CHANGELOG*' -delete
# Delete patch records.
find . -type f -name 'PATCHES.txt' -delete
# Delete markdown files except licenses.
find web -type f -name "*.md" -not -name "LICENSE*" -delete
# Delete txt files except humans and licenses.
find web -type f -name "*.txt" -not -name "salt*" -not -name "humans*" -not -name "LICENSE*" -not -name "COPYRIGHT*" -delete
# Delete testing and demo profiles.
find web/core/profiles -type d -name 'testing*' -exec rm -rf {} +
find web/core/profiles -type d -name 'nightwatch_testing' -exec rm -rf {} +
find web/core/profiles -type d -name 'demo_umami' -exec rm -rf {} +
Remaining tasks
Review the script in MR and add comment whether or not it is a good addition to core.
Also add comments about we should rather consider using this bash script or another implementation such as a composer plugin (e.g. drupal-core-vendor-hardening).
Issue fork drupal-3401562
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
bramdriesenSlack discussion: https://drupal.slack.com/archives/C06GX5T33/p1699958993938969
A test performed by jurgenhaas
Quite impressive to see a ±50% reduction in size!
Comment #3
matthieuscarset commentedComment #5
cilefen commentedIMO this is an ordinary feature request for Drupal Core. I don't think it has to pass the "ideas" phase.
Comment #6
matthieuscarset commentedThere should definetely be an idea about sustainability and how make Drupal more eco-friendly in the core ideas issue queue - i'll check and create one if necessary.
But I agree this present issue is more of a task, a small - but meaningful 🙂 - addition to core.
So moving it back to core.
Comment #7
quietone commented@matthieuscarset, thanks for making this issue and providing a script. I have not tried the script or even read it carefully. What caught my eye is that the script is called "ecofriendly". In my experience that word is over used and too often misleading. I think we should just avoid any controversy about the word and use a descriptive name.
Comment #8
matthieuscarset commentedThank you for the quick review. I agree ecofriendly is not the best name for this script.
Let's call it what it does:
delete-tests-and-files.shMR updated and ready for review.
Comment #9
mfbI like the idea of core tests not being installed when you create a project. They aren't included for composer packages in the vendor directory, so why should core's be included.
Comment #10
bramdriesenRenaming the issue title a bit as well.
Comment #11
smustgrave commentedWonder if these should be kept though? People may look into core for READMEs or the CHANGELOGs after an update.
Comment #12
mfbWhen composer downloads a package, the tests were already removed (i.e. never packaged to begin with), thus reducing both the size of the zip file and the resulting extracted package. So, I'd say tests should be removed at the packaging stage of drupal core.
Comment #13
bramdriesenActually at any package stage? (e.g. contrib).
The script would also clean NPM assets and vendor packages I would assume which can never by covered by packaging here on D.O
Comment #14
matthieuscarset commented#11 the intention for this script was to be run before "packaging for production" thus I thought it was relevant to remove the most files possible
I might be wrong but i dont think Readme-like files are necessary on most prod environments.
I am curious what are good practices/standards regarding "unncessary" files in a web app.
Do we think we should keep them or do we try to minimize the size of Drupal?
Comment #15
matthieuscarset commentedOne thing to consider is that some texts files are required by
drupal-scaffoldMaybe the script should be adapt to delete files specifically, considering this type of execptions.
Comment #16
bramdriesenCorrect, unless you have the help module enabled and some contribs that try to render the readme files.
Comment #17
matthieuscarset commentedTesting this script again, the Drupal projects weights 670Mb locally after a composer install.
Removing only the
testsdirectories saves 62Mb (total disk usage: 602Mb).Removing all text files saves 8Mb (total: 594Mb).
I suggest we keep all texts files because the gain is not so much and there are issues if some text files are not found (e.g. README with the help module, INSTALL.txt file with drupal-scaffold).
Updating code in attached MR.
Comment #18
aaronmchaleJust for the record, I recommended in Slack that an issue be opened in the ideas queue because to me it seemed like there was potential for wider discussion around this topic. Which for the record I fully support! But who's to say we can't find more than one possible solution to tackle this?
The advantage of the ideas queue is that it gives a space early iteration and feedback before any work begins, in my opinion that should be open to all kinds of ideas. It also helps to gain momentum for an idea, because people are more likely to be watching the ideas queue.
Having said that, I see some good work is already being done, and as I'm in favor of this, I'm not going to stand in the way of this progressing! So, carry on!
Comment #19
matthieuscarset commented@AaronMcHale I totally agree. There is a broader topic of making Drupal more sustainable and this should be discussed within the community. Hoever I don't think this current issue is the best place to discuss such ideas. I see this script as a simple/specific addition to core.
I suggest we have conversation about sustainability within the dedicated project: https://www.drupal.org/project/sustainability
I tagged the issue respectively for reference.
Comment #20
mglamanInstead of a script, what about using
.gitattributesto exclude non-API test classes? You can't just purge all test files. Then KernelTestBase would break.This won't work regardless demo_umami usage.
Comment #21
matthieuscarset commented@mglaman I wonder what you mean by "this won't work". Do you mean to run tests on production?
Comment #22
mglamanNo, but when would this script run? If it's not excluded from the packaged Drupal archive then we're still downloading the amount of files. If it's a post-install with Composer script, we're deleting the files and cannot run tests.
How is this intended to run? Inside packaging of a deployment artifact for end users? Then it could just be document for folks on how to minimize container images and other artifacts.
Comment #23
borisson_I think "just documenting" is not good enough, having good defaults is a thing we try to do everywhere in Drupal, so it should be the same for our sustainability efforts.
I think we could add this to core-recommended for example, since that is what sites use, and not what you'd use when trying to develop for drupal?
A much more aggressive solution could be https://github.com/dg/composer-cleaner. I tried this on a laravel project recently and that saved another ~50mb.
Comment #24
mgiffordWith Drupal CMS the impact would be even bigger with all the contrib files. I do think this is worth exploring more. Would there ever be a need to produce a developer version (with the extra code), or is that just what you would find in Git?
And agreed @borisson_ it is about setting up good defaults for when everyone works to download Drupal (or Drupal CMS).
Comment #25
borisson_I have been thinking about this for a while, I'd like to think we can make this a dependency of
core-recommendedand not include it incore-dev. That way it can be a post-install script, however you are right there are some base classes we would need to keep around.