Problem/Motivation

As a developer, I am concerned about eco-friendly and good development practices.

Trying to have a meaningful impact on our world, I think we should try to minimize the digital footprint of all Drupal projects we deploy to the world wide web.

Steps to reproduce

Proposed resolution

Add a script to automatically delete unnecessary directories and files in the whole project - including webroot and vendors.

Example: https://github.com/MatthieuScarset/drupal-template/blob/10.x/scripts/dru...

# Delete Drupal test directories.
# https://unix.stackexchange.com/a/89937
find . -type d -name tests -exec rm -rf {} +
# Delete README files.
find . -type f -name 'README*' -delete
find . -type f -name 'CHANGELOG*' -delete
# Delete patch records.
find . -type f -name 'PATCHES.txt' -delete
# Delete markdown files except licenses.
find web -type f -name "*.md" -not -name "LICENSE*" -delete
# Delete txt files except humans and licenses.
find web -type f -name "*.txt" -not -name "salt*" -not -name "humans*" -not -name "LICENSE*" -not -name "COPYRIGHT*" -delete
# Delete testing and demo profiles.
find web/core/profiles -type d -name 'testing*' -exec rm -rf {} +
find web/core/profiles -type d -name 'nightwatch_testing' -exec rm -rf {} +
find web/core/profiles -type d -name 'demo_umami' -exec rm -rf {} +

Remaining tasks

Review the script in MR and add comment whether or not it is a good addition to core.

Also add comments about we should rather consider using this bash script or another implementation such as a composer plugin (e.g. drupal-core-vendor-hardening).

Issue fork drupal-3401562

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

matthieuscarset created an issue. See original summary.

bramdriesen’s picture

Slack discussion: https://drupal.slack.com/archives/C06GX5T33/p1699958993938969

A test performed by jurgenhaas

# Step 1

composer install --no-interaction --no-progress --no-dev

# Step 2

composer config --no-plugins --no-interaction allow-plugins.drupal/core-vendor-hardening true
composer require --update-no-dev --no-interaction --no-progress drupal/core-vendor-hardening

# Step 3

wget https://github.com/MatthieuScarset/drupal-template/raw/10.x/scripts/drupal_thinner.sh
chmod +x drupal_thinner.sh
./drupal_thinner.sh

| Step | Vendor | Web |
| ---: | -----: | ---: |
| 1 | 73M | 207M |
| 2 | 71M | 207M |
| 3 | 68M | 118M |

Quite impressive to see a ±50% reduction in size!

matthieuscarset’s picture

Project: Drupal core » Drupal core ideas
Version: 11.x-dev »
Component: other » Idea

cilefen’s picture

IMO this is an ordinary feature request for Drupal Core. I don't think it has to pass the "ideas" phase.

matthieuscarset’s picture

Project: Drupal core ideas » Drupal core
Version: » 11.x-dev
Component: Idea » other

There should definetely be an idea about sustainability and how make Drupal more eco-friendly in the core ideas issue queue - i'll check and create one if necessary.

But I agree this present issue is more of a task, a small - but meaningful 🙂 - addition to core.

So moving it back to core.

quietone’s picture

@matthieuscarset, thanks for making this issue and providing a script. I have not tried the script or even read it carefully. What caught my eye is that the script is called "ecofriendly". In my experience that word is over used and too often misleading. I think we should just avoid any controversy about the word and use a descriptive name.

matthieuscarset’s picture

Status: Active » Needs review

Thank you for the quick review. I agree ecofriendly is not the best name for this script.

Let's call it what it does: delete-tests-and-files.sh

MR updated and ready for review.

mfb’s picture

I like the idea of core tests not being installed when you create a project. They aren't included for composer packages in the vendor directory, so why should core's be included.

bramdriesen’s picture

Title: Add eco-friendly scripts to reduce disk usage » Add script to reduce disk usage by removing unnecessary files

Renaming the issue title a bit as well.

smustgrave’s picture

# Delete README files.
find . -type f -name 'README*' -delete
find . -type f -name 'CHANGELOG*' -delete
# Delete patch records.
find . -type f -name 'PATCHES.txt' -delete

Wonder if these should be kept though? People may look into core for READMEs or the CHANGELOGs after an update.

mfb’s picture

When composer downloads a package, the tests were already removed (i.e. never packaged to begin with), thus reducing both the size of the zip file and the resulting extracted package. So, I'd say tests should be removed at the packaging stage of drupal core.

bramdriesen’s picture

at the packaging stage of drupal core.

Actually at any package stage? (e.g. contrib).

The script would also clean NPM assets and vendor packages I would assume which can never by covered by packaging here on D.O

matthieuscarset’s picture

#11 the intention for this script was to be run before "packaging for production" thus I thought it was relevant to remove the most files possible

I might be wrong but i dont think Readme-like files are necessary on most prod environments.

I am curious what are good practices/standards regarding "unncessary" files in a web app.

Do we think we should keep them or do we try to minimize the size of Drupal?

matthieuscarset’s picture

One thing to consider is that some texts files are required by drupal-scaffold

  [RuntimeException]
  Scaffold file assets/scaffold/files/drupal.INSTALL.txt

Maybe the script should be adapt to delete files specifically, considering this type of execptions.

bramdriesen’s picture

I might be wrong but i dont think Readme-like files are necessary on most prod environments.

Correct, unless you have the help module enabled and some contribs that try to render the readme files.

matthieuscarset’s picture

Testing this script again, the Drupal projects weights 670Mb locally after a composer install.

Removing only the tests directories saves 62Mb (total disk usage: 602Mb).

Removing all text files saves 8Mb (total: 594Mb).

I suggest we keep all texts files because the gain is not so much and there are issues if some text files are not found (e.g. README with the help module, INSTALL.txt file with drupal-scaffold).

Updating code in attached MR.

aaronmchale’s picture

Just for the record, I recommended in Slack that an issue be opened in the ideas queue because to me it seemed like there was potential for wider discussion around this topic. Which for the record I fully support! But who's to say we can't find more than one possible solution to tackle this?

The advantage of the ideas queue is that it gives a space early iteration and feedback before any work begins, in my opinion that should be open to all kinds of ideas. It also helps to gain momentum for an idea, because people are more likely to be watching the ideas queue.

Having said that, I see some good work is already being done, and as I'm in favor of this, I'm not going to stand in the way of this progressing! So, carry on!

matthieuscarset’s picture

Issue tags: -eco-friendly, -bash +sustainability

@AaronMcHale I totally agree. There is a broader topic of making Drupal more sustainable and this should be discussed within the community. Hoever I don't think this current issue is the best place to discuss such ideas. I see this script as a simple/specific addition to core.

I suggest we have conversation about sustainability within the dedicated project: https://www.drupal.org/project/sustainability

I tagged the issue respectively for reference.

mglaman’s picture

Status: Needs review » Needs work

Instead of a script, what about using .gitattributes to exclude non-API test classes? You can't just purge all test files. Then KernelTestBase would break.

This won't work regardless demo_umami usage.

matthieuscarset’s picture

@mglaman I wonder what you mean by "this won't work". Do you mean to run tests on production?

mglaman’s picture

Do you mean to run tests on production?

No, but when would this script run? If it's not excluded from the packaged Drupal archive then we're still downloading the amount of files. If it's a post-install with Composer script, we're deleting the files and cannot run tests.

How is this intended to run? Inside packaging of a deployment artifact for end users? Then it could just be document for folks on how to minimize container images and other artifacts.

borisson_’s picture

How is this intended to run? Inside packaging of a deployment artifact for end users? Then it could just be document for folks on how to minimize container images and other artifacts.

I think "just documenting" is not good enough, having good defaults is a thing we try to do everywhere in Drupal, so it should be the same for our sustainability efforts.

I think we could add this to core-recommended for example, since that is what sites use, and not what you'd use when trying to develop for drupal?

A much more aggressive solution could be https://github.com/dg/composer-cleaner. I tried this on a laravel project recently and that saved another ~50mb.

mgifford’s picture

With Drupal CMS the impact would be even bigger with all the contrib files. I do think this is worth exploring more. Would there ever be a need to produce a developer version (with the extra code), or is that just what you would find in Git?

And agreed @borisson_ it is about setting up good defaults for when everyone works to download Drupal (or Drupal CMS).

borisson_’s picture

How is this intended to run? Inside packaging of a deployment artifact for end users?

I have been thinking about this for a while, I'd like to think we can make this a dependency of core-recommended and not include it in core-dev. That way it can be a post-install script, however you are right there are some base classes we would need to keep around.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.