At now Drupal store many temporary and cache files mixed up together with site static content files (attached images, files, and other content) in files directory. So for configuring backups of site content (user attachments and other site content files) too hard to split them for exclude temporary and cache files from backup.

Here is list of most popular folders, that contain many generated files with cache category, that can be easily deleted and recreated at any time:

  • css - many minified css files
  • js - many minified js files
  • languages - generated js translations
  • php/twig - generated twig template files
  • styles - images, generated via image styles
  • translations - downloaded translations for projects

Some other non-core Drupal modules may use other folders in root of files folder, because there are no recommendations and examples from Drupal core where to store cached files.

I want suggest to move all those folders into cache subfolder, for easily split important content files and cache/generated files. And recommend to reuse same folder for cached and generated files to all contributed modules.

Most of projects splits content and cached data in separate folders for easy backup and cleanup!

So, let's split important content files and inconsiderable auto-generated files to separate folders in Drupal core too!

Comments

Murz created an issue. See original summary.

jasonluttrell’s picture

jasonluttrell
Location Washington, DC area
commented

I second this. I was actually trying to figure out how to do this. Does anyone know if it's currently possible? Basically, separate cache files from uploaded content?

murz’s picture

murz
Primary language English
Location Yerevan, Armenia
commented

For my websites I solve this problem manually via creating symlinks for needed folders to cache folder. For symlinks works well, you need also patch Drupal core: #1008402: Allow the use of symlinks within the files directory.

jonathanshaw’s picture

jonathanshaw
Primary language English
Location Stroud, UK
commented
Priority: Normal » Major

Nice idea. Maybe major in scope.

berdir’s picture

berdir
Primary language German
Location Switzerland
commented

There are at least two very different groups of files here, css/js/styles are generated, but I wouldn't say they are a cache. They are actually requested by browsers and at least css/js can't just be deleted without clearing caches in drupal too. And regenerating styles can be very expensive if you have a lot of files.

Also, at least changing styles is actually an API change IMHO, as it would imply changing routes which are required to change those files and might also require server configuration.

php/twig and languages on the other hand are internally and actually private files and they are only in files by default because we know that's available and writable, you could and IMHO should configure them to be outside of the webroot if you can.

So.. I don't think this is necessary/worth doing.

jonathanshaw’s picture

jonathanshaw
Primary language English
Location Stroud, UK
commented

So perhaps what we should say is:
sitebuilders are recommended to configure Drupal to put uploaded files in a subfolder like /files/uploads so they can identify them when they need to sync these files between environments.

If this is our solution to this issue, maybe it's worth considering how (via documentation and default settings for file fields) better encouraging this pattern.

murz’s picture

murz
Primary language English
Location Yerevan, Armenia
commented

The main idea is to split unique site content files, that are very important to store (and do good backups), and other files, that not important to store, and can be easily recreated via drupal cache:rebuild command.

We can name those files as cache, because they are created automatically on cache rebuild action, or on demand (for image cache). I don't see the problem with cache name, even when "They are actually requested by browsers and at least css/js can't just be deleted without clearing caches in drupal too.".

We don't need to hide those cache file from web access, only split them from important unique files to separate location. Now, when users create backup rules, there are too hard to split cached files from important.

Good solution will be create /cache/ folder in root of Drupal web folder, and move all cache (automatically generated files) to this folder, and store in /files/ folder only files, that uploaded manually by user.

For not broke previous behavior and upgrade paths, we can create symlinks from all old folder locations to new, symlinks are supported on all operational systems (linux, windows, macos).

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

karol haltenberger’s picture

karol haltenberger commented
Related issues: +#2670404: Add new stream wrapper(s) to store generated files separately

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.