We should be able integrate with the Purge module to automatically invalidate external cache entries for the replaced file after it has been replaced. While the cache tag for the file entity will already be invalidated by the purge module, it won't do anything, because external caches do not have the cache tags associated with static assets. This is because Drupal, the backend that would add the cache tags to the response, does not serve the static assets. The web server (Apache/Nginx) does.

So, we should be able to purge based on URL instead of cache tag.

This won't purge file assets stored in the browser. We cannot do anything about that, but instruct site owners to lower the cache lifetime.

CommentFileSizeAuthor
#18 media_entity_file_replace-file-cache-purge-3115477-18.patch1.29 KBsjpeters79
#16 media_entity_file_replace-file-cache-purge-3115477-16.patch1.05 KBsjpeters79

Issue fork media_entity_file_replace-3115477

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

bkosborne created an issue. See original summary.

PCate’s picture

PCate CreditAttribution: PCate commented

I'd love to see this added. I've been having to clear varnish caches much more than I'd like to because of updated document files.

bkosborne’s picture

bkosborne
Primary language English
Location New Jersey, USA
CreditAttribution: bkosborne at Princeton University commented
Related issues: +#3106363: Clear image cache after updating media image, +#2920332: Image style changes and purging

For images, we need to also worry about clearing cache of image style derivatives. We are already "flushing" the derivatives since #3106363: Clear image cache after updating media image, but that just clears Drupal's local copies. To clear the proxy cache copies, it makes more sense to concentrate our efforts in #2920332: Image style changes and purging because it's not an issue that's specific to this module.

bkosborne’s picture

bkosborne
Primary language English
Location New Jersey, USA
CreditAttribution: bkosborne at Princeton University commented

Maybe this should not be solved in this module at all. The most generic solution possible would benefit the community the best. Since we call ->save() on the file entity when we replace it, I imagine it would be better if there was some other module that handled purging files whenever they are saved like that.

bburg’s picture

bburg
he/him
Location Washington D.C.
CreditAttribution: bburg at Forum One commented

I was able to get this to implement this in a patch to this module. I write out the example in this issue: https://www.drupal.org/project/acquia_purge/issues/2926619#comment-13952821.

In my case, it's just a single document. Purging a whole set of image styles would likely need more than that.

bkosborne’s picture

bkosborne
Primary language English
Location New Jersey, USA
CreditAttribution: bkosborne at Princeton University commented

After looking at this again, I've gone back and forth on whether or not this file purge integration should be part of this module or not. Ideally it would not be part of this module, though it's a bit trickier.

The separate module would need to hook into file save, and determine if the file has actually changed or not (don't want to queue up purges if the file hasn't actually changed... but then again why would it saved otherwise?). If it has changed, then it could queue up a purge for the original filepath. Additionally if it's an image, it would need to check what image style derivatives exist for that image and queue up purges for those paths as well.

So the benefit of queuing the purges directly in this module is we always know that a file has changed and don't need to worry about hooks and determining if the file is an image or not. The downside is that people not using this module don't benefit from the purger.

I think we can at least start with integration directly in this module and maybe split out into a separate module if needed.

bburg’s picture

bburg
he/him
Location Washington D.C.
CreditAttribution: bburg at Forum One commented

The biggest problem for me here were PDFs, I had a client who needed to replace them frequently on their file system with versions that had new information. And this was becoming a big problem if they didn't remember to go in and clear the Acquia cache.

I noted that image styles would be tricky. I also don't know how much of a problem it would be to queue purges,depends on the site I suppose. I kind of got what I needed out of it, but a module that handled this for you would be a great idea if someone has the time to set that up.

douggreen’s picture

douggreen CreditAttribution: douggreen as a volunteer commented

Yes, a generic solution would be nice, a possible home for this is the https://www.drupal.org/project/filehash module, which is calculating hashes and knows if the file has changed. But I also think you should do it here, especially since you're already calling image_path_flush(), which is IMO an acknowledgment that something needs to be flushed, and you're doing it for image styles already.

douggreen’s picture

douggreen CreditAttribution: douggreen as a volunteer commented

See #3223983: Add integration with Purge module to purge cache of file path when the hash changes

bkosborne’s picture

bkosborne
Primary language English
Location New Jersey, USA
CreditAttribution: bkosborne at Princeton University commented

A new contrib module exists to purge files: https://git.drupalcode.org/project/purge_file

shelane’s picture

shelane
Primary language English
CreditAttribution: shelane commented

The link to the project page instead of the code page would be helpful here:
https://www.drupal.org/project/purge_file

DaleTrexel’s picture

DaleTrexel
Primary language English
Location Minnesota, USA
CreditAttribution: DaleTrexel as a volunteer and at University of Minnesota commented

+1 for purge_file module!

However, I'd argue that this behavior should be part of Purge and not require a separate module. I can't imagine a use case where someone running a Varnish cache would want to purge only Drupal-generated content with cache tags and not also static files (managed by Drupal) by URL.

The current Purge behavior (not handling static files) caused a lot of confusion and frustration by our content managers until we realized what was going on. Old, stale file content was remaining accessible by visitors (non-authenticated users) and indexed by search engines long after the files had been "deleted" by content managers. File cache issues were another reason that we stopped using the Media Entity File Replace module.

bkosborne’s picture

bkosborne
Primary language English
Location New Jersey, USA
CreditAttribution: bkosborne at Princeton University commented

I'm going to update the issue summary to point people towards the Purge File module for now. That module should help people that update documents. For images, there's still the issue where we need to purge image style derivatives too. There's still the open issue in the Purge module for that: #2920332: Image style changes and purging

sjpeters79 made their first commit to this issue’s fork.

sjpeters79 opened merge request !4

sjpeters79’s picture

sjpeters79 CreditAttribution: sjpeters79 commented
FileSize
media_entity_file_replace-file-cache-purge-3115477-16.patch1.05 KB
sjpeters79’s picture

sjpeters79 CreditAttribution: sjpeters79 commented

Added a patch that will invalidate the cache tags for the replaced file thus invoking drupal to purge the required files cache and the cache url for the file.

sjpeters79’s picture

sjpeters79 CreditAttribution: sjpeters79 commented
FileSize
media_entity_file_replace-file-cache-purge-3115477-18.patch1.29 KB