This issue was recreated due to it being closed after committing to D8, even though a D7 patch was made available

Problem/Motivation

When drupal moves a file it issues a copy() and then an unlink() this causes a very significant amount of I/O. If the source and destination are on the same filesystem and rename() is issued instead then virtually no I/O occurs becuase filesystems can perform this operation by just changing a pointer on disk.

Proposed resolution

Instead of file_unmanaged_move() calling file_unmanaged_copy(), break out the common code into a new function file_unmanaged_prepare() and have file_unmanaged_move() attempt to issue a rename(), falling back to copy() and unlink() if this does not work.

This optimizes drupal_build_css_cache(), drupal_build_js_cache(). It also prevents a race condition with these functions where other processes or servers may start reading the files before they are fully written.

A further advanatage of issuing a rename() is that some remote schemes have implemented this method, e.g. it would be possible for a file to be moved to a different location on Amazon S3 without having to download it and re-upload it to the correct location.

Remaining tasks

Reviewing.

User interface changes

None.

API changes

None.

Original report by bvanmeurs

Hi, I'm using Drupal to host a huge photo archive (one node per file). I automatically create taxonomy tags for new directories, and files within those directories are automatically inserted as a node.

When a tag name is updated, I want to automatically change the directory name by creating a new directory, and move all of the files to the new directory. I do this using the function file_move.

When I update the root tag, containing all files (about 1000) it takes more than a minute to complete. I cachegrinded the request and found out that 99% of the resources were spent in the file_unmanaged_move function.

Researching the function, I found out that it simply copies the file and deletes it. I am working with hughe photo files (10mb each) so that means that about 10 gig of data is copied. That takes a whopping time of course, whereas a move would be very fast. I do not see any reason not to implement an efficient move function here. Just adding a parameter to the file_unmanaged_copy and using rename instead of copy would do the trick for me!

Any chance for a fix?

CommentFileSizeAuthor
#2 fast_move-3098526-2-D7.patch2.99 KBnwom

Comments

NWOM created an issue. See original summary.

nwom’s picture

StatusFileSize
new2.99 KB

The following patch was created by Anrikun here: https://www.drupal.org/project/drupal/issues/1377740#comment-9264011, specifically for the File Resumable Upload module.

chaseontheweb’s picture

poker10’s picture

Yes, it seems like a duplicate of that issue.