Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Performance during uploads can be quite poor. This is because _amazons3_get_object() is called multiple times on files that do not yet exist, resulting in misses on amazons3_file table, and causing API calls to S3 to check whether the file is there yet.
The simple solution is to add static caching in that method, preventing multiple API calls on the same file.
Patch forthcoming.
| Comment | File | Size | Author |
|---|---|---|---|
| #5 | 2265587-static-cache.patch | 3.07 KB | beeradb |
Comments
Comment #1
beeradb CreditAttribution: beeradb commentedI did some timing on an upload through media.module with this patch enabled. For this I disabled the media tab during testing to get rid of some noise and reduce the calls to _amazons3_get_object to only those related to uploads. Here are the results from my local install according to xhprof. Time is in microseconds:
With static caching:
First page: 1,264,149
Second page: 2,549,988
Without static caching:
First page: 2,377,830
Second page: 7,369,506
Note that this is just the time spent in _amazons3_get_object, not the entire request time. Furthermore, in this case "first page" is not the initial modal load, it's from when a user adds and image and hits the upload button. The 'second page' is completing the upload and saving the file.
Comment #2
beeradb CreditAttribution: beeradb commentedHere's a re-roll of it combined with #2259295: Catch integrity constraint violations thrown by DB Merge, this one can be ignored for testing purposes.
Comment #3
beeradb CreditAttribution: beeradb commentedTurns out we need a couple more cache resets.
Comment #4
beeradb CreditAttribution: beeradb commentedThis is currently the one to test. Could potentially be improved upon by only clearing the specified URI from the static cache on write operations.
Comment #5
beeradb CreditAttribution: beeradb commentedThe previous patch had some issues with the static cache for new directory creation within the request. Updating to properly handle that situation.
Comment #6
deviantintegral CreditAttribution: deviantintegral at Lullabot for NBCUniversal commentedThis patch looks reasonable to me, but I'm not doing any work on the 7.x-1.x branch. I just opened a PR for this against the 7.x-2.x branch over at https://github.com/justafish/drupal_amazons3/pull/17. If your site isn't implementing any of the module hooks, it should be a pretty straight upgrade to that branch.