amazons3_file_stream_wrapper_uri_normalize_alter is making 100s of calls to curl_multi_exec to check validateBucketExists status [#2674374]

For every image load in the frontend of the site, amazons3_file_stream_wrapper_uri_normalize_alter hook is getting called and trying to validate if the bucket exists thus causing tons of load on web servers with php requests and external calls.

Has anyone faced this issue before with s3 implementation?

Comments

Comment #1

23 February 2016 at 23:06

subson created an issue. See original summary.

Comment #2

subson CreditAttribution: subson commented 23 February 2016 at 23:06

Title:

amazons3_file_stream_wrapper_uri_normalize_alter is making 100s of calls to curl_multi_exec to check validbucketexists status

» amazons3_file_stream_wrapper_uri_normalize_alter is making 100s of calls to curl_multi_exec to check validateBucketExists status

Comment #3

szantog CreditAttribution: szantog as a volunteer commented 7 March 2016 at 10:35

I can confirm the issue is valid. We just ran into at the last week.
Currently I'm trying to set up more sophisticated cache around rendered images using built-in cache of panels/views as a tmp solution.

I'm not sure, we should probably store the result ofS3Client::validateBucketExists($url->getBucket(), $s3, new \Drupal\amazons3\Cache()); in cache?

Comment #4

szantog CreditAttribution: szantog as a volunteer commented 8 March 2016 at 11:26

What about this?

-      S3Client::validateBucketExists($url->getBucket(), $s3, new \Drupal\amazons3\Cache());
+      $cid = 'validbucket: ' . $uri;
+      if (!cache_get($cid)) {
+        S3Client::validateBucketExists($url->getBucket(), $s3, new \Drupal\amazons3\Cache());
+        cache_set($cid, TRUE);
+      }

I'm not sure i understand the big picture of this validateBucketExists(). Any disadvantages?