Currently i am using this module in a site with locale module active, and i getting this warning:
User warning: Stream is not seekable in Aws\S3\StreamWrapper->triggerError() (line 788 of /var/www/html/vendor/aws/aws-sdk-php/src/S3/StreamWrapper.php).
Error message
Warning: file_get_contents(public://languages/es_VDvIBCKufkGMU2tQ1_KFvYXnRgrAosuv8CW9BTOTRJQ.js): Failed to open stream: "Drupal\s3fs\StreamWrapper\PublicS3fsStream::stream_open" call failed in Drupal\system\Controller\JsAssetController->generateHash() (line 43 of core/lib/Drupal/Core/Asset/AssetGroupSetHashTrait.php).My current configuration:
$config['s3fs.settings'] = [
'bucket' => $bucket_name,
'region' => $bucket_region,
'use_customhost' => FALSE,
'use_https' => TRUE,
'use_s3_for_public' => TRUE,
'use_s3_for_private' => TRUE,
'presigned_urls' => TRUE,
'use_cname' => FALSE,
'no_rewrite_cssjs' => FALSE,
'torrents' => FALSE,
];
$settings['s3fs.use_s3_for_public'] = TRUE;
$settings['s3fs.use_s3_for_private'] = TRUE;
$settings['php_storage']['twig']['directory'] = "/tmp/php";
$config['system.file']['default_scheme'] = "s3";
$config['system.performance']['js']['preprocess'] = FALSE; // This is the hack
As you can see the only way to stop this warning is to disable the JS agregation.
Also Drupal warns me about the the .htaccess file doesn't exists in public://and private:// that's cause internally drupal use @file_get_contents() to verify file contents.
Note that all files exists in my bucket(languages/es_VDvIBCKufkGMU2tQ1_KFvYXnRgrAosuv8CW9BTOTRJQ.js, public://.htaccess and private://.htaccess).
I think we must implement a temporary download to local file for ready only operations that requires seek()
Can you guide to decide how to manage this so i can try to help to make some MR to fix this?.
| Comment | File | Size | Author |
|---|---|---|---|
| #16 | s3fs-add-checksum-validation-config-3573338-15.patch | 919 bytes | metallized |
Comments
Comment #2
metallized commentedComment #3
metallized commentedComment #4
metallized commentedComment #5
cmlaraI think we must implement a temporary download to local file for ready only operations that requires seek()I believe that should already be done by https://git.drupalcode.org/project/s3fs/-/blob/88e5265fa00d515bdf07b0bcc... (NOTE: the AWS SDK should only ever see s3:// based paths, public:// and private:// are aliased to their relevant path. This comes from #2496033: Using V3 of the AWS SDK.
Do you have a s3fs_stream_open_params_alter hook that is removing the seekable option?
Warning: file_get_contents(public://languages/es_VDvIBCKufkGMU2tQ1_KFvYXnRgrAosuv8CW9BTOTRJQ.js): Failed to open stream: "Drupal\s3fs\StreamWrapper\PublicS3fsStream::stream_open" call failed in Drupal\system\Controller\JsAssetController->generateHash() (line 43 of core/lib/Drupal/Core/Asset/AssetGroupSetHashTrait.php).My first question, is assuming this is a recent version of core, since these appear to be dynamically generated JS why are they not on assets://? After https://www.drupal.org/node/3328126 we stopped dealing with dynamic javascript creation. Is there some config somewhere that is forcing this onto
public://?Can you provide more steps to reproduce? IIRC Locale is part of a normal core standard profile install and I haven't seen this occur, though I don't usually deal with multi-lingual sites personally.
I've seen that when S3FS is enabled without core having ever been deployed, or no syncing of local files to s3fs. I imagine most sites usually disable the warning in those cases as it is irrelevant to s3:// storage.
Do they exist in the s3fs_file database table? The S3FS module relies on the database cache to determine if a file is present instead of calling out to AWS for every file requests, this could create a situation where the object exists in bucket however are not readable during normal operations.
IIUC
file_get_contents()doesn't usually seek, it is only when provided an offset (technically PHP declares its operation undefined on remote stream wrappers though the AWS SDK does make attempts to buffer the file). That does make this a bit interesting as to why a seek is occurring here in the first place and is making theComment #6
metallized commentedHey, thanks for your answer:
No
I have drupal 10.6.3 and never set
$settings['file_assets_path']in mysettings.php(the line is commented)I don't have yet, but maybe this is the reason: first and second, these should use the
assets://stream?, btw in default locale configjavascript.directoryvalue islanguagesYeah all of them exists also in database.
Comment #7
metallized commentedI think i've identified the root cause of this issue.
The problem is in the AWS SDK StreamWrapper, not in the S3FS module itself.
Root Cause:
When using S3-compatible endpoints (like Tigris), the AWS SDK's `openReadStream()` method sets `$command['@http']['stream'] = true`, which causes the SDK to attempt seek operations on a non-seekable HTTP stream (internally it tries to do seek operations on the stream, but Guzzle's HTTP stream is not seekable by default, causing the "Stream is not seekable" error).
Temporary Workaround:
Commenting out line 734 in `StreamWrapper.php` resolves the issue.
Proper Solution:
This needs to be fixed upstream in the AWS SDK. Should i report this issue to the AWS SDK repository?.
Configuration for Tigris/S3-Compatible Services:
The S3FS module may want to consider:
1. Documenting this limitation when using S3-compatible services
2. Providing a configuration option to disable streaming in the wrapper
3. Patching the AWS SDK StreamWrapper or creating a wrapper class that handles this
Comment #8
cmlaraA few lines below seeking is enabled on non seekable streams.
https://github.com/aws/aws-sdk-php/blob/5db21cd0e5c8a5b5344596649033acf8...
Here is a sample script to prove seeking:
The only aspects that doesn't test is:
Seeking very far into a file over a slow data connection, though I would expect that to be a blocking read.
The SDK is a little vague in it says it allows seeking back on data that has already been read, IIRC the SDK pulls the object from the begining, it it doesn't jump ahead so that the entire file should essentially be readable. Network speed simulation could help confirm/deny this.
https://github.com/aws/aws-sdk-php/blob/5db21cd0e5c8a5b5344596649033acf8...
I do recall early alpha releases of s3fs having some weird truncation on upload issues when the temporary storage was full, in theory something similar could happen here if the PHP temporary stream storage is full or not available (would need to investigate
CachingStream)Additionally I do somewhat recall that there multiple ways Guzzle would make connections I can't recall if it was a specific library was needed or not, perhaps this is something related to the connection type, though I would still expect the caching noted above to handle this.
Given my above comments I would hold off on reporting this to AWS. I'm not convinced yet that this is an SDK issue.
It is interesting that core is hard-coding the public:// path. Re-reading perhaps these are not not dynamically generated JS, or perhaps core missed them during the assets:// deployment, I'm not sure off hand. I'm not impressed that core has hard-coded the scheme, however that is an entirely different subject.
I'm going to postpone this on awaiting reproduction steps. Once you have those (preferably based on a Drupal Core Standard profile install) we can narrow down where this actually is.
You might try the sample script above( replace with the path to an object that exists in your bucket if that errors it does at least tell us something is different between your environment and my lab which would help narrow this down.
Comment #9
metallized commentedHey, i just realize that the initial error applies to all the files (not just
language/*.jsor.htaccess), so i don't think is something about locale module.I can reproduce the error following this steps:
1. Install drupal using standard profile (
composer create-project drupal/recommended-project drupal)2. Install sf3s module (
composer require drupal/s3fs)3. Modify
settings.phpto add bucket config:4. run
validateacction inadmin/config/media/s3fs/actions5. run
drush s3fs:refresh-cache6. run
drush cr7. run drush
php:eval "echo file_get_contents('public://.htaccess');"(this will fail with the reported error)8. comment out line
734ofvendor/aws/aws-sdk-php/src/S3/StreamWrapper.php9. run
drush php:eval "echo file_get_contents('public://.htaccess');"(it works)Drupal: 11.3.3
PHP: 8.3.6
Nginx: 1.24.0
PostgreSQL: 16.11
Comment #10
cmlaraThat we can work with. Unfortunately your experiences do not match my local lab tests or our CI testing results.
I'm more interested as to what happens at line 740, does it enter the if statement and provision a CachingStream for the response body or not. Step debugging would be best, at a minimal a
die('test point: Enabled caching"');should confirm if it is trying to provision caching.Comment #11
metallized commentedMaybe its cause im using nginx/php-fpm?
I can't debbuging rigth now (IDE configuration) but i tested the
diesuggestion, the code never enters theifstatement, once it get into$result = $client->execute($command);, the execution stops, no futher code is executed.Comment #12
cmlaraTake a look at https://www.drupal.org/project/s3fs/issues/3087363#comment-14091193 for a small snipit that can be added to the S3FS code to get some more in depth debug logs. You may be able to see some response data from the Object Store that explains the fault (streamWrappers do mask some errors by catching them and instead returning errors that they just couldn't complete the operation).
If there isn't something in those debug logs it would tend to lean towards an AWS issue.
Removing
$command['@http']['stream'] = true;will not be an acceptable solution (streaming is very important for performance) however it might at least give the AWS devs some clues on what they need to look into.Before opening an issue with the AWS SDK I would suggest adding the debug logs (they will likely need them too) and testing against an AWS S3 bucket, or at least a LocalStack development image to rule this out as a possible Tigris issue.
I'm not really qualified to say it is or isn't the cause. I wouldn't expect nginix FPM to be too different from Apache FPM, however I haven't tested that so it is indeed a variable difference between the two.
Comment #13
metallized commentedHey i got somthing like this:
So
GuzzleHttp\Psr7\Utils::hash()tries toseek()the$response->getBody()and fires the error.I tried to install the
awscrtmod in php and got something like this:With
awscrtmodule enable the error dissapears but also i can't get nothing from stream (is empty).I think i will patch the
aws sdk phpto remove the stream and use sink instead.Comment #14
cmlaraStack trace agrees with your report regarding stalling out at
execute().Closing this since there doesn't appear at the moment to be any indication this is caused by failures on S3FS or anything we can reasonably solve inside of S3FS. Strictly speaking yes S3FS could re-write the AWS
StreamWrapper::stream_open()method, however in S3FS we would want streaming to be present (it is partly why we don't have proxy support yet) and would want all of AWS Validators to still process (it appears this flaw is inside one one of the validators), we would essentially want the existing implementation.I wouldn't be surprised if the root cause of this is an API "breach" by Tigris that combined with a bug in the SDK (in such a case the SDK team might defer it to Tigris as IIUC they don't officially support non AWS targets).
I will mention #3513322: Backblaze B2 - Unsupported header 'x-amz-checksum-crc32' had some information regarding disabling checksums (was focused on uploads however IIRC there are also response processing options). Perhaps those related configs for the SDK may provide another alternative.
Comment #16
metallized commentedI think its valuable for
S3FSthe option to disabled check validations, i make this patch which solves my errors, also need to adding this to mysettings.php:$config['s3fs.settings']['response_checksum_validation'] = 'when_required';Thanks for your help, up to you what is next.