I've got some uploaded files with 'weird' characters in the filenames, for example: filename(version 2).jpg
.
I can upload this file and view it via Drupal, use image styles on it etc, however, it's not served via the farfuture cdn feature.
This is because the farfuture code checks to see if the file exists, but does so based on the URL encoded version of the file path, however, it's not URL encoded on my filesystem.
FileUrlGenerator::generate
is where the file_exists
is called, but this might want to be fixed in ::getRelativeUrl
because that seems to be where the code knows it's coming from a stream wrapper (and generates the local path).
Comment | File | Size | Author |
---|---|---|---|
#20 | 2925507-20.patch | 14.16 KB | Wim Leers |
Comments
Comment #2
Wim LeersThanks for the crystal-clear bug report!
\Drupal\Tests\cdn\Unit\File\FileUrlGeneratorTest::testGenerate()
is testing "special" file names already. But\Drupal\Tests\cdn\Functional\CdnIntegrationTest::testFarfuture()
isn't yet. So we should be able to expand that test to trigger a failure.Comment #3
Wim LeersComment #4
Wim LeersI was wrong in #1,
\Drupal\Tests\cdn\Unit\File\FileUrlGeneratorTest::testGenerate()
only tests non-far-future cases. It's\Drupal\Tests\cdn\Unit\File\FileUrlGeneratorTest::testGenerateFarfuture()
that we should expand to test a "special" file name, because that contains the bug described in the last paragraph of the IS.But on top of that, we also should expand
\Drupal\Tests\cdn\Functional\CdnIntegrationTest::testFarfuture()
, but we should convert it to a proper integration test.Comment #5
Wim LeersFirst, let's fix the imperfect simulation of how
PublicStream
behaves. We have to mock that, becausePublicStream
can only ever be actually available in a kernel or functional test, whereas this is a unit test (which we want, because it's much faster).Comment #7
Wim LeersVoila, this shows that our unit tests weren't reflecting reality. That's of course the danger of unit tests. But in this case, they're worth the trouble. This is (or should be) the only flaw in them.
Comment #8
Wim LeersComment #9
Wim LeersNow that our unit tests do reflect reality, we can expand it with the missing unit test coverage.
This should fail.
Comment #11
Wim LeersThis is the failure:
We expected the file to be served using a farfuture URL, but that was not the case. This is exactly the bug that was reported. So now we have a regression test 👍
Comment #12
Wim LeersAnd this then finally updates the integration test coverage. Instead of testing with
public://druplicon.png
andcore/misc/drupal.js
, this changes that all topublic://druplicon ❤️.png
.This should cause another failure. And this means the regression test coverage is now complete.
Comment #14
Wim LeersOops, I re-uploaded the #9 patch in #12. The interdiff was right though.
Comment #16
Wim LeersGreat, those failures were expected! Now let's fix it.
The root cause lies in using relative URLs as paths, but relative URLs may look like paths, but they are of course URL-encoded. And that's what I overlooked here. I got this right when I ported farfuture functionality from D6 to D7 in #974350: Far Future setting for Origin Pull mode, but I got it wrong in the D8 port. My bad.
Comment #17
Wim LeersGreen, yay!
And now we can actually clean up the logic a little bit (because with the few added lines, it looks weird). Slightly out of scope, but doing this in a separate issue also seems silly. Also fixing the 2 coding standards violations that I accidentally introduced earlier.
Also, this definitely is a major bug. Nothing actually breaks, which is why it's not critical, and which is why it went unnoticed until now!
Comment #18
Wim LeersCan you please apply the patch, and confirm that it solves the problem? Thank you!
Comment #19
Wim LeersComment #20
Wim LeersSelf-review.
s/non-shipped/managed/
We should delete this file.
This code is unnecessary, we should delete it.
Comment #21
Wim LeersCredited Steven Jones.
Comment #22
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedSeems to work nicely!
Comment #24
Wim LeersGreat, thank you for testing!
Comment #26
Wim LeersTurns out the fix here was incomplete 😑 #3069516: Farfuture functionality + RFC3986-reserved characters = wrong URLs adds the finishing touch.