Problem/Motivation

In #2940383: [META] Unify file upload logic of REST and JSON:API we are trying to re-use \Drupal\file\Upload\FileUploadHandler::handleFileUpload() However, that method takes a \Symfony\Component\HttpFoundation\File\UploadedFile as its argument.

Through investigation, we have found that UploadedFile is only really mean for multi-part form file uploads which set the $_FILES superglobal. The methods UploadedFile::isValid() andUploadedFile::move() call the PHP is_uploaded_file() and move_uploaded_file() methods which only work for files uploaded via forms.

We don't call UploadedFile::move() directly because \Drupal\file\Upload\FileUploadHandler::handleFileUpload() makes a call to our own \Drupal\Core\File\FileSystemInterface::moveUploadedFile(). However that does call move_uploaded_file() in its implementation.

On the otherhand, the REST and JSON API approaches do not use multi-part form file uploads. The POST the files as the body, and use the content-disposition header to specify the filename. We need to read the file contents from the php://input and save it to a temp directory ourselves, then move it to the destination directory once validation is complete.

Steps to reproduce

Proposed resolution

We can replace UploadedFile with a new UploadedFileInterface and pass that to FileUploadHandler::handleFileUpload(). This will allow us to have a bridge to the Symfony UploadedFile and allow us to create our own RawUploadedFile to handle raw uploaded files for JSON API and REST modules.

NB. GraphQL module is using Symfony UploadedFile so not sure how they get around those limitations.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

kim.pepper created an issue. See original summary.

larowlan’s picture

Priority: Normal » Major

This is a 9.3 target, so bumping to major

kim.pepper’s picture

Status: Active » Needs review
StatusFileSize
new9.71 KB

Initial patch for proposed solution.

dww’s picture

Issue tags: +Needs tests
StatusFileSize
new9.83 KB
new2.85 KB

Thanks for getting this going, @kim.pepper. Definitely important to land this quickly so we're not trapped.

Most of this code looks very familiar from reviewing #2940383: [META] Unify file upload logic of REST and JSON:API. It's all pretty solid and straight forward.

I found a bunch of tiny docs nits, which I'm fixing via a patch/interdiff instead of a long dreditor review. Also added return types for getPathname() and getFilename(). Can't cleanly do so for getRealPath() until we require PHP8 and can use a union type (e.g. string|FALSE).

My only concerns before RTBC:

  1. There's no new test coverage of this interface. I ran core/modules/file/tests/src/Kernel/FileUploadHandlerTest.php locally between changing UploadedFileInterface.php to add the return types before I fixed FormUploadedFile.php, hoping to see a fail. But that test passed, anyway. Probably we need some new tests in here before this is ready.
  2. There's nothing in here to show the reuse in a non-form case. Presumably that's follow-up material, but it'd be nice to know when/where that's coming, and probably add a @todo/@see to this patch so we all know what's coming. I.e. we've got this:
       * This method allows re-use of ::handleFileUpload() for implementations of
       * UploadedFileInterface that are not multi-part form uploads.
    

    But the method seems to directly assume a FormUploadedFile. Or maybe the intention is that NonFormUploadedFile::getRealPath() (for lack of a better name) will actually be what provides the magic to allow reuse? In that case, maybe the above comment is misplaced?

    My sense is that it's not gonna fly to commit this without proof that the reuse is actually possible, or at least a documented plan on when/where that's gonna happen.

But overall, this seems like a small, simple, important change so the new core/modules/file/src/Upload/FileUploadHandler.php in 9.3.x isn't locked in with a public API we can't reuse.

kim.pepper’s picture

StatusFileSize
new9.88 KB
new1.25 KB

Re: #4

  1. Not sure what additional tests we should add? There is extensive coverage of file uploads already in \Drupal\Tests\file\Functional\SaveUploadTest and sub-class.
  2. I might not have spelled out the intention clearly enough. I've updated the comment, added a todo and linked to the file field upload issue.

    The intention in #2940383: [META] Unify file upload logic of REST and JSON:API is to sub-class FileUploadHandler and override this method. Because both form and raw uploads are moved by calling services it made it hard implement that inside an UploadedFileInterface implementation. Happy for alternate suggestions!

kim.pepper’s picture

StatusFileSize
new9.88 KB
new655 bytes

Fix code style.

dww’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

Re: #5:

  1. Oh right! I thought the Kernel test changed in this patch was the one that would fail, but we're only touching it here to change how a mock is created. Breaking the real class wouldn't matter for a test that's only mocking the class. 🤦‍♂️ Actual testing of this class indeed happens at core/modules/file/tests/src/Functional/SaveUploadTest.php. When I re-broke FormUploadedFile in the way I described in #4.1 and ran that, I got a sea of errors. 🎉 Noice. Removing the tag.
  2. The updated comment is way better, thanks. That makes much more sense. You're deeper in this snake pit than I am, but on first glance, that strategy sounds good, and I don't see a cleaner alternative. Perhaps there's something, but delaying this issue for hypothetical improvements seems highly unwise.

Given that all I touched in here were nitpicky docs improvements, I think I'm still qualified to RTBC this. Since it's urgent to fix this before 9.3.0-alpha1, let's get it in. It gives us a level of indirection to help us get around limitations in various underlying systems that are standing in the way of a unified solution at #2940383: [META] Unify file upload logic of REST and JSON:API. We might not get that completely done before alpha deadline, but at least we won't paint ourselves into a corner with "generalizations" that only work with form uploads.

Hopefully core committers will agree that we're gonna need this, even if we're not actually using it just yet. 🤞

Thanks!
-Derek

larowlan’s picture

Status: Reviewed & tested by the community » Fixed

Fixed on commit

diff --git a/core/modules/file/src/Upload/UploadedFileInterface.php b/core/modules/file/src/Upload/UploadedFileInterface.php
index 2cbabc2159..b20b8490ad 100644
--- a/core/modules/file/src/Upload/UploadedFileInterface.php
+++ b/core/modules/file/src/Upload/UploadedFileInterface.php
@@ -10,11 +10,11 @@ interface UploadedFileInterface {
   /**
    * Returns the original file name.
    *
-   * It is extracted from the request from which the file has been uploaded.
-   * Then it should not be considered as a safe value.
+   * The file name is extracted from the request that uploaded the file and as
+   * such should not be considered a safe value.
    *
    * @return string
-   *   The original filename supplied by the client.
+   *   The original file name supplied by the client.
    */
   public function getClientOriginalName(): string;

(1/2) Stage this hunk [y,n,q,a,d,j,J,g,/,s,e,?]? y
@@ -35,7 +35,7 @@ public function isValid(): bool;
   public function getErrorMessage(): string;

   /**
-   * Returns the upload error core.
+   * Returns the upload error code.
    *
    * If the upload was successful, the constant UPLOAD_ERR_OK is returned.
    * Otherwise, one of the other UPLOAD_ERR_XXX constants is returned.

It took me a while to work out why this was needed, but then I realised that the move code in the sub-class in #2940383: [META] Unify file upload logic of REST and JSON:API is using FileSystemInterface;:moveand here we use FileSystemInterface::moveUploadedFile

Committed 384d615 and pushed to 9.3.x. Thanks!

  • larowlan committed 384d615 on 9.3.x
    Issue #3245244 by kim.pepper, dww: Make FileUploadHandler re-usuable for...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.