New "file_upload" REST resource plugin: allows file uploads of any size, creates File entities

By Wim Leers on 3 April 2018

Change record status:

Published (View all published change records)

Project:

Drupal core

Introduced in branch:

8.6.x

Introduced in version:

8.6.0

Issue links:

https://www.drupal.org/node/1927648

Description:

TL;DR

This adds the file_upload @RestResource plugin. If you configure it using https://www.drupal.org/project/restui, you'll be able to upload a file:

by POSTing the file's contents
to the path /file/upload/{entity_type_id}/{bundle}/{field_name}, which means that we're uploading a file to be used by the file field of the specified entity type+bundle, and the settings/validation constraints of that field will be respected.
… don't forget to include a ?_format URL query argument, this determines what format the response will be in
sending file data as a application/octet-stream binary data stream, that means with a Content-Type: application/octet-stream request header. (This allows uploads of an arbitrary size, including uploads larger than the PHP memory limit.)
and finally, naming the file using the Content-Disposition: file; filename="filename.jpg" header

Step 1: upload a file

So, a request will look like:

POST /file/upload/node/article/field_hero_image?_format=json HTTP/1.1
Content-Type: application/octet-stream
Content-Disposition: file; filename="filename.jpg"

[… binary file data …]

The response to such a request looks like this (omitting the less interesting details and with explanatory comments):

{
  "fid": [{"value": 345345}],
  "uuid": [{"value": …}],
  "langcode": …
  // The authenticated user that made this POST request will own the file.
  "uid": [{
      "target_id": 98,
      "target_type": "user",
      "target_uuid": …,
      "url": "/user/98"
  }],
  // If a file with the given name already exists, it'll get a number suffix automatically, just like elsewhere in Drupal core!
  "filename": [{"value": "filename_2.jpg"}],
  "uri": [
    {
      "value": "public://filename_2.jpg",
      "url": "/sites/default/files/filename_2.jpg"
    }
  ],
  "filemime": [{"value": "image/jpeg"}],
  "filesize": [{"value": 20011988}],
  // This is very important! The created file entity is NOT YET "permanent"! Only once an entity "uses" this file, it becomes permanent.
  "status": [{"value": false}],
  "created": …,
  "changed": …"
}

The response contains a serialized File entity: the one we just created.

Step 2: use (reference) the uploaded file

Now we want to "use" (reference) it. So we need to either POST a new article Node, or PATCH an existing one. Let's assume we need to create it:

POST /node?_format=json HTTP/1.1
Content-Type: application/json

{
  "type": [{"value": "article"}], // "type": [{"target_id": "article"}], since Drupal 9
  "title": [{"value": "Dramallama"}],
  // Note that this is using the file ID we got back in the response to our previous request!
  "field_hero_image": [
    {
      "target_id": 345345,
      "description": "The most fascinating image ever!"
    }
  ]
}

That should yield a 200 response, and voila: we uploaded a file, and used it in a file field!

Wondering about something?

If you wonder why it is designed to work this way, see the very detailed issue summary of #1927648: Allow creation of file entities from binary data via REST requests, which explains every key design decision!

Attachment	Size
File Upload Rest endpoint configuration	114.49 KB
File Upload : Postman Headers	107.24 KB
File upload: Postman Body as Binary and select the file from popup	95.49 KB
Postman File upload: Basic Authentication	127.96 KB
File Upload: Success Response 201	109.3 KB
New File entity is created	94.91 KB

Impacts:

Site builders, administrators, editors

Module developers

Themers

Updates Done (doc team, etc.)

Online documentation:

Not done

Theming guide:

Not done

Module developer documentation:

Not done

Examples project:

Not done

Coder Review:

Not done

Coder Upgrade:

Not done

Other:

Other updates done

Comments

X-CSRF-Token

malik.kotob commented 3 July 2018 at 16:16

For both steps 1 and 2, and seemingly any request that uses core's rest module that writes, an `X-CSRF-Token` needs to be included in the request.

What structure needs to be sent in [… binary file data …]

siddhant_chopra commented 14 September 2018 at 09:32

I have been using Drupal 8.6.1 and I'm trying to upload the files using restful api "file_upload" plugin mentioned above. But I'm finding a difficulty in sending image or a file to drupal via this plugin.
Please help me with the structure of [… binary file data …] , what data needs to be sent here.

You should send there file

gsin44 commented 25 September 2018 at 06:41

You should send there file byteArray.
for ex.
I use FileReader.readAsArrayBuffer(file from input type=file) and in callback func reader.result will be your [… binary file data …]
MDN Link
maybe it worked with FileReader.readAsText

Create file entity from js

gsin44 commented 25 September 2018 at 06:16

Hi, I try to create file entity from js using axios.
File creates succesfully, but when I try open it browser shows "The image 'my-file-path/name.jpg' cannot be displayed because it contains errors'. To get file data content I use js FileReader.readAsDataURL. What I'm doing wrong? Maybe I need to call FileReader.readAsArrayBuffer method instead?

I solved it with method

gsin44 commented 25 September 2018 at 06:43

I solved it with method FileReader.readAsArrayBuffer() and send reader.result as data in my request.

And to get the token you need

jackbravo commented 8 November 2018 at 18:08

And to get the token you need to do a GET to /rest/session/token. Reference: https://www.drupal.org/project/drupal/issues/2976542

UUID

hedeshy commented 13 December 2018 at 15:11

When sharing a file between multiple Drupal websites, it would be nice if they all share the same UUID. So, is there a way to set the UUID at the time we upload the file?

Paragraphs & file_upload

jerodfritz commented 22 December 2018 at 19:21

Can someone offer some advice if this is compatible with Paragraphs?:

I'm continually getting a 404 trying to POST to this resource for an Image field on a paragraph. For example: My paragraph type is called delivery with an image field named field_signature.

The requested URL /file/upload/paragraph/delivery/field_signature was not found on this server.

Similar response for Post's with correct header and binary data as described above result for the following urls as well:

/file/upload/paragraphs/delivery/field_signature
/file/upload/paragraphs_type/delivery/field_signature
/file/upload/entity/delivery/field_signature

Service unavailable

marctty commented 6 January 2019 at 23:06

Halo , i followed this rest api instruction, but i do not understant what i'm missing.
I receive the message 500 "Service unavailable"
Content-Type : application/octet-stream
X-Csrf-Token : XXXXXXXXXXXXXXXXXXXXXXXX
Content-Disposition: file; filename="filename.jpg"
Auth: User: admin Password : xxxxx
i used the Body with binary code between [] brackets
I'm sure that the api is enabled on REST UI as POST instruction.
thanks

Getting Error: administer users permission is required

stillfinder commented 11 January 2019 at 18:23

Hi, I'm getting {"message":"The \u0027administer users\u0027 permission is required."}

Request details:

POST {{url}}/file/upload/user/user/user_picture?_format=json

Headers
Content-Type:application/octet-stream
Content-Disposition:file; filename="file1.jpg";

I tried with user 1, but result the same.

Thanks for any help!

see this thread for help

taylormade415 commented 27 September 2019 at 21:21

see this thread for help https://www.drupal.org/forum/support/module-development-and-code-questio...

Hi all - thanks for the

katzilla

she/her

German

Göttingen, Germany

commented 24 January 2019 at 14:20

Hi all - thanks for the effort so far - it works great in my case, I can POST the images directly to drupal from my vue-frontend, when I have an authenticated user and then reference the returned FID as the user_picture target_id.
Now I am building the registration form in my decoupled frontend and it should have an image upload field for uploading the user avatar.
I was wondering how that could work - anonymous users are not allowed to upload images via REST. The POST request does not have credentials/a valid user or session at that point and so the answer from my resource is "No authentication credentials provided." - Does anyone have a strategy how to solve this problem? Or is there any documentation I did not find? Or is this use-case not covered by the file_upload resource?

Hi ! It seems that there is

potassium commented 28 April 2020 at 16:29

Hi ! It seems that there is an issue for this problem: https://www.drupal.org/project/drupal/issues/2983404#comment-13575955

I'm also trying to add a file field in my register form but it can't work because anonymous users are still not allowed to upload images via REST.

I only found a "hack" which is to not set "file/upload/user/user/user_picture" as endpoint but use an other entity and set the permissions example: "file/upload/node/article/field_image". that's very bad but is there a clean solution ?

Did you find something ?

REST file upload : The website encountered an unexpected error.

stefangas commented 21 March 2019 at 17:50

We are using Drupal 8.6.4.

Using Postman to test. Using Postman I can comfortably request a token, update (PATCH) user profile fields, retrieve standard & custom views etc.

Now I want to upload an image to the server and afterwards link that image to the user_picture field.

Following all available guidelines, I have the following setup that fails with the message:

The website encountered an unexpected error. Please try again later.

Using Basic Authorization, which succeeds for all the above mentioned calls.

POST to http://{{url}}/entity/file?_format=hal_json

Also tried for a couple of days to use http://{{url}}/file/upload/user/user/user_picture?_format=hal_json

headers: {  "Authorization": basic,
                        "X-CSRF-Token": "EBwOxoBUGs2arfOYGzTcfL-hsXQBsTnCQ4XU9h0aLzk",
                        "Content-Type":"application/octet-stream",
                        "Content-Disposition":'file;filename="test.jpg"'
                    }
data:
{
    "_links": {
        "type": {
            "href": "http://{{url}}/public"
        }
    },
    "filemime": [
        {
            "value": "image/jpeg"
        }
    ],
    "type": [
        {
            "target_id": "image"
        }
    ],
    "data": [
        {
            "value": "data:image/png;base64,iVBOR.............."
        } ]
}

Change user_picture via REST

tom.drupal commented 4 June 2019 at 09:23

I tried Content-type application/json and {base_url}/user/{user_id}?_format=json to patch user_picture. So far no luck.
has anyone been able to change the user_picture via Webservices?

any update on this sending

boby_ui commented 12 June 2019 at 02:47

any update on this sending the data via the raw binary broke the image? and cant seem to find a better documentations on how to upload

Try it with postman:Query

stf54 commented 14 June 2019 at 09:19

Try it with postman: POST request
Query params:
?_format=json
Headers:
Content-Disposition: file; filename="testing-api.jpg"
Content-Type: application/octet-stream
X-CSRF-Token: 'your token from login'
Body:
binary: select image to upload
URL: /file/upload/{entity_type_id}/{bundle}/{field_name}
Example with node type article:
file/upload/node/article/field_image?_format=json

turns out I solved as about

boby_ui commented 23 June 2019 at 06:28

turns out I solved as about submitting the rest upload with a readerarray of the file and it works just as intended.

paragraphs "file_upload"

stf54 commented 14 June 2019 at 09:54

Hi,
i'm trying this "file_upload" POST request with POSTMAN and it works correctly... The problem coming when i try to use it with paragraphs entity type.

I have created a postman request for "node/article/field_image" and it works. Then i duplicate the same request changing "paragraph/paragraph_type/field_paragraph_image" and I get a 403 Forbidden with empty message. In recent logs page i can only see 'access denied' with message '/test_project/web/file/upload/paragraph/p_worksheet/field_pw_image?_format=json'.

I tried this with the admin user.
Also i have posted this to paragraphs project: https://www.drupal.org/project/paragraphs/issues/3056253

Route not found error

Alireza.Tabatabaeian commented 27 August 2019 at 01:40

I'm using drupal 8.7.1, I've enabled this modules :

Hal
Http Basic Authentication
JSON API
REST UI
RESTful Web Services
Serialization
I also have set admin/config/services/jsonapi to

Accept all JSON:API create, read, update, and delete operations
The Problem is I always get the Route Not Found for my requests.

Other information about the request :

URL : http://testfileupload.dd:8083/file/uplaod/node/article/field_image?_form...

header:
Content-Type : application/octet-stream
Content-Disposition : file; filename="test.jpg"
X-CSRF-Token : 'TOKEN-Given-By-Drupal'

I'm using Postman for my test.

Fix your URL http:/

clemens.tolboom

Dutch

Groningen, 🇳🇱/🇪🇺

commented 28 August 2019 at 12:18

Fix your URL http://testfileupload.dd:8083/file/uplaod/node/article/field_image?_form...

It should be upload http://testfileupload.dd:8083/file/upload/node/article/field_image?_form....

Thanks for your reply

Alireza.Tabatabaeian commented 3 September 2019 at 06:25

That didn't work, actually the URL was correct in postman, I just had written it incorrectly here.

Can't get this to work! The

Joe Huggans

English

Kiel, Germany

commented 27 January 2020 at 23:29

Can't get this to work! The error is No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided.

But I am providing it.

CURLOPT_HTTPHEADER => [
      "Content-Disposition:file; filename='test.png'",
      "Content-Type: application/octet-stream",
      "authorization: Basic YWRtaW46dGfggfpbmRhMDEhIQ==",
    ],

The if statement causing this error is here -> https://api.drupal.org/api/drupal/core%21modules%21file%21src%21Plugin%2...

"No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided'"

But there is one! This surely is a bug?

maybe help you

Cangurin limpiezas

English

Zaragoza

commented 28 January 2020 at 01:54

i get this error and solve escaped the ' simbol
like "file;filename=\"a.jpg\"" in java

my problem is when i upload one file image via rest with basic auth , i upload one cero bytes file :(, any tip with this?

This happened to me when I

Joe Huggans

English

Kiel, Germany

commented 28 January 2020 at 14:10

This happened to me when I was using base_64 encode, I just sent the image without this and it worked for me.

I have changed the line to

Joe Huggans

English

Kiel, Germany

commented 27 January 2020 at 23:41

I have changed the line to

"Content-Disposition: file; filename=\"attachmedsfdsfnt.png\"\r\n",

But now I am getting an error No route found that matches "Content-Type: "application/octet-stream""

Ok this is definitely a bug,

Joe Huggans

English

Kiel, Germany

commented 27 January 2020 at 23:46

Ok this is definitely a bug, something to do with how the headers are parsed, since this works with the filename at the end.

CURLOPT_HTTPHEADER => [
      "Content-Type: application/octet-stream",
      "Cache-control: no-cache",
      "authorization: Basic YWRtaW46dfdsgdsgsthisisnotrealZpbmRhMDEhIQ==",
      "X-CSRF-Token:qlLwLBbLKfg908547jgfkjgflpwZq9FgYcg",
      "Content-Disposition: file; filename=\"attachmedsfdsfnt.png\"\r\n",
    ],

I'm confused on how we can

sadikyalcin commented 14 June 2021 at 16:41

I'm confused on how we can send the file binary in JS? Data being the `base64` string of the file. An empty image does get uploaded. So the request works but obviously I'm not sending the binary in the right manner. Could someone advise?

  let headers = {
    'Content-Type': 'application/octet-stream',
    'Content-Disposition': 'file; filename="' + _filename + '"',
    'X-CSRF-Token': _xscrf_token,
    Authorization: 'Bearer ' + _accessToken,
  };

  return api({
    method: 'post',
    url: uri + '?_format=hal_json',
    data: JSON.stringify(_data),
    headers: headers,
  });

Provided brief description

djassie commented 9 August 2022 at 18:01

Provided brief description with screenshot:

First: install restui plugin and activate the file endpoint

File Upload Rest endpoint configuration

Enable Basic authentication only for testing: and put authentication in postman

Basic Authentication

File Headers in Postman

File Header in Postman

Select Image as Binary

Select File as Binary in Postman

Successful Post response with 201

Successful Postman 201 response with fileid

See the Image in the /admin/content/files

Image in admin content files

webform file field

prompt-h commented 19 April 2023 at 17:35

Works for a node, but when I try the same for a webform file (like for example: `/file/upload/webform/contact/my_file?_format=json`, I get:

The website encountered an unexpected error. Please try again later.<br><br><em class="placeholder">LogicException</em>: Getting the base fields is not supported for entity type Webform. in <em class="placeholder">Drupal\Core\Entity\EntityFieldManager-&gt;buildBaseFieldDefinitions()</em> (line <em class="placeholder">226</em> of <em class="placeholder">core/lib/Drupal/Core/Entity/EntityFieldManager.php</em>). <pre class="backtrace">Drupal\Core\Entity\EntityFieldManager-&gt;getBaseFieldDefinitions() (Line: 346)
Drupal\Core\Entity\EntityFieldManager-&gt;getFieldDefinitions() (Line: 443)
Drupal\file\Plugin\rest\resource\FileUploadResource-&gt;validateAndLoadFieldDefinition() (Line: 243)
Drupal\file\Plugin\rest\resource\FileUploadResource-&gt;post()
call_user_func_array() (Line: 219)
Drupal\rest\RequestHandler-&gt;delegateToRestResourcePlugin() (Line: 87)
Drupal\rest\RequestHandler-&gt;handleRaw()
call_user_func_array() (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber-&gt;Drupal\Core\EventSubscriber\{closure}() (Line: 580)
Drupal\Core\Render\Renderer-&gt;executeInRenderContext() (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber-&gt;wrapControllerExecutionInRenderContext() (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber-&gt;Drupal\Core\EventSubscriber\{closure}() (Line: 169)
Symfony\Component\HttpKernel\HttpKernel-&gt;handleRaw() (Line: 81)
Symfony\Component\HttpKernel\HttpKernel-&gt;handle() (Line: 58)
Drupal\Core\StackMiddleware\Session-&gt;handle() (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle-&gt;handle() (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache-&gt;pass() (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache-&gt;handle() (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware-&gt;handle() (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware-&gt;handle() (Line: 23)
Stack\StackedHttpKernel-&gt;handle() (Line: 707)
Drupal\Core\DrupalKernel-&gt;handle() (Line: 19)
</pre>

How do I upload a file with JS via webforms?