Allow creation of file entities from binary data via JSON API requests

I think this should actually live in a separate module, not JSON API itself.

I have a 2 proposed flows. The first takes advantage of JSON API and its concept of relationships routes which gets us a DX improvement over core REST, the second is a fallback for the case of required file fields which takes us back to core REST DX. Both can live side-by-side.

For an existing resource:

Request:

POST /jsonapi/entity_type/bundle/{uuid}/relationships/file_field

Content-Type: application/octet-stream
Accept: application/vnd.api+json

// binary data

Response:

200 /jsonapi/entity_type/bundle/{uuid}/relationships/file_field

Content-Type: application/vnd.api+json

// file entity

The above would only work for a resource which already exists and has an image field. The advantage over core REST with the above is that we can automatically upload the file, create the file entity and reference the file entity all in one request!

This of course will not work for creating resources with a required file field. In that case, I propose to fall back to the following flow:

Request:

POST /jsonapi/file/file

Content-Type: application/octet-stream
Accept: application/vnd.api+json

// binary data

Response:

200 /jsonapi/file/file

Content-Type: application/vnd.api+json

// file entity

2nd Request:

POST /jsonapi/entity_type/bundle

Content-Type: application/vnd.api+json

{
  // data with relationship to newly created file entity
}

Maybe, but not convinced about that just yet.

This would be yet another extension to the spec that we'd be putting in the module without a negotiation or advertisement mechanism. That's why I said it should be separate. I would be willing to be convinced otherwise.

This does not work because it doesn't allow for validating the file before uploading it.

Darn! It was so elegant. I think we're still fine though. /jsonapi/file/file becomes /jsonapi/entity_type/bundle/file_field. The spec is silent about URLs, so this is okay.

Updated the IS with the proposal.

Updating the 1st flow paths from /jsonapi/entity_type/bundle/relationships/file_field to /jsonapi/entity_type/bundle/file_field because the spec enforces that the documents to and from relationships must contain only resource identifiers not resource objects.

FYI, there's already the JSON API File module: https://www.drupal.org/project/jsonapi_file

Might be a good chance to pull in the code from that module and deprecate it.

+1

Should this be postponed until #2940383: [META] Unify file upload logic of REST and JSON:API lands, which is now active?

Should this be postponed until https://www.drupal.org/project/drupal/issues/2940383 lands, which is now active?

As much as that makes sense, I don't trust the core process to get that service in, in time. So, I'd be fine if this issue remained open and we created our own, temporary service.

@gabesullice so JSON API would create its own temporary service, and simply swap it out for the one #2940383: [META] Unify file upload logic of REST and JSON:API provides, once it is completed? @e0ipso or @gabesullice, do we know what kind of timeline JSON API is working with to try to close this out? Trying to make a decision on how to proceed with supporting remote file upload for a client site since both #2983404: [PP-1] RESTs FileUploadResource plugin is only able to check `create` access to a parent entity, should be able to check `edit` also and #2940383: [META] Unify file upload logic of REST and JSON:API are outstanding. If there are no plans to fix this issue in the next few weeks, I can either:

1. Attempt to write a custom rest plugin of my own that essentially copies what was done in #2940383: [META] Unify file upload logic of REST and JSON:API, but takes in the entity as a parameter. This may not be a very big lift, but would duplicate core's code.

2. Leverage the jsonapi_file module. My understanding is that this route is not desirable since that module will become obsolete once the current issue is fixed, and it introduces a dependency on the file_entity module.

What're your thoughts on the best route to take?

@gabesullice so JSON API would create its own temporary service, and simply swap it out for the one #2940383: Create "file upload" service that core REST, JSON API and GraphQL can all use provides, once it is completed?

That's my hope!

@e0ipso or @gabesullice, do we know what kind of timeline JSON API is working with to try to close this out?

This is high on my list, and @Wim Leers' too... but you could make it higher! If you post a patch, I can almost guarantee you'll have reviews from either @Wim Leers or I within a business day :)

I would say that you should do something like #1, but instead, make the service we need out of the existing plugin :P If this isn't committed in time for you, you can still pretty trivially make a REST plugin of your own that will wrap it. You'll be no worse off.

The only way I think that this doesn't land in JSON API within a few weeks is if there's a specification incompatibility that we can't work around. But in THAT case, we'll just move the code to a contrib module (maybe jsonapi_file 2.x), and you'll still be good.

So, starting that service is definitely the best way forward.

Alrighty @gabesullice, this patch introduces the temporary service to JSON API but doesn't actually create the necessary endpoints. It might be quicker for you or @Wim Leers to take on that part as opposed to providing me guidance on how to do it. Feel free to provide feedback on what should change with the service.

P.S. Since it doesn't actually introduce the JSON API endpoints, I actually hacked core's file upload rest resource plugin to point to this service to test it and was able to upload a file successfully.

Thanks @malik.kotob! This is a great start!

Here's my very rough, "first reaction" review:

I could be wrong about lots of things below :P

1. +++ b/jsonapi.services.yml
   @@ -136,6 +136,9 @@ services:
   +  jsonapi.file.upload:
   

   Let's mark this service as "private" with: public: false

2. +++ b/jsonapi.services.yml
   @@ -136,6 +136,9 @@ services:
   +    class: Drupal\jsonapi\FileUploadHandler
   

   We should probably change this from "Handler" to something else. "Handler" has a specific meaning WRT to entities.

   Not sure what yet though.

3. +++ b/src/FileUploadHandler.php
   @@ -0,0 +1,491 @@
   +namespace Drupal\jsonapi;
   

   I wonder if we should add a namespace like Drupal\jsonapi\Shim.

4. +++ b/src/FileUploadHandler.php
   @@ -0,0 +1,491 @@
   +class FileUploadHandler {
   

   Needs docblock and to be marked @internal

   Perhaps we need an interface? That could be added later though.

5. +++ b/src/FileUploadHandler.php
   @@ -0,0 +1,491 @@
   +  public function handleUploadForField($entity_type_id, $bundle, $field_name, $filename) {
   ...
   +  protected function streamUploadData() {
   +    // 'rb' is needed so reading works correctly on Windows environments too.
   +    $file_data = fopen('php://input', 'rb');
   

   I think this method would be more useful if it accepted a stream
   like the one fopen() returns.

   Then handleUploadForField would become handleUploadForField($entity_type_id, $bundle, $field_name, $filename, $stream)

6. +++ b/src/FileUploadHandler.php
   @@ -0,0 +1,491 @@
   +    $access_result = $entity_access_control_handler->createAccess($bundle, NULL, [], TRUE)
   +      ->andIf($entity_access_control_handler->fieldAccess('edit', $field_definition, NULL, NULL, TRUE));
   

   What do we need to do to this service to handle the case of an existing parent entity?

I love seeing progress on this issue!

1. Done
2. I changed it to FileUploadService, what do you think?
3. Done (I think that makes sense)
4. Done (with FileUploadServiceInterface created as well)
5. Done (not sure if I implemented it as you envisioned though)
6. I was brainstorming a couple of solutions here:

a) add the entity as a parameter, if it’s present, check access at the field level (that doesn’t change), and also check access against the entity via EntityAccessControlHandler::access(), passing along the entity and the operation (update), if it’s not present, continue to check access using EntityAccessControlHandler::createAccess() and the field-level check

b) add a parameter that specifies whether or not to perform an access check, if the value is true, we’ll do the check as is, if its not true (which would be how jsonapi/rest/graphql are more likely to use it), they can perform the access check themselves. The resource specific to files being created against new entities would likely do the access check against createAccess, and the resource specific to files being created against existing entities would do the check against access

What do you think?

Here's a patch with some fix ups after I tested (again, using core's file upload rest resource plugin). I also went ahead and incorporated option 6a here just to get something going. With it, I was able to successfully upload a file against the current user entity.

This is awesome :)

Some initial thoughts:

1. +++ b/src/Shim/FileUploadService.php
   @@ -0,0 +1,509 @@
   +  public function handleUploadForField($entity_type_id, $bundle, $field_name, $entity = NULL, $filename, $stream = NULL) {
   

   Let's make this a lot simpler, I think it can just receive FieldDefinitionInterface $field_definition, $filename, $stream with no optional params.

   JSON API, REST and GraphQL can do all the figuring out WRT to which field definition and how to check access.

   This is just a file upload handler after all.

2. +++ b/src/Shim/FileUploadService.php
   @@ -0,0 +1,509 @@
   +  protected function streamUploadData($stream = NULL) {
   

   Let's rename this to something like readFileData($stream) (no optional param).

3. +++ b/src/Shim/FileUploadService.php
   @@ -0,0 +1,509 @@
   +    $file_data = isset($stream) ? $stream : fopen('php://input', 'rb');
   

   Instead of the optional parameter, let's add a new public static method on the service: getUploadStream() which returns fopen('php://input', 'rb').

I went ahead and took those thoughts and put them into code.

Mainly, I wanted the file uploader service to just have a single method that does just what it's named. Convenience functions for access checking and validation etc are not on the interface and are public+static, so they can be used even if someone swaps out the upload handler (that should make it easier to handle uploads to s3, for instance).

Let me know what you think.

More thoughts:

We might want to get rid of HTTP-specific exceptions. For example, we could change UnprocessableEntityHttpException into an InvalidArgumentException (since the $file argument failed validation). If this is a generic file upload handler, then I think we need to let JSON API/REST/GraphQL map those exceptions into whatever makes sense in context. I'm not convinced of that though.

The more I think about this, the more I think it belongs in a submodule, not the main module.

I didn't like the owner of the file being optional, I don't think that makes much sense.

One thing I didn't document and is not clear in the previous interdiff is that I removed calls to fclose($file_data). Since we're passing that in, I think it's the responsibility of the caller to close the stream when it's done with it.

+++ /dev/null
@@ -1,509 +0,0 @@
-          fclose($file_data);
...
-          fclose($file_data);
...
-      fclose($file_data);
...
-    // Close the input stream.
-    fclose($file_data);

+++ b/src/Shim/FileUploader.php
@@ -0,0 +1,458 @@
+  /**
+   * Gets a standard file upload stream.
+   *
+   * @return resource
+   *   A stream ready to be read. Do not forget to close the stream after use
+   *   with fclose().
+   */
+  public static function getUploadStream() {

Awesome work @gabesullice!

RE #22:

1. This makes sense to me, good call
2. Makes sense
3. Makes sense

We might want to get rid of HTTP-specific exceptions. For example, we could change UnprocessableEntityHttpException into an InvalidArgumentException (since the $file argument failed validation). If this is a generic file upload handler, then I think we need to let JSON API/REST/GraphQL map those exceptions into whatever makes sense in context. I'm not convinced of that though.

This actually makes sense to me, but it seems like it may be a good fit to perform that refactoring in #2940383: [META] Unify file upload logic of REST and JSON:API.

RE #23:

I agree on both of these, both good catches!

Separately, I found a minor leftover "optional" comment in the doc block for a parameter on a function. Went ahead and a attached a new patch that has that fix.

How do we move forward from here? I think what's left is:
1. Deciding if this needs to get moved into another module
2. Adding in the two jsonapi endpoints that actually invoke this service

Is there anything else?

Good catch!

How do we move forward from here? I think what's left is:
1. Deciding if this needs to get moved into another module
2. Adding in the two jsonapi endpoints that actually invoke this service

1. Given #3 + #5 + #8, which I had forgotten about, I think this needs to be in a submodule.
2. #3 + #5 describes what I think would be the ideal set of routes for this. @malik.kotob do you want to try that, or would you rather @Wim Leers or I take that on?

Is there anything else?

Not that I can think of off the top of my head.

I'm working on #2953346: Define related/relationship routes per field, not dynamically (with route parameters that need validating) ATM. That will change how relationship field access is handled and will make access checking here even easier. So I think we can do everything except access handling for now.

Okay great, @gabesullice! If you're able to provide some guidance/best practices on how to define the routes and make it compliant with the spec, I can take a crack at this.

WRT to spec compliance, there's not a lot to do... because the spec doesn't define anything about file uploads :P

But, #3 mentions a few request and response headers that I think we should use. Those are pretty close in spirit to the spec.

For the response to the upload request, after you've created/saved the file entity in code, I'd make a subrequest for the file entity, then just update the self link in the response.

To get started, I'd look at this module's jsonapi.routing.yml and src/Routing/Routes.php. Just look at the routes and getEntryPointRoute methods to begin with (ignore line 107).

If you get started with the most basic thing, we can build it up from there.

This patch is very very very rough :) It does two things:
1) Pulls out the code from previous patches into a submodule
2) Introduces the new route for existing resources mentioned in #3.

I tested that the file uploader service was still in good shape after I finished step 1, but don't actually have the route code working. I'm posting this work-in-progress because my bandwidth is limited at the moment, so I'm leaving it here in the event that someone else can pick it up and run with it.

Alrighty, here's another rough patch, but with more progress.

Highlights:
- Added the controller for file upload requests for file uploads to existing entities
- Added a service provider to register 'application/octet-stream' as a known (bin) format. Core does this in 8.6.x, but we probably can't/don't want to add in that dependency here.

Outstanding:
- Add the routes and corresponding controller code for files being uploaded to new entities.
- Add in the code to handle subrequests so that a file being uploaded to an existing entity can be taken care of in one fell swoop (I guess this doesn't have to be MVP, but it's up to you all!)
- Address any comments from a future review :D

THIS IS AMAZING PROGRESS! Nice work!

1. diff --git a/jsonapi_file_upload/jsonapi_file_upload.info.yml b/jsonapi_file_upload/jsonapi_file_upload.info.yml
   

   Let's move this to a modules directory

2. +++ b/jsonapi_file_upload/jsonapi_file_upload.info.yml
   @@ -0,0 +1,7 @@
   +description: Provides JSON API endpoints for uploading files.
   

   I try to avoid the word "endpoint"

   "Provides JSON API file upload support"

3. +++ b/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,133 @@
   +    /** @var \Drupal\jsonapi_file_upload\Shim\FileUploader $fileUploader */
   +    $fileUploader = $this->fileUploader;
   

   I don't think this is necessary.

4. +++ b/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,133 @@
   +    $entity_type_id = $resource_type->getEntityTypeId();
   ...
   +    $entity = $request->get($entity_type_id);
   

   Nit: Let's just do this in one line.

5. +++ b/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,133 @@
   +      $reason = $access_result->getReason() ?: 'Access Denied';
   

   You'll need to check $access_result instanceof AccessResultReasonInterface.

6. +++ b/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,133 @@
   +    fclose($stream);
   

   The Go language has a defer keyword which I always miss for this kind of thing.

7. +++ b/jsonapi_file_upload/src/JsonapiFileUploadServiceProvider.php
   @@ -0,0 +1,23 @@
   +/**
   + * Adds 'application/octet-stream' as a known (bin) format.
   + */
   +class JsonapiFileUploadServiceProvider implements ServiceModifierInterface {
   

   😘👌

   Itty bitty nit: add @internal

8. +++ b/jsonapi_file_upload/src/Routing/Routes.php
   @@ -0,0 +1,175 @@
   +    assert(is_string($jsonapi_base_path));
   +    assert($jsonapi_base_path[0] === '/');
   +    assert(substr($jsonapi_base_path, -1) !== '/');
   

   Let's not worry about these assertions, we're already doing it in the main module.

9. +++ b/jsonapi_file_upload/src/Routing/Routes.php
   @@ -0,0 +1,175 @@
   +  public function routes() {
   

   This method looks perfect 👍

10. +++ b/jsonapi_file_upload/src/Routing/Routes.php
    @@ -0,0 +1,175 @@
    +   * @param string $path_prefix
    +   *   The root path prefix.
    

    Nit: root vs base vs prefix

11. +++ b/jsonapi_file_upload/src/Routing/Routes.php
    @@ -0,0 +1,175 @@
    +      $individual_file_upload_route = new Route("/{$path}/{{$entity_type_id}}/relationships/{file_field_name}");
    

    As per above, this should be defined to be /{$base_path}/ {{$entity_type_id}}/{filed_field_name}

    (so, missing relationships)

12. +++ b/jsonapi_file_upload/src/Routing/Routes.php
    @@ -0,0 +1,175 @@
    +      $individual_file_upload_route->setRequirement('_csrf_request_header_token', 'TRUE');
    

    As of #2953346: Define related/relationship routes per field, not dynamically (with route parameters that need validating), JSON API has its own custom relationship field access check that we can use :)

    So we can add:

    $individual_file_upload_route->setRequirement(RelationshipFieldAccess::ROUTE_REQUIREMENT_KEY, $relationship_field_name);

13. +++ b/jsonapi_file_upload/src/Routing/Routes.php
    @@ -0,0 +1,175 @@
    +      // Resource routes all have the same controller.
    +      $routes->addDefaults([RouteObjectInterface::CONTROLLER_NAME => 'jsonapi_file_upload.request_handler:handle']);
    +      $routes->addPrefix($path_prefix);
    

    Let's move these to routes()

Okay, I've addressed the above feedback and completed the outstanding items mentioned in #29, with the exception of the subrequests feature. Gabe and I discussed and feel that one is not necessarily MVP and can be postponed to a future ticket.

This patch now provides 2 routes, one that accepts file upload requests given an existing entity to upload to, and one that accepts requests without the existing entity (with the understanding that a new entity will be created). The route for existing entities is defined at: /jsonapi/entity_type/bundle/{uuid}/{file_field_name}. The route for new entities is defined at: /jsonapi/entity_type/bundle/{file_field_name}.

Sorry for the noise, but now that this issue doesn't address the subrequest portion, that means we're no longer technically updating an entity with a file. We're now only creating new file entities. As such, I've attached a new patch that updates the method on the route that accepts an existing entity to POST from PATCH.

Just moving the module to a modules directory. No other changes.

@malik.kotob realized and discussed the fact that we need to do entity validation all over the place with me and suggested moving that out to a trait. Done.

Assigning to myself for a few more changes to come.

EDIT: ignore this interdiff, it has uncommitted changes in it.

Proper interdiff for #34.

Just code standards and cleanup.

Here's my though process and explanation for the attached changes.

P.S. I realize my above interdiffs result in broken code. They're mostly split up in a way that's easy to see what was being changed. This patch should make it all workable again.

I think next up is a set of good tests.

1. +++ b/modules/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,139 @@
   +  public function handle(Request $request, ResourceType $resource_type) {
   
   +++ b/modules/jsonapi_file_upload/src/Routing/Routes.php
   @@ -0,0 +1,175 @@
   +    $routes->addDefaults([RouteObjectInterface::CONTROLLER_NAME => 'jsonapi_file_upload.request_handler:handle']);
   

   Instead of using the same controller method for both new and existing resources, let's make dedicated ones.

2. +++ b/modules/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,139 @@
   +    $access_result = FileUploader::checkFileUploadAccess($this->currentUser, $field_definition, $entity);
   

   Let's do the access checking before anything else.

3. +++ b/modules/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -0,0 +1,139 @@
   +      $reason = 'Access Denied';
   

   Let's make this message a little more helpful to the user.

4. +++ b/modules/jsonapi_file_upload/src/Routing/Routes.php
   @@ -0,0 +1,175 @@
   +      $individual_file_upload_route = new Route("/{$path}/{file_field_name}");
   ...
   +      $individual_file_upload_existing_entity_route = new Route("/{$path}/{{$entity_type_id}}/{file_field_name}");
   

   Let's rename these to new_resource_file_upload_route and existing_resource_file_upload_route.

5. +++ b/modules/jsonapi_file_upload/src/Routing/Routes.php
   @@ -0,0 +1,175 @@
   +      $routes->add(static::getFileUploadRouteName($resource_type, 'individual.post'), $individual_file_upload_route);
   ...
   +      $routes->add(static::getFileUploadRouteName($resource_type, 'individual.post'), $individual_file_upload_existing_entity_route);
   

   Let's also rename these using that "new" and "existing" language.

6. +++ b/modules/jsonapi_file_upload/src/Routing/Routes.php
   @@ -0,0 +1,175 @@
   +      $routes->addOptions(['parameters' => [$entity_type_id => ['type' => 'entity:' . $entity_type_id]]]);
   

   A cool trick we can do here, since this is new development, is name the parameter "entity" and not vary it per entity type.

   That will let us add it as a method argument in the controller.

Rebased.

Adding subrequests and fixed an entity validation issue. Also, a variable name typo.

1 new CS violation and a leftover line.

Awesome job, @gabesullice!! I tested the patch from #40 on user entities with success. I haven't had a chance to code review yet, but as you mentioned in chat earlier, this could really use a review from @Wim Leers and/or @e0ipso.

Hey guys! Great progress on this. Just wanted to wanted to make you aware of a bug we experienced earlier today when testing jsonapi_file_upload.

On line 367 of src/Shim/FileUploader.php there is the following line:

    if (!$this->systemFileConfig->get('allow_insecure_uploads') && preg_match(FILE_INSECURE_EXTENSION_REGEX, $filename) && (substr($filename, -4) != '.txt')) {

which relies on the FILE_INSECURE_EXTENSION_REGEX constant. Unfortunately that constant is undefined in core < 8.6, and results in the following error:

Warning: preg_match(): Delimiter must not be alphanumeric or backslash in Drupal\jsonapi_file_upload\Shim\FileUploader->prepareFilename() (line 367 of /mnt/www/html/---/docroot/modules/contrib/jsonapi/modules/jsonapi_file_upload/src/Shim/FileUploader.php) #0 /mnt/www/html/---/docroot/core/includes/bootstrap.inc(582): _drupal_error_handler_real(2, 'preg_match(): D...', '/mnt/www/html/r...', 367, Array) #1 [internal function]: _drupal_error_handler(2, 'preg_match(): D...', '/mnt/www/html/r...', 367, Array) #2 /mnt/www/html/---/docroot/modules/contrib/jsonapi/modules/jsonapi_file_upload/src/Shim/FileUploader.php(367): preg_match('FILE_INSECURE_E...', '58309e68-1e98-4...') #3 

So perhaps to be backwards compatible we need to check to see if the constant is defined, and if not define it?

Thanks @btully! I was not aware. Great catch!

I think by the time this makes it into JSON API (in the 2.x branch) 8.6.0 will be the minimum supported version.

That was an oversight, but this issue got delayed long enough for it to not be a problem, oddly enough :)

Hello,

Thanks all for the work done here.

As it is impossible to apply patch from comment 40 on 2.0.0-beta1, I have rebased the patch against the last dev version.

Finally on a project using JSON API (but not Entity share...).

Hello again! Thanks @Grimreaper!

Flow 2 OK! \o/

Can't do Flow 1 because on the media entity the file field is processed and required.

curl -X POST \
  http://XXX/fr/jsonapi/media/settlement/field_media_file \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Basic XXX' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Disposition: attachment; filename=\"XXX.pdf\"' \
  -H 'Content-Type: application/octet-stream' \
  -H 'Postman-Token: 362968b8-1528-4580-b3a8-76d75d0c2263'

Then

curl -X POST \
  http://XXX/fr/jsonapi/media/settlement \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Basic XXX' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/vnd.api+json' \
  -H 'Postman-Token: dda55410-d2ff-4418-a398-501fc851a642' \
  -d '{
  "data": {
    "type": "media--settlement",
    "attributes": {
        "status": true,
        "name": "Règlement 1 (JSON API)"
    },
    "relationships": {
		"field_media_file": {
		    "data": {
		        "type": "file--file",
		        "id": "cd9815bb-119c-4046-a2a4-26277fd511c0",
		        "meta": {
		            "display": null,
		            "description": null
		        }
		    }
		}
          }
	}
  }'

Maybe see you guys at Drupal Europe next week!

That's awesome.

Unfortunately, I don't think anyone from the team is gonna be there :(

Is the PATCH/UPDATE of file entities supported?

PS: I have seen the work on the documentation pages. Great!

Is the PATCH/UPDATE of file entities supported?

I'm not sure what you mean by that. Can you describe it in more detail?

Using the BO I think it is impossible.

But programmatically on the project I was for months, we had to do custom imports (absolutely not related to JSON API or webservices).

Depending on a file hierarchy with naming rules on a path on the server, if the client (or customer, not a client as in client/server :)) updated a file, if there was an already existing file entity in Drupal, we updated this file entity and we did not crreate a new one.

So I am asking, is it possible to update an existing file entity using JSON API? Or should a new file entity be created each time?

I was testing this patch uploading image media and I came across this error message:

ArgumentCountError: Too few arguments to function Drupal\jsonapi\JsonApiResource\JsonApiDocumentTopLevel::__construct(), 1 passed in /var/www/html/web/modules/contrib/jsonapi/modules/jsonapi_file_upload/src/Controller/RequestHandler.php on line 145 and at least 2 expected in Drupal\jsonapi\JsonApiResource\JsonApiDocumentTopLevel->__construct() (line 51 of /var/www/html/web/modules/contrib/jsonapi/src/JsonApiResource/JsonApiDocumentTopLevel.php) 

#0 /var/www/html/web/modules/contrib/jsonapi/modules/jsonapi_file_upload/src/Controller/RequestHandler.php(145): Drupal\jsonapi\JsonApiResource\JsonApiDocumentTopLevel->__construct(Object(Drupal\file\Entity\File)) 

#1 [internal function]: Drupal\jsonapi_file_upload\Controller\RequestHandler->handleFileUploadForNewResource(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\jsonapi\ResourceType\ResourceType), 'field_media_ima...') .....

The curl command I was using:

curl -X POST \
  http://XXXX/jsonapi/media/image/field_media_image \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Basic XXX==' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Disposition: attachment; filename=\"test1.png\"' \
  -H 'Content-Type: application/octet-stream' \
  -H 'Postman-Token: 17dc46b4-3603-405b-9d4f-8dfa43ed83b1'

The attached patch fixed the issue for me. Please review and let me know if something looks incorrect.

@pfrilling, thanks! You're correct that it needs an array of links. However, it will need a self link. See EntityResource::buildWrappedResponse() for an example :)

Thanks for the feedback @gabesullice! Attached is a new patch and interdiff. I think I did this correctly. Let me know if this looks okay.

I forgot to mark this as needs review earlier.

+++ b/modules/jsonapi_file_upload/src/Controller/RequestHandler.php
@@ -142,7 +143,9 @@
+    $links['self'] = LinkManager::getRequestLink($request);

getRequestLink is not a static method. So the link manager service will need to be injected and called with an arrow ->.

Thanks again for the feedback.

Attached is a new patch. Hopefully I have it correct now.

👌 looks great @pfrilling!

Would you like to try your hand at #42.5?

Hello,

I now have another use case. Due to the document size, files will be placed in the public/private stream wrapper folder by a process (not important). And I need to create the corresponding file entities.

I tried the following requests without success:

curl -X POST \
  http://XXX/fr/jsonapi/file/file \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Basic XXX' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/vnd.api+json' \
  -H 'Postman-Token: a5194d0c-34bd-4083-ad4f-aab68653f51e' \
  -d '{
  "data": {
    "type": "file--file",
    "id": "MY_ID",
    "attributes": {
        "filename": "test.txt",
        "status": true
    }
  }
}'

Returns a 403 forbidden, I think due to the permissions on file entity as there is no explicit "create" permission on the file entity.

And

curl -X POST \
  http://XXX/fr/jsonapi/media/settlement/field_media_file \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Basic XXX' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/vnd.api+json' \
  -H 'Postman-Token: 49c6a648-d38a-4ba8-a9b6-32dcf5549cae' \
  -d '{
  "data": {
    "type": "file--file",
    "id": "MY_ID",
    "attributes": {
        "filename": "test.pdf",
        "uri": {
            "value": "public://test.pdf",
            "url": "/sites/default/files/test.pdf"
        },
        "status": true
    }
  }
}'

Returns a 415 Unsupported media type.

Same result if I try on jsonapi/media/settlement/UUID/field_media_file endpoint.

So, it is no more possible to create a file entity without posting data?

@Grimreaper, does your first example request work without this patch?

@gabesullice,

Without the patch:

• the first request is still a 403
• the second request is now a 405 method not allowed

So it seems it was not possible even before the patch.

Hello,

@Wim Leers thanks for the reply.

I am trying to create a file entity without file uplaod because the physical file has been already placed in the file system.

But if it can't be done, it is ok, I will find a solution for the project.

Another thing I realized, with either flows, it seems that it is impossible to create a file entity with a predefined UUID. Or I am missing something?

I think for Entity share, I will have problems with that, because currently on JSON API 1.x, using deserialization I can create file entities with predefined UUID, but with JSON API 2.x and the need to make subrequest, I don't know if it will still be possible. I will see when I will be confronted to the problem.

The file is placed using a FTP connexion, there is no Drupal code involved in this process.

So yes, I think I will need to implement a custom endpoint to create those file entities.

Thanks for your replies.

@Grimreaper, check out https://www.drupal.org/project/jsonrpc for that custom solution.

Thanks @gabesullice for the suggestion.

I will give it a look. I do not garantee to use it, but at least I will think more about it in the future.

Hello,

Rebasing patch from comment 60 to be able to apply it on version 8.x-2.0-beta2.

Edit: the only conflict was in the imports of jsonapi/src/Controller/EntityResource.php

1. +++ b/modules/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -154,9 +154,11 @@ class RequestHandler {
   -    $links['self']['href'] = $this->linkManager->getRequestLink($request);
   +    // @todo Remove line below in favor of commented line in https://www.drupal.org/project/jsonapi/issues/2878463.
   +    $links['self']['href'] = Url::fromRoute('jsonapi.file--file.individual', ['entity' => $file->uuid()])->setAbsolute(TRUE)->toString(TRUE)->getGeneratedUrl();
   

   I'm not sure I agree that this link should be the link for the individual file entity. Remember, the resource object under the data key will have its own self link that is the same as this one.

   This self link is on the document level and should be the URL of the requested resource.

2. +++ b/modules/jsonapi_file_upload/src/Controller/RequestHandler.php
   @@ -154,9 +154,11 @@ class RequestHandler {
   -    return new ResourceResponse(new JsonApiDocumentTopLevel($file, new NullEntityCollection(), $links), '200', []);
   +    return new ResourceResponse(new JsonApiDocumentTopLevel($file, new NullEntityCollection(), $links), 201, []);
   

   200 -> 201 | Good catch!

@e0ipso, do you also agree?

I don't have a strong preference. I can see benefits one way or the other. My vote goes to whatever the majority think.

I do believe it should be a separate module which needs to be enabled. My preference is a to have it as a submodule as it already is in the patch. However, if @Wim Leers feels strongly that it should be in contrib, I'm fine w/ that too.

Unless we're saying we want to add both modules to core.

Can core modules have submodules?

🙈

Okay... hm.

I would not like JSON API to be in core and have file uploads only supported by a contrib module.

That means our only choice would be between trying to get two modules into core, or to merge this into JSON API itself.

So... maybe we should make it part of the main module. Previously, I said:

[File upload support] would be yet another extension to the spec that we'd be putting in the module without a negotiation or advertisement mechanism. That's why I said it should be separate. I would be willing to be convinced otherwise.

That means I'm now arguing against myself. What's changed?

In some respects, this actually lives "outside" the spec, because the HTTP request Content-Type header is not the JSON API media type. The HTTP response part of the upload flow is within the spec though, but the response is actually spec-compliant.

Given those facts, I'm not too worried that we'll end up in conflict with the spec... only that we may end up competing with the spec if the spec maintainers come up with an alternative pattern (I think this is very unlikely). In that case, we'll have to implement both which is a poor DX.

Last point: the next version of the spec actually does have a negotiation method now (it didn't when I wrote that comment). That means that unlike before, we'd have a way to support two different file upload strategies if we wanted to.

I think that in #90 + #91 we agreed to move this into jsonapi itself?

Yes, this should be moved into the main module.

@Wim Leers, you're absolutely right! The current code is just vestige of the past and a reminder how long this issue has been open.

Back in July, @malik.kotob and I had this conversation:

Malik Kotob [9:54 AM]
okay, that makes sense, so the difference between this new route and the existing ones is then just the trailing `/relationships/field_file`

so i guess that should really be `/relationships/{field_name}`

Gabe Sullice [9:55 AM]
That would be a good start :slightly_smiling_face: In the end, I think we'd want to inspect the resource type and make routes for each file field (but don't worry about that for now)
I _just_ did a similar thing here: https://www.drupal.org/project/jsonapi/issues/2953346
Mar 15th
so, let's let that land before worrying about it for files

The issue I linked to was: #2953346: Define related/relationship routes per field, not dynamically (with route parameters that need validating)

And it has already landed! So it's time to do that here as well :)

1. +++ b/src/Controller/FileUpload.php
   @@ -0,0 +1,249 @@
   +      }, (array) $violations->getIterator()));
   

   Let's use iterator_to_array($violations) here. Technically, getIterator() isn't part of the interface.

2. +++ b/src/ForwardCompatibility/FileUploader.php
   @@ -0,0 +1,460 @@
   +    // Validate the file entity against entity-level validation and field-level
   

   Nit: s/-//g

3. +++ b/src/ForwardCompatibility/FileUploader.php
   @@ -0,0 +1,460 @@
   +    // Firstly, check the header exists.
   

   Ubernit: s/Firstly/First

4. +++ b/src/ForwardCompatibility/FileUploader.php
   @@ -0,0 +1,460 @@
   +      throw new BadRequestHttpException('"Content-Disposition" header is required. A file name in the format "filename=FILENAME" must be provided');
   ...
   +      throw new BadRequestHttpException('No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided');
   ...
   +      throw new BadRequestHttpException('The extended "filename*" format is currently not supported in the "Content-Disposition" header');
   ...
   +          $this->logger->error('Temporary file data for "%path" could not be written', ['%path' => $temp_file_path]);
   +          throw new HttpException(500, 'Temporary file data could not be written');
   ...
   +      $this->logger->error('Temporary file "%path" could not be opened for file upload', ['%path' => $temp_file_path]);
   +      throw new HttpException(500, 'Temporary file could not be opened');
   
   +++ b/tests/src/Functional/FileUploadTest.php
   @@ -0,0 +1,800 @@
   +    $this->assertResourceErrorResponse(404, 'Field "field_rest_file_test_invalid" does not exist', $invalid_uri, $response);
   ...
   +    $this->assertResourceErrorResponse(400, '"Content-Disposition" header is required. A file name in the format "filename=FILENAME" must be provided', $uri, $response);
   ...
   +    $this->assertResourceErrorResponse(400, 'No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided', $uri, $response);
   ...
   +    $this->assertResourceErrorResponse(400, 'No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided', $uri, $response);
   ...
   +    $this->assertResourceErrorResponse(400, 'No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided', $uri, $response);
   ...
   +    $this->assertResourceErrorResponse(400, 'The extended "filename*" format is currently not supported in the "Content-Disposition" header', $uri, $response);
   

   Nit: need trailing stops (.)

5. +++ b/src/ForwardCompatibility/FileUploaderInterface.php
   @@ -0,0 +1,46 @@
   + * This is implemented at the field-level for the following reasons:
   

   Nit: s/-//

6. +++ b/src/ForwardCompatibility/FileUploaderInterface.php
   @@ -0,0 +1,46 @@
   + *   - Permission to upload a file can be determined by a users field level
   

   Nit: s/users/user's/

7. +++ b/src/ForwardCompatibility/FileUploaderInterface.php
   @@ -0,0 +1,46 @@
   +   *   The owner of the file. Defaults to the current user. Note, it is the
   

   Remove: "Defaults to the current user" since it's not an optional arg.

Only nits... and they're all fixed! RTBC!

But let's not commit this until 2.0 stable. Also, still needs a CR.

#118:

1. Sure!
2. Honestly, I was just doing it for consistency, but I think I did it wrong! Turns out, if it's a "single concept" modifying a noun, then it's hyphenated.

   So, "field level access" should be "field-level access", but "implemented at the field level" is correct, because "field" is modifying "level".

   Sources: https://www.grammarbook.com/punctuation/hyphens.asp and https://english.stackexchange.com/questions/113422/how-to-use-hyphens-ap...

   I added a "corrected correction" interdiff which shows just the hyphen changes to #116.

   TIL: @Wim Leers is better and englishing than me.

I applied this patch and tried to test it but it did not work for me: Here is my post request exported as curl from postman

curl -X POST \
  http://d8.vdev/jsonapi/node/article/9cff7586-deb8-4f5f-8fe6-dc05cf839ec0/field_image \
  -H 'Content-Type: application/octet-stream' \
  -H 'Postman-Token: b63d2e10-6347-4090-a89e-8c44ab68d0e4' \
  -H 'X-CSRF-Token: K4M-4LmQYkKt0gx9fvxCGDmvT9AmhF2BZGOFq5oAi_c' \
  -H 'cache-control: no-cache'

and request body is a binary file.

The response I get is

{
    "data": null,
    "jsonapi": {
        "version": "1.0",
        "meta": {
            "links": {
                "self": {
                    "href": "http://jsonapi.org/format/1.0/"
                }
            }
        }
    },
    "links": {
        "self": {
            "href": "http://d8.vdev/jsonapi/node/article/9cff7586-deb8-4f5f-8fe6-dc05cf839ec0/field_image"
        }
    }
}

But nothing gets attached to article with uuid 9cff7586-deb8-4f5f-8fe6-dc05cf839ec0. Am I missing anything?

I tried adding the accept header but as when I do I get a message "Could not get any response"

Ok tested this again on vannila drupal "8.6.4-dev " + jsonapi "8.x-2.0-rc2 " with the patch applied. here is my command line curl request:
note: cookie, csrf token, and path to file must be obtained and changed before you can try this your

  curl -X POST \
  http://d8.vdev/jsonapi/node/article/307a5965-6a20-4f19-a928-2663a3e05d9a/field_image \
  -H 'Accept: application/vnd.api+json' \
  -H 'Content-Type: application/octet-stream' \
  -H 'X-CSRF-Token: lBndnnbaeScxg1nRGFCxbsUq7xFyL-klXJ3mwyEj95w' \
  --trace-ascii debug.text \
  --data-binary @/path/to/file/sample.jpg \
  --cookie "SESS31b6c7d842b10fcad1130ff4da73e525=1hkBCEQ5PzsZ3b1VbCy0GjYqpnCOALlB-Glx9J8VO9Q; path=/; domain=.d8.vdev; HttpOnly; Expires=Fri, 28 Dec 2018 20:09:37 GMT;"

The result I get in apache access log is

"POST /jsonapi/node/article/307a5965-6a20-4f19-a928-2663a3e05d9a/field_image HTTP/1.1" 400 - "-" "curl/7.54.0"

and

in the error log I get;

[Wed Dec 05 17:12:06.429200 2018] [php7:notice] [pid 1404] [client 192.168.44.1:49995] Uncaught PHP Exception InvalidArgumentException: "Class "jsonapi.file_upload" does not exist." at /var/www/d8/web/core/lib/Drupal/Core/DependencyInjection/ClassResolver.php line 24

Tested this again and it looks like content-despostion header is requried otherwise I get a 400 bad request.

Here is what worked for me on vanilla drupal core + jsonapi + patch:

curl -X POST \
  http://d8.vdev/jsonapi/node/article/307a5965-6a20-4f19-a928-2663a3e05d9a/field_image \
  -H 'Accept: application/vnd.api+json' \
  -H 'Content-Type: application/octet-stream' \
  -H 'Content-Disposition: attachment; filename="sample5.jpg"' \
  -H 'X-CSRF-Token: lBndnnbaeScxg1nRGFCxbsUq7xFyL-klXJ3mwyEj95w' \
  --trace-ascii debug.text \
  --data-binary @/path/sample5.jpg \
  --cookie "SESS31b6c7d842b10fcad1130ff4da73e525=1hkBCEQ5PzsZ3b1VbCy0GjYqpnCOALlB-Glx9J8VO9Q; path=/; domain=.d8.vdev; HttpOnly; Expires=Fri, 28 Dec 2018 20:09:37 GMT;"

I am successfully uploading files from REACT app via the /file/upload endpoint and attaching it to the content node using the /jsonapi/node/[entity_type]/[entity_uuid]/relationships/[field_name] end point without this patch. I'm confused then as to what this patch is going to be doing. I thought it was adding the binary file upload capability - but this is working with out any patches for me. I've tested with different file types and sizes < 100Meg including MP4 video files.

Just did a clean test on a new install so I could be certain this was all that was needed - see attached image of the modules and versions installed.

Thanks @Win for clarifying!

Hi everybody,

In jsonapi_file it is possible to set the file UUID along with the file data. How to do it in this approach?

I mean, maybe we can have another flow for such a case.

Flow 3 (creating a new entity with defined uuid)

Request:

POST /jsonapi/entity_type/bundle/file_field/{uuid}

Content-Type: application/octet-stream
Content-Disposition: file; filename=image.jpg
Accept: application/vnd.api+json


// binary data

@hedeshy, can you open a feature request like this: [PP-1] Support client-generated UUIDs on file uploads and then mark it Postponed and related to this issue? I don't think that that feature needs to be in this patch.

Regarding your proposal, I think GET parameters would be a more fitting solution than a path parameter. So /jsonapi/entity_type/bundle/file_field/{uuid} would become /jsonapi/entity_type/bundle/file_field?id={uuid}. OTOH, it would be worth researching the Content-Disposition header to see if it could support a UUID, since that is already the mechanism in place for setting the file's metadata.

EDIT: I did a (very quick) bit of research. It looks like the ABNF for Content-Disposition parameters would allow for setting the file entity's UUID, like so:

POST /jsonapi/entity_type/bundle/file_field

Content-Type: application/octet-stream
Content-Disposition: file; filename=image.jpg; resource-id={uuid}
Accept: application/vnd.api+json

// binary data

Hi,

Thank you for the #119 patch, but after the upgrade to the 8.x-2.0-rc4, I found this error.

after doing some diff and merge using Diffmerge.
this is the patch that I used after updating to 8.x-2.0-rc4.

This patch fails on 8.5 (results) and 8.7 (results)

This should fix this for 8.7 and 8.6 (I haven't tested it on 8.5, perhaps it will help there too). Also, I added a missing piece of a docblock and Accept: application/vnd.api+json to FileUploadTest::fileRequest().

In #79, we decided not to support file uploads on Drupal core < 8.6. Typically, we support 8.5 and above on the 2.x branch. However, this is a completely new feature, so we're not dropping support for 8.5.

We didn't previously have a problem with this because we had all this in a submodule which required >8.6 in its dependencies.

I wrote a bunch of version checks and skipped tests if < 8.6 is detected. I also changed the version detection mechanism in Routes.php from this:

    if (!defined('FILE_INSECURE_EXTENSION_REGEX')) {
      return $routes;
    }

to this:

    if (\Drupal::VERSION < 8.6) {
      return new RouteCollection();
    }

That's what we do everywhere else. What this check does is prevent any file upload routes from being generated on 8.5 and below.

IMHO, if @Wim Leers or @e0ipso agrees, this can be committed.

Hello everybody,

I'm trying to apply the patch https://www.drupal.org/files/issues/2019-01-08/2958554-139.patch , but when I run composer update, I get: "Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2019-01-08/2958554-139.patch". Am I doing anything wrong?

Thanks for any help!

P.S.: I've added the following lines to my composer.json:

"drupal/jsonapi": {
        "Allow creation of file entities from binary data via JSON API requests": "https://www.drupal.org/files/issues/2019-01-08/2958554-139.patch"
      }

#145: Thank you for your response, but when I'm trying #140 I get the same error:

Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2019-01-09/2958554-140.patch

Added to composer:

"drupal/jsonapi": {
        "Allow creation of file entities from binary data via JSON API requests": "https://www.drupal.org/files/issues/2019-01-09/2958554-140.patch"
      }

Maybe I need another patch for my Drupal Version 8.5.7 ?
Also, my JSON API Version is 8.x-2.0-rc1

#147 Thanks a lot for your answer, it helped to apply #139 patch. But I have faced with another problem - I'm getting 405 Method Not Allowed in response.

I have the user_picture(machine name) field, with the field type: Image. Maybe I miss something in my request. Or maybe it's because of my Drupal Version 8.5.7 ?

Request:
POST {{url}}/jsonapi/user/user/4d861a6a-8f69-45e5-a06b-9fd25b207cb9/user_picture

Content-Type: application/octet-stream
Accept: application/vnd.api+json
X-CSRF-Token:fPvqqR4UHrK_YuGO2oYHLkW-kKefmZMzuHqMGG0uYCI
cache-control: no-cache

Body: binary

Response

{
    "errors": [
        {
            "title": "Method Not Allowed",
            "status": 405,
            "detail": "No route found for \"POST /jsonapi/user/user/4d861a6a-8f69-45e5-a06b-9fd25b207cb9/user_picture\": Method Not Allowed (Allow: GET, HEAD)",
            "links": {
                "via": {
                    "href": "https://theoptimalme.local.thebrain4web.com/jsonapi/user/user/4d861a6a-8f69-45e5-a06b-9fd25b207cb9/user_picture"
                },
                "info": {
                    "href": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.6"
                }
            }
        }
    ],
    "jsonapi": {
        "version": "1.0",
        "meta": {
            "links": {
                "self": {
                    "href": "http://jsonapi.org/format/1.0/"
                }
            }
        }
    }
}

Thank you for any help.

@stillfinder, see comment #139 where we say that Drupal 8.5 is not supported. You'll have to update to Drupal 8.6. Running uncommitted patches can be a bit of a pain. If you get it working, we'd like to know how it works for you :)

#149 Thank you! It works with 8.6.
But, is there any older patch that works with Drupal 5.8 ?

@Wim Leers - Thank you for your response!

Hi, I've found that after update the jsonapi from 2.0-rc1 to 2.0 and apply the patch #139 the following request {{url}}/jsonapi/user/user?filter[uid]=1&fields[user--user]=name returns the next:

{
    "errors": [
        {
            "title": "Bad Request",
            "status": 400,
            "detail": "Invalid nested filtering. The field `uid`, given in the path `uid`, does not exist.",
            "links": {
                "via": {
                    "href": "https://domain.com/jsonapi/user/user?fields%5Buser--user%5D=name&filter%5Buid%5D=1"
                },
                "info": {
                    "href": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1"
                }
            }
        }
    ],
    "jsonapi": {
        "version": "1.0",
        "meta": {
            "links": {
                "self": {
                    "href": "http://jsonapi.org/format/1.0/"
                }
            }
        }
    }
}

but before, the response was:

{
    "data": [
        {
            "type": "user--user",
            "id": "0a8c8f12-c592-41ce-801e-cc5c7e7e7bf5",
            "attributes": {
                "name": "Admin"
            },
            "links": {
                "self": {
                    "href": "https://domain.com/jsonapi/user/user/0a8c8f12-c592-41ce-801e-cc5c7e7e7bf5"
                }
            }
        }
    ],
...

@Wim Leers thank you, I'll do

Hi I recently included this patch in a site I'm working on and am currently running into issues, I was wondering if anyone might be able to point me in the right direction.

I'm generating a CSRF token first with /rest/session/token

Then I'm sending a POST request to the file field which looks like:

curl -X POST \
http://d8.dev/jsonapi/media/image/field_image_file \
  -H 'Accept: application/vnd.api+json' \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/octet-stream' \
  -H 'Content-Disposition: attachment; filename="con-amore.jpg"'
  -H 'Authorization: Basic xxxx' \
  -H 'X-CSRF-token: orUgAL8fGOlu7T3ydSK5NsWnzr2FbMGX7CV1n74eVqc' \
  --trace-ascii debug.txt \
  --data-binary @/Users/steveworley/Downloads/Con-Amore.jpg

The user that I'm trying it with is an administrator and has all permissions. The error I'm getting is:

The controller result claims to be providing relevant cache metadata, but leaked metadata was detected. Please ensure you are not rendering content too early. Returned object class: Drupal\\jsonapi\\ResourceResponse.

I'm sure I've just missed something.

One other question I had was - it looks that the field has to be "file" in Drupal and image fields are not supported. Just wondering if I missed something there as well.

Thanks for your help!

Is there anyone successfully create a file entity by POSTING file to file/file endpoint or update an existing file entity by PATCHING to file/file/uuid endpoint?
With "Content-Type: application/octet-stream", I got code "415"/error "Unsupported Media Type" for both POST and PATCH methods (8.x-2.x+core 8.6.x).

Server side log:
Symfony\Component\HttpKernel\Exception\UnsupportedMediaTypeHttpException: No route found that matches "Content-Type: application/octet-stream" in Drupal\Core\Routing\ContentTypeHeaderMatcher->filter() (line 49 of drupal8/core/lib/Drupal/Core/Routing/ContentTypeHeaderMatcher.php).

Still having this issue in D8.8
Request
Content-Type | application/octet-stream
Content-Disposition | file; filename="Testing1.jpg"

Response: 415Unsupported Media Type

Moving to Drupal core's issue queue.

I'm working on https://www.drupal.org/project/drupal/issues/3122113