Add support for multipart/form file upload

I have attach a patch to add multipart form support. It doesn't allow uploading but with the change the attaching of files to nodes(other entities) can be done in postprocess or preprocess functions. I using the module with this patch to do this for the node/create call.

I plan to create a sandbox project that shows how this can be done.

I have attached a very very basic "services_upload" module. This shows that with the above patch you can use postprocess functions to upload files to a field. I have included a basic html file that can be used to hit the service endpoint.

You would have to edit the action attribute in the html form.

After working with Aaron for a bit, this is a much better patch

oops better patch.

Nice patch.

Just some initial comments re code

foreach($_FILES['files']['name'] as $field_name => $file_name) {

should be

foreach ($_FILES['files']['name'] as $field_name => $file_name) {

$file = file_save_upload($field_name, $validators ,file_default_scheme() . "://");

should be

$file = file_save_upload($field_name, $validators, file_default_scheme() . "://");

if($file->fid) {

should be

if ($file->fid) {

} else {
     return services_error(t('An unknown error occured', 500);
}

should be

} 
else {
   return services_error(t('An unknown error occured', 500));
}

One question is why are we still passing in the file object as it is no longer used as far as I can tell, shouldn't the signature of resource be amended?

Also I can't get tests on d7 to pass but this might just be the wonders of windows! But it appears as if the tests are sending the data multipart encoded.

Updated patch, attempts to fix file tests, but doesnt, will look into it later.

Updated patch with fixes test, Thanks Aaron

+++ b/resources/file_resource.inc
@@ -115,53 +108,28 @@ function _file_resource_definition() {
+  foreach ($_FILES['files']['name'] as $field_name => $file_name) {

You should only create one resource per POST - not multiple ones imho.

Overall dislike that we should introduce support for something as complicated as multipart/form.

There's splendid support for streaming uploads which are very suitable for something like file creations. For some reason someone decided to not use it and now there's no reference for how to use it. It's however able to stream the data straight from the request, through authentication mechanisms and down to disk. I'm attaching a patch showing how to use it in a controller. However - more code would probably need to be modified for it to work.

Powered by Dreditor.

I much prefer the multi part approach given that the alternative is being implemented at a server level which seems wrong, especially given that we can offer a common resource for all services using the multipart solution.

@marcingy: I'm not sure I understand?

The streaming approach I'm suggesting makes the REST Server use an ordinary parser for handling the files - just like it would with any other resource with any other arguments. The REST Server is always responsible for parsing request data it receives - why should this be an exception?

We can _not_ offer a common solution for all servers by doing multipart handling. Eg. the XML-RPC server packages all arguments within XML-envelopes and can't and shouldn't utilize multipart requests to send arguments outside of that envelope.

Resources shouldn't ever touch any superglobal like $_GET, $_POST or $_FILES as that bypasses the servers responsibilities and therefor may make it incompatible with some servers. Therefor we shouldn't include a resource like this in Services core as that would promote a bad practice.

@acstewart: There's nothing non-standard in uploading files in single-part requests. The only thing multi-part does is saying that a request body consists of not one but multiple request bodies. That makes the requests rather complicated and therefor something I think we should avoid in an API where it actually sometimes matters how the HTTP-requests look.

Ok - I won't block this issue in any way. Go ahead and commit this when you're satisfied with the solution. It won't break anything other than possibly this specific resource - nothing that can't be fixed by those needing alternate solutions.

You now know my standpoint on this - I'm sorry for not being aware of anyone trying to solve this earlier.

No support for multipart/form-data is something I just ran into. I initially starting creating a webservice for uploading image files with the expectation that it was there, and I was surprised when I discovered it wasn't supported. The webservice will be handling some very large, high quality images, so base64 encoding is something I want to avoid. Would accessing the $_FILES superglobal be a reasonable workaround, similar to the patches posted above?

I think were all happpy with the patch in #8 it just hasnt been committed yet. Wanna give it a test?

Sure, I'll try it out and see how it goes.

I got a chance to try out the patch in #8. It works great for me. I created an html page with a file field named 'files[]', posted to the web service url, and it worked like a charm. This patch would solve my issues nicely.

Having tried out patch #8 successfully, changing the status to RTBC.

FIxed, heres the updated patch.

I've started looking at bring this to 6 as a patch. The simplest parts are :

diff --git a/servers/rest_server/includes/RESTServer.inc b/servers/rest_server/includes/RESTServer.inc
index b81dd30..d88041b 100755
--- a/servers/rest_server/includes/RESTServer.inc
+++ b/servers/rest_server/includes/RESTServer.inc
@@ -473,6 +473,9 @@ class RESTServer {
     return json_decode(self::contentFromStream($handle), TRUE);
   }
 
+  public static function parseFile($handle) {
+    return self::contentFromStream($handle);
+  }
   public static function parseYAML($handle) {
     module_load_include('php', 'rest_server', 'lib/spyc');
     return Spyc::YAMLLoadString(self::contentFromStream($handle));
diff --git a/servers/rest_server/rest_server.module b/servers/rest_server/rest_server.module
index 5fdb99d..25c4734 100755
--- a/servers/rest_server/rest_server.module
+++ b/servers/rest_server/rest_server.module
@@ -48,6 +48,7 @@ function rest_server_request_parsers() {
       'application/x-yaml' => 'RESTServer::parseYAML',
       'application/json' => 'RESTServer::parseJSON',
       'application/vnd.php.serialized' => 'RESTServer::parsePHP',
+      'multipart/form-data' => 'RESTServer::parseFile',
     );
     drupal_alter('rest_server_request_parsers', $parsers);
   }

I'm then trying to upload a fule to a custom action, but the parseFile returns nothing. Which results in "not acceptable" as required fields are not returned.

Instead :

+  public static function parseFile($handle) {
+    return $_POST;
+  }

Works, but expect its a HUGE secruity hole and a very bad thing (tm). How else should it be done?

How do I use the multipart/form-data content type header? I have a form that includes an image upload and some text fields. I can fill the text fields via json REST services with content type application/json. How do I add the upload in the same request using multipart/form-data?

I see two step processes suggested in Drupal.org - upload the image, get the fid, create the node now that you have the fid. But most Drupal content type forms aren't set up for that - they expect an upload, not a fid.

The overall Services approach of using drupal_execute to integrate with existing content types seems solid - but many of the forms drupal_execute is executing are multipart/form-data.

Advice or samples would be much appreciated.

Thanks.

I use version 3.1 and I have

'multipart/form-data' => 'RESTServer::parseMultipart',

but it's just return post

public static function parseMultipart() {
    return $_POST;
  }

I can replace it on

return self::contentFromStream($handle);

but multipart header means we can have both file and plain data, what the solution for that?

Here's an initial 6.x-3.x patch to get the ball rolling. This is just the 7.x-3.x patch applied to 6.x-3.x plus my attempt at porting whatever _file_resource_access and _file_resource_create_raw do, so it's not guaranteed to be useful or pass its tests. I'll finish it this week if nobody else is feeling more ambitious.

The 7.x-3.x patch has more parentheses in some conditionals than it needs to in _file_resource_access. Reducing them would improve clarity. The attached patch against 7.x-3.x does this.

both of thse look great.

I've fixed one issue, but the #26 still causes some tests to fail and prevents the full test suite from running. It'll definitely need some love before it gets pushed.

What is the status of multipart file uploads in d7 per now? Has these patches made their way into core services yet, or do I have to apply one of this patches?

Its already in 7.x

Do you have any specific idea of how to tackle uploading a file and sending JSON params in the same request (say, the client needs to tell something about the file currently uploaded)? I also use a two-step approach now but it would be much cleaner to solve it in one step.

+1 to #24 & #34
How to upload a file and send JSON params in the same REST request?
Your tips are greatly appreciated

Hi, guys
How can I use this multipart upload functionality with D6 ?
Which patch should I use?

Drupal 6 is no longer supported