The PUT line here is working:

curl -X PUT --user ws_user1:xxxxxx --header "Content-Type: application/xml" --data @test1.xml "http://www.example.com/node/65"

with the xml file containing:

<?xml version="1.0" encoding="utf-8"?>
<node>
<body><value>test node init body...</value>
<format>filtered_html</format>
</body>
<title>TEST 3 - edited</title>
<promote>1</promote>
<status>1</status>
<sticky>0</sticky>
<type>page</type>
<author>1</author>
</node>

However, to create a node I need to use

curl -X POST --user ws_user1:xxxxxx --header "Content-Type: application/xml" --data @test1.xml "http://www.example.com/node/1"

or that could be node/2, node/65 etc.

I'm trying to create a node of type 'page' - so according to the docs I should be POSTing to "http://www.example.com/page" but this returns a Drupal page containing

...
<div class="content">
    The requested page "/page" could not be found.  </div>
</div>
...

Trying with "http://www.example.com/node" gives me a '403 Forbidden' - which is strange because if I'm logged into the site via a browser as that user then I can view "http://www.example.com/node" fine.

CommentFileSizeAuthor
#126 creating_nodes_with-1720602-126.patch12.41 KBbtopro
#118 creating_nodes_with-1720602-116.patch12.37 KBrobertwb
#114 creating_nodes_with-1720602-114.patch12.3 KBrobertwb
#112 creating_nodes_with-1720602-112.patch12.5 KBrobertwb
#110 creating_nodes_with-1720602-110.patch11.37 KBrobertwb
#107 creating_nodes_with-1720602-107.patch11.37 KBrobertwb
#105 creating_nodes_with-1720602-105.patch11.34 KBrobertwb
#103 creating_nodes_with-1720602-103.patch11.69 KBrobertwb
#101 creating_nodes_with-1720602-101.patch11.61 KBrobertwb
#98 creating_nodes_with-1720602-98.patch11.28 KBrobertwb
#96 creating_nodes_with-1720602-95.patch11.28 KBrobertwb
#94 creating_nodes_with-1720602-94.patch11.26 KBrobertwb
#91 creating_nodes_with-1720602-91.patch11.28 KBrobertwb
#87 creating_nodes_with-1720602-87.patch10.72 KBitamair
#84 creating_nodes_with-1720602-84.patch9.5 KBitamair
#80 creating_nodes_with-1720602-80.patch9.46 KBitamair
#75 creating_nodes_with-1720602-75.patch9.46 KBitamair
#74 restws-1720602-72.patch9.46 KBitamair
#71 restws-1720602-71.patch9.71 KBitamair
#58 restws-1720602-58.patch9.25 KBfriedjoff
#55 restws-1720602-55.patch6.93 KBmariacha1
#55 interdiff--50_0.txt6.85 KBmariacha1
#52 restws-1720602-50.patch1.45 KBplach
#50 restws-1720602-50.patch1.39 KBplach
#44 restws-node_create_returns_403-1720602-44.patch4.04 KBGuyPaddock
#38 restws-node_create_returns_403-1720602-38.patch3.98 KBGuyPaddock
#36 restws-node_create_returns_403-1720602-34.patch3.7 KBGuyPaddock
#33 restws-node_create_returns_403-1720602-33.patch2.78 KBGuyPaddock
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

bailey86’s picture

Priority: Normal » Major

I'm raising the level for this bug as it feels quite important to me.

To create a node I can use:

$ curl -X POST --user icmsws:xxxxxx --header "Content-Type: application/xml" --data @test2.xml "http://www.example.com/node/76"

providing node 76 is the same type of node as the node I'm creating. Any other type of node will not work - and I can't find any other URL which works.

I have the LDAP authentication module installed. And also the Field permissions module.

The XML file contains:

<?xml version="1.0" encoding="utf-8"?>
<node>
<field_store_adjusted>26</field_store_adjusted>
<field_card_qty_adjusted>-5000</field_card_qty_adjusted>
<field_cam_qty_adjusted>-5000</field_cam_qty_adjusted>
<field_stb_qty_adjusted>-5000</field_stb_qty_adjusted>
<type>stock_adjustment</type>
<log>Requested from Delphi app.</log>
<author>45</author>
</node>
sepgil’s picture

To create a node you need to "POST" to "http://www.example.com/node" without any id. You can't specify the id...

bailey86’s picture

OK - thanks.

If you've got that working could you post up the exact curl command (or whatever was used) and any sample XML/JSON used to create a node.

Currently, POSTing to /node produces a 403 error for me - maybe it's an authentication module bug - or a clash with another module.

sepgil’s picture

Your command is correct...
curl -u restws_user:xxxx --header "Content-Type: application/xml" --data @test1.xml http://www.example.com/node
..but your username isn't. I assume you want to use the included Basic authentication login module, and in order for it to work, you have to use the prefix 'restws' on your user name your trying to log in with.
You can find more details in the Readme.

bailey86’s picture

Thanks for the comment.

The username is correct in my case - I've made the setting in the settings.php file RE
$conf['restws_basic_auth_user_regex'] = '/^web_service.*/';

and changed to to match our requirements.

Interestingly - it does look from the user log that most of the time the user is being reported as anonymous - even if we're using the correct username.

bailey86’s picture

Removed. Posted to wrong thread.

kamkam’s picture

Hello bailey86, Were you able to resolve this issue? I am seeing the same "403 Forbidden" error when I do a POST
Is there an example of POST with XML payload that works?

Thanks.
Kamran

bailey86’s picture

Nope - not resolved.

The node creation *will* work - but you have to use a URL of an existing node - which is not how it should work. But - as I say - it does work. This is the POST request:

curl -X POST --user ws_user1:xxxxxx --header "Content-Type: application/xml" --data @test1.xml "http://www.example.com/node/1"

and this is the example test1.xml file:

<?xml version="1.0" encoding="utf-8"?>
<node>
<field_store_adjusted>26</field_store_adjusted>
<field_card_qty_adjusted>-5000</field_card_qty_adjusted>
<field_cam_qty_adjusted>-5000</field_cam_qty_adjusted>
<field_stb_qty_adjusted>-5000</field_stb_qty_adjusted>
<type>stock_adjustment</type>
<log>Requested from Delphi app.</log>
<author>45</author>
</node>

Further details are at the beginning of this post.

aalamaki’s picture

Hi,

My guess is that it is related to the issue which I'm facing as well when doing CURL from PHP code, using the latest Drupal core and latest dev snapshot of Restws. My code is the following, the code is taken from http://www.barnettech.com/content/restful-webservices-module-restws with slight modifications:

<?php

$node = array(
      'title'     => 'Hello world',
      'body' => array(
        'value' => 'hi there',
        'summary' => 'hi there',
        'format' => 'filtered_html'
      ),
      'comment'   => 2,
      'promote'   => 0,
      'revision'  => 1,
      'log'       => '',
      'status'    => 1,
      'sticky'    => 0,
      'type'      => 'page',
      'language'  => 'und',
      'author'    => 1,
    );

$result = create_entity(json_encode($node));

function create_entity($data) {
  //global $user;
  $crl = curl_init();
  curl_setopt($crl, CURLOPT_URL,"http://<drupal_site>/user/login");
  curl_setopt($crl, CURLOPT_HEADER, 0);
  curl_setopt($crl, CURLOPT_USERAGENT, 'PHP script');
  curl_setopt($crl, CURLOPT_FOLLOWLOCATION, 1);
  curl_setopt($crl, CURLOPT_COOKIEJAR, "/tmp/cookie.txt");
  curl_setopt($crl, CURLOPT_COOKIEFILE, '/tmp/cookie.txt');
  curl_setopt($crl, CURLOPT_POST, TRUE);
  curl_setopt($crl, CURLOPT_POSTFIELDS, "name=<username>&pass=<password>&form_id=user_login");
  curl_setopt($crl, CURLOPT_VERBOSE, true);
  curl_setopt($crl, CURLOPT_COOKIE, session_name() . '=' . session_id());

  ob_start();

  curl_setopt($crl, CURLOPT_HTTPGET, TRUE);
  curl_setopt($crl, CURLOPT_URL, 'http://<drupal_site>/restws/session/token');
  curl_setopt($crl, CURLOPT_NOBODY, FALSE);
  curl_setopt($crl, CURLOPT_HTTPHEADER, '');

  $token = curl_exec($crl);

  ob_end_clean();

  curl_setopt($crl, CURLOPT_HTTPGET, FALSE);
  curl_setopt($crl, CURLOPT_POST, TRUE);
  curl_setopt($crl, CURLOPT_POSTFIELDS, $data);
  curl_setopt($crl, CURLOPT_URL, 'http://<drupal_site>/node');
  curl_setopt($crl, CURLOPT_HTTPHEADER, array('Content-Type: application/json', 'X-CSRF-Token: ' . $token));
  curl_setopt($crl, CURLOPT_RETURNTRANSFER, 1); // output to command line

  $ret = new stdClass;
  $ret->response = curl_exec($crl); // execute and get response
  $ret->error    = curl_error($crl);
  $ret->info     = curl_getinfo($crl);

  var_dump($ret);

  curl_close ($crl);

  unset($crl);

  return;
}

What happens is the user authentication in the first step passes fine with HTTP 200 but when saving the node, it returns response "403 Access Denied: CSRF validation failed".

If I change the url into an existing node url, it will not give the 403 but node is not created either. This behavior is to be expected though as it is supposed to take in only PUT when an existing node url is given.

On a bit of debugging the result, I realized that printing out the $token returns 1 instead of the hash; it appears also as the X-CSRF-Token in the code output.

bailey86’s picture

Priority: Major » Critical

Hmmm...

So what we're saying is that this functionality is still not correct - or is now not working at all. The key problem I came across was that to create a *new* node it was necessary to POST to an *existing* node of the same content type - where it should be POSTed to a simpler URL (such as /page) to create a new node.

My worry is that as this module is now going to be part of D8 core the dev focus has moved away from the D7 version. We have a production site using this module and so thanks for the warning - I'll be extra careful if the site is being upgraded.

What I'll do is bump this up to 'critical' to see if it gets the dev's attention - I get the feeling that it wouldn't take long for one of the devs to sort this out.

My feeling is that this issue is to do with when the module switched around to using POST instead of PUT and PUT instead of POST. See https://drupal.org/node/1472634 - this was the core reason for the 2.x release.

This could be a bigger problem than the devs realise - the stats currently show a huge number of downloads (10,468) but to me it looks like a relatively low number of sites reporting using the module (608). Maybe people a hitting this problem and then not using the module.

aalamaki’s picture

Did some debugging on the code and it is actually working as expected; the curl_exec returns boolean true/false and in this case, it returns the 1 for success.

The authentication and token retrieval both return HTTP 200 with response "1" on curl_exec() so they obviously get returned correctly but after this when trying to access the url /node for adding new node with POST, it is giving me the 404.

The module sets HTTP status in a few different situations and tried to debug the 404's there but doesn't seem that it's the module which sets the 404 in this case.

Sidenote to my Drupal installation for this case, I'm using Panopoly with Drupal core 7.19.

bailey86’s picture

but after this when trying to access the url /node for adding new node with POST, it is giving me the 404

That's what I was getting. But if I POSTed to /node/123 where 123 was an existing node the new node would be created (with a nice new ID).

However, the /node page can be accessed from a browser.

So - wild guess - is there a hook which is causing the issue?

Anyway - were you able to get your code working if you POSTed to an existing node?

BTW - Panopoly usage looks curious - only 1 reported current site - any idea why only 1?

klausi’s picture

Category: bug » support
Status: Active » Postponed (maintainer needs more info)

As you can see in the simpletests of this module POST for creating nodes is working as expected.

This is wrong:

$token = curl_exec($crl);

curl_exec() does not return the contents, you need to set CURLOPT_RETURNTRANSFER, see the PHP documentation about that.

Please try that and report back here.

aalamaki’s picture

Hi,

Yes that was actually in my last test, missed it first but added it there. The code now is:

<?php

$node = array(
      'title'     => 'hello from restful services',
      'body' => array(
        'value' => 'hi there',
        'summary' => 'hi there',
        'format' => 'filtered_html'
      ),
      'comment'   => 2,
      'promote'   => 0,
      'revision'  => 1,
      'log'       => '',
      'status'    => 1,
      'sticky'    => 0,
      'type'      => 'page',
      'language'  => 'und',
      'author'    => 1,
    );

$result = add_node(json_encode($node));

function add_node($post_data) {

  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL,"<drupal_site>/user/login");
  curl_setopt($ch, CURLOPT_HEADER, 0);
  curl_setopt($ch, CURLOPT_USERAGENT, 'PHP script');
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
  curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookie.txt");
  curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookie.txt');
  curl_setopt($ch, CURLOPT_POST, TRUE);
  curl_setopt($ch, CURLOPT_POSTFIELDS, "name=<username>&pass=<password>&form_id=user_login");
  curl_setopt($ch, CURLOPT_VERBOSE, true);
  curl_setopt($ch, CURLOPT_COOKIE, session_name() . '=' . session_id());

  ob_start();

  curl_exec ($ch);
  curl_setopt($ch, CURLOPT_URL, "<drupal_site>/restws/session/token");
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

  $HTTP_X_CSRF_TOKEN = curl_exec($ch);

  curl_setopt($ch, CURLOPT_HTTPGET, FALSE);
  curl_setopt($ch, CURLOPT_POST, TRUE);
  curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
  curl_setopt($ch, CURLOPT_URL, '<drupal_site>/node');
  curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json', 'X-CSRF-Token: ' . $HTTP_X_CSRF_TOKEN));
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

  ob_end_clean();

  $ret = new stdClass;

  $ret->response = curl_exec($ch);
  $ret->error    = curl_error($ch);
  $ret->info     = curl_getinfo($ch);

  print_r($ret);

  curl_close ($ch);

  unset($ch);

  return;
}

What happens now is it returns HTTP 200 for the authentication and for the token; also the token gets set correct. However, now when it tries to add the node, it gives me HTTP 403 Forbidden. The user has full admin rights to the system. Paste of the $ret:

 [response] => 403 Forbidden
    [error] => 
    [info] => Array
        (
            [url] => <drupal site>/node
            [content_type] => text/html
            [http_code] => 403
            [header_size] => 407
            [request_size] => 522
            [filetime] => -1
            [ssl_verify_result] => 0
            [redirect_count] => 0
            [total_time] => 0.407991
            [namelookup_time] => 1.8E-5
            [connect_time] => 1.9E-5
            [pretransfer_time] => 5.8E-5
            [size_upload] => 223
            [size_download] => 13
            [speed_download] => 31
            [speed_upload] => 546
            [download_content_length] => 13
            [upload_content_length] => 223
            [starttransfer_time] => 0.407943
            [redirect_time] => 0
            [certinfo] => Array
                (
                )

            [redirect_url] => 
        )

klausi’s picture

Looks like you are populating the cookie on your client:

curl_setopt($ch, CURLOPT_COOKIE, session_name() . '=' . session_id());

But you should of course use the cookie that is returned by the server.

aalamaki’s picture

Hmm, I don't think it is that one because I just ran it on my home server on a different Drupal installation but same d-core 7.19 and it worked perfect as-is, didn't change a thing except for adding CURLOPT_USERPWD right on the beginning of the code after curl_init because that's needed to get on the server. Will have to continue investigating...

klausi’s picture

CURLOPT_COOKIE is definitely wrong, since the docs say that this sets the cookie header. But yeah, you are using it on the login request, where it does not make a difference. You can just remove it.

aalamaki’s picture

Hi,

Yeah I guess so...will get rid of it, going to check on the other drupal installation tomorrow and try debug why it still gives the 403 there.

Another thing, I just tried to create a Scald Atom for the scald file provider but it gives me again that 403 when trying to create, even tried using the uid 1 of drupal for login and creation.

My question is, has this ever been tested with Scald entities? The code that I'm using is the same as before with just the exception that I changed the following:

curl_setopt($ch, CURLOPT_URL, '<drupal_site>/node');

into

curl_setopt($ch, CURLOPT_URL, '<drupal_site>/scald_atom');

Also changed the $node array into the following:

$node = array(
      'title'     => 'testing scald creation',
      'sid'       => null,
      'provider'  => 'scald_file',
      'base_id'   => 132,
      'data'      => array(),
      'actions'   => 5,
      'scald_authors'    => array(),
      'scald_tags'       => array(),
      'type'      => 'file',
      'publisher'    => 1,
    );

As far as I know, the above should be more than enough for Scald atom creation so don't really see that this could be the reason for the 403.

giorgio79’s picture

Thx aalamaki for your code at #14, it worked for me as a simple restws php client :)

oceanos’s picture

Has anyone solved the 403 issue already? I constantly get 403 - forbidden when trying to POST a new node to /node. POSTing to /node/1 (an existing node of the same content type) to create a NEW node works as described in #8, but this doesn't make sense.

When setting the permissions for the logged in user to "Bypass content access control", POSTing to /node works, but I don't want to give the user that permission.

Interestingly, when giving the anonymous user access to the REST resource "node" and giving him the permission to create nodes, POSTing to /node works fine, even when the "Bypass content access control" is NOT set for the anonymous user.

This is really a show stopper for this otherwise great module.

klausi’s picture

What does the 403 error message say? Looks like you are not sending the CSRF token as authenticated user?

Did you try restws_basic_auth where you don't need the CSRF token?

oceanos’s picture

Yes, I send the CSRF-Token along in the header. Therefore, PUT and DELETE work as they should, and POST also works, but only to node/* instead of to /node. If I did not send the token, neither of that would work.

The return is just '403 - forbidden'. If I omitted the token, it would say '403 - Access Denied: CSRF validation failed'. But that is not the case.

In the Drupal logs the corresponding message is just "access denied".

BTW: I'm testing using Chrome and the Dev http Client plugin.

klausi’s picture

It is required that the user POSTing the node has the "bypass node access" permission, because entity_metadata_no_hook_node_access() has no other way of granting access on creation. There might be an Entity API issue somewhere to fix that for the create operation.

oceanos’s picture

I tried both restws_basic_auth that comes with restws and the session based authentication that comes with the Services module. With both methods, authentication works flawlessly, but I can't POST a new node to the /node URI without attaching the nid of an existing node.

klausi’s picture

oceanos’s picture

I applied patch #136 to the Entity API module as suggested in the thread you pointed to, but with no effect. Could you give some more details?

Thanks very much for your support so far!

klausi’s picture

No further ideas - you need to debug the cause of the 403. What else is registered on /node on your site? Does the request arrive at RESTWS if you enable logging?

oceanos’s picture

The /node path is actually a drupal system path pointing to the the default front page of each drupal installation. Each node that has the "promoted to frontpage" setting is shown on that page.

It is possible to set another page (or view etc.) as the frontpage in the drupal settings, but the /node URL and the corresponding system page still live in the background. So maybe the attempt to POST something to the /node URL interferes with the drupal core itself?

Can you reproduce the behaviour?

oceanos’s picture

I enabled logging and the request arrives at restws. The logfile says:

2013-07-31T14:59:02+0200
Resource: node
Operation: create
Format: application/json
Id: 
Payload: {
"title":"TEST",
"type":"fs_sub",
"author":{
"id":"5"
}

}

I think something's going wrong in restws.entity.inc after line 286:

if (!user_access('bypass node access')) {
        $this->propertyQueryOperation($query, 'Condition', 'status', 1);
      }

Why? When I set "Bypass content access control" for authenticated users in the permission settings, POSTing to /node works fine.

GuyPaddock’s picture

Status: Postponed (maintainer needs more info) » Active

I am also seeing this issue.

The problem is that restws_entity_node_access() expects a node to be passed-in, but it's getting a NULL instead. That function is actually called indirectly through the entity_access() function (it's the access callback registered on the node entity), which is in turn being called by RestWSEntityResourceController::access(), which is providing NULL for the node parameter when the $id arg is NULL.

So, the call chain looks like this:

  1. restws.module: restws_handle_request() calls $resource->access() with a NULL for $id.
  2. restws.entity.inc: RestWSEntityResourceController::access() calls entity_access() with a NULL for $entity.
  3. entity.module: entity_access() calls restws_entity_node_access() with a NULL for $node.

The isset() check in restws_entity_node_access() looks wrong...

GuyPaddock’s picture

Hmm, nope, wasn't the isset()... It looks like it always needs a node to be passed in to know what type it should be. The question is, how do we get an empty node to pass in? I'm thinking the issue is in restws_handle_request().

GuyPaddock’s picture

Version: 7.x-2.0-alpha1 » 7.x-2.1
Status: Active » Needs review

Hmm, an even larger problem -- how does the /node endpoint know the content type of the node that is being created? I mean, it can be specified in the JSON, but... it seems like that's too late. All the security checking of handle_request() happens by the time we're digging into the request content.

GuyPaddock’s picture

Oops, didn't mean to set NR on the last comment; meant to set it on this one.

Attached is a patch that address this issue. It's a bit of a larger change than I was hoping for -- I had to change how the security check is being done so that we can detect the bundle type from the incoming request.

With this patch applied, Rest WS seems to now work as expected for the "create" case.

Status: Needs review » Needs work

The last submitted patch, restws-node_create_returns_403-1720602-33.patch, failed testing.

GuyPaddock’s picture

Ummm... pretty sure, my patch didn't break those tests... were they passing previously?

GuyPaddock’s picture

Status: Needs work » Needs review
FileSize
3.7 KB

Nope... looks like I *did* break them. It was subtle, but lethal.

Attached is a re-roll with additional concerns addressed. :: fingers crossed ::

Status: Needs review » Needs work

The last submitted patch, restws-node_create_returns_403-1720602-34.patch, failed testing.

GuyPaddock’s picture

Status: Needs work » Needs review
FileSize
3.98 KB

How about.... this one?

Status: Needs review » Needs work

The last submitted patch, restws-node_create_returns_403-1720602-38.patch, failed testing.

klausi’s picture

Status: Needs work » Postponed (maintainer needs more info)

First we need to know the actual problem. As you can see in the test cases creating nodes with POST works fine, so we need to know what goes wrong with your request.

GuyPaddock’s picture

Actually, the tests *don't* show that. They pass because the logged-in user has permission to bypass the access checks...

I explained the actual problem in #30. The request is not at fault.

GuyPaddock’s picture

Status: Postponed (maintainer needs more info) » Active
GuyPaddock’s picture

PS Here's a sample request to /node that fails unless the user has a role with the "bypass content permissions" permission:
{"type": "article", "title": "This will fail"}

The CSRF header is being passed in, along with the session cookie. Saving works normally with my patch applied.

So... yeah, it's NOT the request.

As was noted earlier in the thread, though, if I create a dummy node (say, nid "2"), and then use the endpoint "/node/2" instead of "node", it works -- for the reasons I explained previously. The code is flawed.

GuyPaddock’s picture

Status: Active » Needs review
FileSize
4.04 KB

Let's see if this works...

Status: Needs review » Needs work

The last submitted patch, restws-node_create_returns_403-1720602-44.patch, failed testing.

klausi’s picture

Title: POST for create is not working with expected URL » Creating nodes with POST should work without bypass node access permission
Category: support » feature
Priority: Critical » Normal

Aha, so this is a feature request that nodes should be working not only with the "bypass node access" permission.

So we already have restws_entity_node_access() as custom access callback for nodes. Now the question is if it gets called correctly from entity_access() and what happens on create.

I don't think we have to change that much code, it is just a matter of correctly invoking restws_entity_node_access().

Also the issue summary is now completely wrong, could you fix that as well?

GuyPaddock’s picture

Category: feature » bug
Status: Needs work » Active

Okay, several issues:

1. This isn't a feature request. It *is* a bug. The docs indicate that if the user has the appropriate permissions to create the type of content that they're trying to create, they should be able to do so by posting to "/node" with appropriate headers set.

2. The issue summary is *not* completely wrong. It is accurate -- the OP was a symptom of the same problem.

3. As far as I know, the Drupal community takes security very seriously. If we have to give a system account permission to completely bypass all security on all content types just to be able to create one or two posts of a given type, this does not seem to be in line with community expectations.

bailey86’s picture

Issue summary: View changes

Have we not drifted off the point here?

Say I want to create a new 'Basic page' - I should be POSTing to /page - not to /node. That was the main point of this post.

What I found originally was that POSTing to /page would not work - but if I POSTED to /node/123 - or node/124 - or any existing node (of type page) then the new node of type page would be created.

(The XML posted contains a field type - i.e. page)

So why are we talking about POSTing to /node? The module is supposed to create a new node by POSTing to /[content type] i.g. /page.

For reference again - here is part of my original post with some samples etc:

The PUT line here is working:

curl -X PUT --user ws_user1:xxxxxx --header "Content-Type: application/xml" --data @test1.xml "http://www.example.com/node/65"

with the xml file containing:

<?xml version="1.0" encoding="utf-8"?>
<node>
<body><value>test node init body...</value>
<format>filtered_html</format>
</body>
<title>TEST 3 - edited</title>
<promote>1</promote>
<status>1</status>
<sticky>0</sticky>
<type>page</type>
<author>1</author>
</node>

However, to create a node I need to use

curl -X POST --user ws_user1:xxxxxx --header "Content-Type: application/xml" --data @test1.xml "http://www.example.com/node/1"

or that could be node/2, node/65 etc.

I'm trying to create a node of type 'page' - so according to the docs I should be POSTing to "http://www.example.com/page" but this returns a Drupal page containing

...
<div class="content">
    The requested page "/page" could not be found.  </div>
</div>
...
muschpusch’s picture

Ok the entity API issue from #25 got solved but this problem still persists. So #46 is pending or after having a quick look at code.: Can we just remove that access check below and entity API will be smart enough to manage the access control by itself? Of course we would need to set an entity API dependency to the latest version.

/**
 * Access callback for the node entity.
 *
 * Replacement for entity_metadata_no_hook_node_access() because it does not
 * work with the create operation.
 *
 * @todo Remove this once https://drupal.org/node/1780646 is fixed.
 *
 * @see restws_entity_info_alter()
 * @see entity_metadata_no_hook_node_access()
 */
function restws_entity_node_access($op, $node = NULL, $account = NULL) {
[...]
plach’s picture

The attached patch solves the issue here and might pass tests: it basically moves the access check for the create op after populating the entity object and before saving it.

plach’s picture

Status: Active » Needs review
plach’s picture

FileSize
1.45 KB

Sorry, wrong format

The last submitted patch, 50: restws-1720602-50.patch, failed testing.

klausi’s picture

Version: 7.x-2.1 » 7.x-2.x-dev
Status: Needs review » Needs work
+++ b/restws.module
@@ -129,7 +129,7 @@ function restws_handle_request($op, $format, $resource_name, $id = NULL, $payloa
-    if (user_access('access resource ' . $resource_name) && $resource->access($access_op, $id)) {
+    if (user_access('access resource ' . $resource_name) && ($op == 'create' || $resource->access($access_op, $id))) {

we should not handle this in the global restws_handle_request() function but rather in the access() method of the RestWSEntityResourceController. We should also add a comment why we are making an exception for the create operation.

And we should update our node creation test case, it should work now without the bypass node access permission.

mariacha1’s picture

Status: Needs work » Needs review
FileSize
6.85 KB
6.93 KB

In this patch I've moved the check for access to the access() method with a note.

Because the patch at #52 passes an entity wrapper to the access() function instead of a string, I've also updated the text on the interface class.

I've also removed all reference to "bypass node access" permissions in the tests. This might be totally wrong, and probably should be done more discriminately. I'm interested to see if they pass! :)

Patch and interdiff between this and 50_0 attached.

Status: Needs review » Needs work

The last submitted patch, 55: restws-1720602-55.patch, failed testing.

mariacha1’s picture

Ahh, of course. Well, I'll keep looking at that later!

friedjoff’s picture

Status: Needs work » Needs review
FileSize
9.25 KB

This patch is based on #55 and updates the permissions in the tests. A new test which creates a node with missing permission has been added. Also the access() method will only allow empty entity wrapper, otherwise the create X content permissions is not checked.

Media Crumb’s picture

Was this patch push live? It seem like a fairly large scale issue to just leave open. Giving a user full bypass access means they can have control to delete, edit, any other users content. I'm not sure why this is ok given the huge security issues it opens up. To @GuyPaddock point the docs state that the REST server will properly give access to roles that are granted, but this isn't the case. I spent the better half of 2 days trying to figure this out before coming to find this thread.

opdavies’s picture

Status: Needs review » Reviewed & tested by the community

The patch in #58 works for me.

My API user has the "Access the resource node" permission from restws, and the "Create {node type} content" from node module, but not "bypass node access" and I'm able to successfully create a node via a POST request.

opdavies’s picture

Priority: Normal » Major

I think that this should be higher.

Media Crumb’s picture

Agreed. this should be reviewed and rolled into dev

opdavies’s picture

After applying the patch in #58, I still got a 403 error on occasion with my API user. After querying the response from the API, I was getting a

Error: 403 Forbidden: Not authorized to set property status

message, and the same for the language property.

After I stopped trying to set these properties, the POST worked fine.

bailey86’s picture

So - I need to create Drupal entities via REST.

Is this still an issue with restws? Maybe this module is best for reading data - and services would be best used for adding data. I wonder if they can be used at the same time.

opdavies’s picture

I'm successfully POSTing nodes into Drupal using the restws module and the patch in #58, as well as the configuration in #60 and #63.

Media Crumb’s picture

did something change is 2.4 release? Just tried to apply patch but im getting errors

restws-1720602-58.patch
patching file restws.entity.inc
Hunk #2 FAILED at 167.
Hunk #3 succeeded at 341 (offset 1 line).
1 out of 3 hunks FAILED -- saving rejects to file restws.entity.inc.rej
patching file restws.test
Hunk #4 succeeded at 131 (offset 53 lines).
Hunk #6 succeeded at 173 (offset 53 lines).
Hunk #8 succeeded at 252 (offset 53 lines).
Hunk #10 succeeded at 341 (offset 53 lines).
Hunk #12 succeeded at 391 (offset 53 lines).
Hunk #14 succeeded at 592 (offset 53 lines).

EDIT: it looks like its conflicting with https://www.drupal.org/node/2090177

opdavies’s picture

Media Crumb, It applies cleanly for me. How are you applying the patch?

Media Crumb’s picture

Yeah opdavies, the issue was having already patched for the other bug with taxonomy terms. Really with these fixes would get rolled into dev soon

opdavies’s picture

Ah, OK. Yes, hopefully they will get merged in soon.

lokapujya’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs issue summary update, +Needs security review
  1. The issue summary is still about being about something other than the issue title.
  2. Security Review: So the idea is good in that a REST client can be limited to only posting nodes for which the REST client has access to. Since this was RTBC'd over a year ago, let's just make sure there are no holes opened up. Convince me the user isn't somehow still bypassing node access.
  3.   +++ b/restws.entity.inc
      @@ -74,8 +74,8 @@ interface RestWSResourceControllerInterface {
      -   * @param int|string $id
      -   *   The id of the resource.
      +   * @param int|string|object $id
      +   *   The id (or EntityDrupalWrapper in case of 'create') of the resource.
      

    should this variable still be called $id if it's not always an id?

itamair’s picture

FileSize
9.71 KB

this patch didn't passed the test

itamair’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 71: restws-1720602-71.patch, failed testing.

itamair’s picture

FileSize
9.46 KB

This corrects the #71 patches ... (making it test passing)

itamair’s picture

FileSize
9.46 KB

Patch in #58 allows the correct creation of the new node (without bypass node access permission), but still misses the properties settings task.
This new patch fixes that, allowing the settings of the "status" and "language" properties too.

itamair’s picture

lokapujya’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 75: creating_nodes_with-1720602-75.patch, failed testing.

The last submitted patch, 75: creating_nodes_with-1720602-75.patch, failed testing.

itamair’s picture

FileSize
9.46 KB

This new patch fixes that, allowing the settings of the "status" and "language" properties too.
I don't know why this fails testing. It apply correctly (to me) to the current 7.x-2.x-dev

itamair’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 80: creating_nodes_with-1720602-80.patch, failed testing.

lokapujya’s picture

23:25:35 RESTWS Basic Auth 15 passes, 1 fail, and 0 exceptions
23:25:35
23:25:35 Fatal error: Call to a member function attributes() on a non-object in /var/www/html/sites/all/modules/restws/restws.test on line 280

itamair’s picture

FileSize
9.5 KB

Last try. This new patch just amends two lines from the #58 one (fixing and allowing the settings of the "status" and "language" properties too)

In the /restws.entity.inc file:

- // Get the ID and bundle property names.
- $entity_keys = array_intersect_key($entity_info['entity keys'], array('id' => 1, 'bundle' => 1));

+ // Get the entity keys properties.
+ $entity_keys = array_merge($entity_info['entity keys'], ['status' => 'status']);

It passes all "RESTful web services tests" Simpletests in my local environment.

itamair’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 84: creating_nodes_with-1720602-84.patch, failed testing.

itamair’s picture

This new patch fixes the same problem both for creation and update operations, calibrating the entity properties update (language, status, etc.) on the user entity_access permissions ...

robertwb’s picture

Confirming that the patches in 84+ resolve issues with adding and updating entities via REST. My case was using 3 different types of custom entities, and provided that the permissions were implemented properly (I had to work through a veil of ignorance first!), this goes nicely. Not sure why the testbot keeps crapping out...

Anyhow - goods stuff @itamair !!

robertwb’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 87: creating_nodes_with-1720602-87.patch, failed testing.

robertwb’s picture

This patch fails testbot due to a problem with restws.test code in:
Fatal error: Call to a member function attributes() on a non-object in /var/www/html/sites/all/modules/restws/restws.test on line 280

Adding a modification to the user creation, and some additional test lines to attempt to diagnose.

robertwb’s picture

Status: Needs work » Needs review

Triggering testbot

Status: Needs review » Needs work

The last submitted patch, 91: creating_nodes_with-1720602-91.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
11.26 KB

Further testbot testing.

Status: Needs review » Needs work

The last submitted patch, 94: creating_nodes_with-1720602-94.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
11.28 KB

Re-enabling bypass to determine if it is indeed a permissions failure (the perms set here work on my own system).

Status: Needs review » Needs work

The last submitted patch, 96: creating_nodes_with-1720602-95.patch, failed testing.

robertwb’s picture

Testing is failing due to the following message "Fatal error: Call to a member function attributes() on a non-object " (line 280 in patch #87). The error tells me that the simplexml_load_string() call is failing to produce a result.

279:     $xml_element = simplexml_load_string($result);
280:     $nid = $xml_element->attributes()->id;

My first thought was that this was a permissions issue but I added "bypass node create" back into the permissions in Patch #95, but it failed testing with the same error. (also tried to get it to throw a proper testbot error with assertTrue but didn't make a diff).

-      'access content', 'bypass node access', 'access resource node',
+      'edit any article content', 'create article content', 'access resource node', 'administer users','bypass node access'
...
     $xml_element = simplexml_load_string($result);
+    $this->assertTrue(is_object($xml_element), 'simplexml_load_string(' . print_r((array)$result,1) . ') returns valid object.');
     $nid = $xml_element->attributes()->id;

This makes me conclude that perhaps it is the XML insert itself is malformed in some way rather than the permission. One last test of permissions here by re-adding "access content" and then will explore tweaking the XML for insert.

robertwb’s picture

Status: Needs work » Needs review

Changing status to trigger test.

Status: Needs review » Needs work

The last submitted patch, 98: creating_nodes_with-1720602-98.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
11.61 KB

Patch #98 also failed, so it seems to me that this is NOT an issue with perms, but something else that is preventing the completion of the XML insert. Trying a simpler xml here (in case it is a case of malformed taxonomy data). If this does not resolve it, I will file a bug report on the restws.test.

Status: Needs review » Needs work

The last submitted patch, 101: creating_nodes_with-1720602-101.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
11.69 KB

Trying a modified test of smlx insert, asserting only that a 201 code is returned under the assumption that the problem is the result loading by simplexml_load_string().

Status: Needs review » Needs work

The last submitted patch, 103: creating_nodes_with-1720602-103.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
11.34 KB

Last patch certainly managed to get some fails ... re-testing.

Status: Needs review » Needs work

The last submitted patch, 105: creating_nodes_with-1720602-105.patch, failed testing.

robertwb’s picture

At the risk of being completely spurious, it seems that the testing for CRUD without bypass has many fails. The simplexml error in previous versions seemed to mask the other errors - now 12 errors are reported all but one having to do with permissions (cache poisoning in auth sub-module also fails). The attached patch tries to resolve a single issue in testCRUD().

robertwb’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 107: creating_nodes_with-1720602-107.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
11.37 KB

Testing CRUD part 2

Status: Needs review » Needs work

The last submitted patch, 110: creating_nodes_with-1720602-110.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review
FileSize
12.5 KB

Attempting to fix errors in method testResourceArray() by checking only for code 200 and 201 - this may not be a robust fix, but it seems that the technique that was used in many of these MAY BE inserting/updating successfully, but failing on node retrieval.

Status: Needs review » Needs work

The last submitted patch, 112: creating_nodes_with-1720602-112.patch, failed testing.

robertwb’s picture

More tests.

robertwb’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 114: creating_nodes_with-1720602-114.patch, failed testing.

robertwb’s picture

Status: Needs work » Needs review

Last test got close - xml update seemed to pass, but xml insert failed. REtesting with insert disabled.

robertwb’s picture

Status: Needs review » Needs work

The last submitted patch, 118: creating_nodes_with-1720602-116.patch, failed testing.

robertwb’s picture

Finally some progress. #118 managed to pass testResourceArray() by avoiding the XML update. I also suspect that the json insert was faulty,but that was not directly screened. XML update seemed to pass, but more rigorous testing (assertequal on something like a field or property) is needed.

This patch to the module works in practice (at least on one install for me) -- seems like the testing is in need of work.

itamair’s picture

Thank you #robertwb for this in deep review ;-)

robertwb’s picture

Yeah @itaimar - sorry to trigger so many tests -- I am fairly ignorant of the testbot workings so it took a while. Your enhancements really make secure REST in D7 a reality - I can now verify proper function on 2 systems with fine grained control of access to content/entity types and controlled access to functions.

I will keep at it but try to be a bit more efficient now that I get the gist.

lokapujya’s picture

At some point, it might be help reviewers if someone can post a diff from #107 (where we had the CI error) to where we are now.

robertwb’s picture

Agreed @lokapujya, but I would point out that the last patch to pass all tests was #58, and we have had CI errors since #75. Note also that I believe that the testbot changed between those two patches, so not sure if that impacts the test failures or not.

So, in short, I would resubmit #58 to see if it passes, then figure out what's up with the testing from there.

btopro’s picture

+    // The "create" operation requires an existing entity to check access.
+    // See https://www.drupal.org/node/1780646
+    if ($op == 'create' && empty($id)) {
+      return TRUE;
+    }

This needs tweaked I think. In our testing, $id wasn't empty, it was an empty value object. The following helped account for this use-case

if ($op == 'create' && (empty($id) || empty($this->wrapper($id)->value()))) {
  return TRUE;
}
btopro’s picture

patch from what was mentioned in 125

lokapujya’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 126: creating_nodes_with-1720602-126.patch, failed testing.

bramface’s picture

Also eager for resolution ... second pilot project begins in two weeks, depends on this fix as far as we can tell.

fweiss6’s picture

I'm trying to use this great module to automate content creation via my organization's delivery pipeline. I get 403 when POST /node unless the user's role has "bypass node access" permission. I'm trying to create a node of type swaggerdoc (custom type), so should only need the "create swaggerdoc content" permission.

Looking into the entity_access bug linked by klausi at #25, I tried a little patch which gave me some joy, the following change in restws.entity.inc:

public function access($op, $id) {
     $node = entity_create('node', array('type' => 'swaggerdoc'));
     return entity_access($op, $this->entityType, isset($id) ? $this->wrapper($id)->value() : $node);
  }

It's not perfect, but solved my immediate problem. Perhaps this will inspire the maintainers and contributors to fix this bug.

misjao’s picture

The patch above in #130 is not safe.

With this patch users with access to 'swaggerdoc' can create any type of node.
The patch only checks if the user has access to swaggerdoc, not if the node the user is trying to create is also of the type 'swaggerdoc'.

robertwb’s picture

Is this issue still valid? Is it related to the recent security patches? I mean, it seems like a security bug to have "bypass permissions" anytime... https://www.drupal.org/sa-core-2019-003

btopro’s picture

it is not related. The recent core / contrib fix is related to sanitization of fields. This is a logic centric issue. If you have no problem giving bypass (say to a super admin / non-user web service account) then sure. If you need to bypass for normal users (and that causes problems given that normal users are not super users in your setup) then you shouldn't use this.

robertwb’s picture

Thanks @btopro