I'm not sure if this is a bug on my side, or the bug is inside the getControllerArguments() routine of RESTServer.inc, but I am leaning towards the bug being in getControllerArguments() after watching the logic run several times incorrectly. I also am using the 6.x-3.x-dev code from Jan 21, 2011.

The bug appears to be in the foreach loop beneath the comment "Map source data to arguments"; specifically, the line if(isset($sources[$info['source']][$info['name']])) will always be false for a PUT operation's 'data'.

Explaining in a bit more detail:

Here's the 'update' portion of my hook_services_resources() implementation:

     'update' => array( 'help'                    => 'Updates a note',
'callback'                => '_api_shell_note_update',
'access callback'         => '_api_shell_note_access',
'access arguments'        => array('update'),
'access arguments append' => TRUE,
'args' => array( array( 'name'        => 'id',
'type'        => 'string',
'description' => 'The note Id',
'source'      => array('path' => '0'),
'optional'    => FALSE,
  ),
array( 'name'        => 'data',
'type'        => 'struct',
'description' => 'Updated note data',
'source'      => 'data',
'optional'    => FALSE,
  ),
   ),

This is almost the same logic as in Hugo's 'note resource' example, the only difference is that the id's 'type' is a string here, where in Hugo's example the id is an int. And here's the logic for issuing a PUT with javascript to that above definition:

  gApiShell.note.update = function (data, callback) {
    console.log('inside note.update, about to execute PUT on url: ' + this.apiPath + '/' + data.id);
    $.ajax({ type:        "PUT",
             url:         this.apiPath + '/' + data.id,
             data:        JSON.stringify(data),
             dataType:    'json',
             contentType: 'application/json',
             success:     callback
          });
  };

This is also the same logic as in Hugo's 'note resource' example.
However, when attempting to make a PUT through the javascript routine above, I see the following occur inside getControllerArguments($controller,$path,$method,&$sources=array()):

  1. I printed all the input arguments, and observed that $controller, $path & $method all have the expected and correct values.
  2. At first, the $sources parameter is empty, but I figured out that is how it is passed. The initial logic of getControllerArguments() is the populating of the $sources parameter. After population, $sources looks like this:
    array( path => array( 0 => '13' ),
              param => array( q => 'api_shell/dev/note/13' )
              data => array( subject => 'subject str', note => 'note str', id => '13' )
            )
  3. A bit lower in the code, in the "Map source data to arguments" loop, when it is time to map the PUT data to the argument with index '1', this conditional: if(isset($sources[$info['source']][$info['name']])) will always fail because $sources['data']['data'] does not exist.
  4. after not finding the PUT data, an exception is thrown because the PUT's data is not optional.
    1. The correct conditional to test for the existence of the 2nd argument of a PUT (the 'data') would just be if(isset($sources[$info['source']])) because $sources[data] points to the PUT data in its entirety, not some element within $sources[data] with the key 'data' (as is currently being requested in the logic.)

      I attached the "api_shell' and 'api_shell_test' modules that exhibit this issue. This issue posting is related to my earlier documentation request posting http://drupal.org/node/1034540; I made a new issue posting because this looks like a bug, rather than a continuation of my documentation support request.

Comments

bsenftner’s picture

Priority:Normal» Critical

(I've elevated the status of this issue to 'critical' because according to http://drupal.org/node/45111, this issue renders the REST server system unusable - Create and Update operations are impossible with how the code is written and how the example shows one to use the Create and Update operations.)

In an effort to get this working, I've modified the source of getControllerArguments() such that both POST & PUT are being handled correctly. Here's my reasoning, using Hugo's noteresource & noteresourcejs modules as my example to illustrate how this bug occurs:

In Hugo's notes.js example, the data for a Create-POST is constructed like this:

var data;
data = {
        subject: $(subject).val(),
        note: $(text).val()
      };
noteapi.create(data, note_saved);

while the Create controller inside noteresource.module is defined like this:

'create' => array(
       'help' => 'Creates a note',
       'file' => array('file' => 'inc', 'module' => 'noteresource'),
       'callback' => '_noteresource_create',
       'access callback' => 'user_access',
       'access arguments' => array('note resource create'),
       'access arguments append' => FALSE,
       'args' => array(
         array(
           'name' => 'data',
           'type' => 'struct',
           'description' => 'The note object',
           'source' => 'data',
           'optional' => FALSE,
         ),
       ),
     ),

Which means that inside getControllerArguments(), after the "Get argument sources" portion, the $sources array will look like this:

array( path => array(),
          param => array( q => {the endpoint path} ),
          data => array( subject => 'a subject', note => 'a note' )
        )

Then this code fragment a bit lower inside the "Map source data to arguments" portion is how the Create-POST's data is assigned to the arguments array:

              if(isset($sources[$info['source']][$info['name']])) {
                $arguments[$i] = $sources[$info['source']][$info['name']];
              }
which is the same as:
              if(isset($sources['data']['data'])) {
                $arguments[$i] = $sources['data']['data'];
              }

And as you can see, that isset() contents does not exist, it is trying to access inside the data rather than assign the data itself to the $arguments array.

Now I modified that assignment logic like this:

             if(isset($sources[$info['source']][$info['name']])) {
                $arguments[$i] = $sources[$info['source']][$info['name']];
                _devReport('Original-good!');   // appends this message to a text file 'cause I don't have step debugging
              }
              else {
                if(isset($sources[$info['source']])) {
                    $arguments[$i] = $sources[$info['source']];
                    _devReport('New-Good!');   // appends this message to a text file 'cause I don't have step debugging
                }
              }

This modification makes all the data for a Create-POST arrive at the controller callback and it also makes Update-PUTs operate correctly for the exact same reasons.

It may also be worth noting, that as I test all the CRUD verbs, the 'Original-good' message never gets echoed. I've left in it place because I don't know if that original logic is necessary for Actions, Targeted Actions or Relationships.

Sorry, I have not rolled this modification into a patch. I don't use CVS or git, and according to the 'Creating a patch' documentation here at Drupal.org, making a patch with diff is unreliable.

kylebrowning’s picture

So changing those lines,

if(isset($sources[$info['source']][$info['name']])) {
                $arguments[$i] = $sources[$info['source']][$info['name']];
                _devReport('Original-good!');   // appends this message to a text file 'cause I don't have step debugging
              }
              else {
                if(isset($sources[$info['source']])) {
                    $arguments[$i] = $sources[$info['source']];
                    _devReport('New-Good!');   // appends this message to a text file 'cause I don't have step debugging
                }
              }

Make this work for you?

bsenftner’s picture

Yes it does. That is the bug that prevents Create-POSTs and Update-PUTs from working correctly with how Hugo's example demonstrates the packaging of the data for the POST and PUT.

Perhaps Hugo's data packaging example is incorrect, and rather than:

var data;
data = {
        subject: $(subject).val(),
        note: $(text).val()
      };
noteapi.create(data, note_saved);

The data should be packaged:

var data;
data = {
        data: { subject: $(subject).val(),
                   note: $(text).val()
                }
      };
noteapi.create(data, note_saved);

That would make the original logic for a Create-POST work correctly (but break Hugo's example) and a similar data-object-inside-the-outter-most-data-object packaging for Update-PUT would also work with the original logic.

kylebrowning’s picture

Priority:Critical» Normal

Also, we have several test cases written for 6.x that include creating data, and using actions. Maybe you can write a test case and show us it failing so that we can actually identify the problem.

I changed the RESTServer.inc with what you had and its resulting in 7 failed tests. It would be extremely helpful if you could write a test case to show us the issue, rather then me trying to get a JS implementation of services running.

Also I dont think this is critical as we can easily CRUD and use actions and its provable via tests.

If you can show me a failed test case that I can easily run I will gladly mark this as critical and attempt to figure it out.

Edit:
I think its fairly agreeable by all that a lot has changed with services 3.x branch since Hugo left it, which was several several months ago. This might be a good indication that his code is outdated and not be used as an example.

bsenftner’s picture

Priority:Normal» Critical

I'm happy to write a test case, but I'm unfamiliar how. I've not used simpletest before; that is what you're referring to, correct?

I have been doing this debugging in an api_shell and api_shell_test modules, where the Service implementation lives in api_shell, and the javacript logic that I'm using to do my debugging tests lives in api_shell_test.

It is my intention to contribute the working api_shell to the Services team as a working example that demonstrates CRUD as well as Actions, Targeted Actions and Relationships.

Would making the api_shell & api_shell_test modules more easy to use, as in not requiring FireBug to run the tests, be good enough?

I'll take a look at the simpletest documentation to see if it's a no-brainer...

kylebrowning’s picture

Priority:Critical» Normal

Take a look at the file
tests/functional/ServicesResourceNodeTests.test

Its pretty straight forward and easy.

Is the api_shell that you uploaded to this ticket up to date? If not can you make sure it is and Ill download and install and look at your implementation?

THanks

Also, im not sure if you bumped it back to critical on purpose, but Im going to assume it was an accident.
Again until I can recreate the issue I dont see it as critical.

bsenftner’s picture

Sorry, yes, bumping it back to critical was an accident.

Looking at tests/functional/ServicesResourceNodeTests.test, I see that it is object orientated PHP, which I've not done much development in. Now, C++ OOP I'm great at, but I hesitate because my weak OOP-PHP may implement a buggy test.

However, reading both the POST and PUT tests in tests/functional/ServicesResourceNodeTests.test, I can see that both the POST and PUT data is packaged as I suggest in my post #3, above.

So, I think that is the solution. Give me a few beats (since it is end of day here) and I will bring my api_shell & api_shell_tests up to date with this solution and post them here, or somewhere else you direct me to place them.

BTW, thank you so much for your work here. I've privately told Hugo that I believe Services is the most significant Drupal module, period. I'm not really a 'web guy', and Services enables me to create critical systems for my business.

bsenftner’s picture

Status:Active» Closed (works as designed)
StatusFileSize
new11.65 KB

As noted in post #7 above, the original Services code is correct, but the example for calling both POST & PUT operations from java script at http://drupal.org/node/783460 is incorrect.

Attached is a zip file containing two modules:
api_shell.module an example showing how to setup a RESTful API, with the CRUD operators working except for index parameters
api_shell_test.module a java script testing framework for verifying that the API setup in api_shell.module works

camil.sumodhee’s picture

As a consequence in the 6.x-3.0-beta2 version
If you need to POST or PUT a resource using an XML representation, you will have to parse your XML file as a json string or php array right inside your 'rest request parsers' handler.
This was not necessary in version 6.x-3.0-beta1 as the posted data was passed in 1 big chunk (provided you had specified $info['type'] == 'array' || $info['type'] == 'struct')

== Example : Start

<?php
function test_services_resources() {
return array(
'res' => array(
     
'create' => array(
       
'help' => 'Post Ressource',
       
'file' => array('file' => 'inc', 'module' => 'test'),
       
'callback' => '_test_res_create',
       
'access callback' => '_test_res_access',
       
'access arguments' => array('create'),
       
'access arguments append' => FALSE,
       
'args' => array(
          array(
           
'name' => 'data',
           
'type' => 'array',
           
'source' => 'data',
           
'optional' => FALSE,
          ),
        ),
       
'rest request parsers' => array('text/xml' => '_test_xml_parser'),
      ),
  )
)

// == The xml parsing function == //
function _test_xml_parser($handle) {
 
//Retrieve POST data : i.e. Resource as an XML representation
 
$data = RESTServer::contentFromStream($handle);
 
//Parse the XML representation into a PHP array and return
  //This way the resource will be passed to the 'access callback' (i.e. _test_res_access) as an array.
 
return parseXMLtoArray($data);
}
?>

== Example : End

bsenftner’s picture

Camil,

Thank you for your post! I was just sitting down to write code that would have required exactly what you just posted.

Question: where does your parseXMLtoArray() routine come from? As I Google around, I'm seeing various resources for conversion to and from XML/PHP-Arrays. Yours looks very compact and easy to use, verses many I'm finding on the 'net with apparent complexity that seems unnecessary to me...

My Googling has also revealed various Drupal.org forum posts recommending the QueryPath.module for working with XML. I'm looking into that too... (Wow, just finished reading/watching info about QueryPath. If you're unfamiliar, it's worth your time.)

camil.sumodhee’s picture

StatusFileSize
new1.49 KB

Hi,
I have attached the class doing the parsing job.
In my case the xml file is posted as part of a multipart form data
Beware : the multipart form parsing process is a quick hack to get to my xml data! Don't use this class for anything but testing purpose!
Call the parsing function like this :

<?php
function _test_xml_parser($handle) {
 
$data = RESTServer::contentFromStream($handle);
 
$x = new testXmlParser();
 
// The 2nd parameter is the name of the key of the array returned
  // It should be set according to the argument name in your resource definition
  // required in order for $sources[$info['source']][$info['name']] to return expected data in the callback !
 
return $x->parseXmlToArray($data, "data");
}
?>

Hope this will help!