I know you're not responsible for the behavior of the Services Module or the services_oauth module, but I think this problem is in some way related to the Oauth module itself. It may be connected to http://drupal.org/node/1987350.

Background
I am trying to use Oauth in conjunction with the Services Module. I set up 2-Legged Authentication and have successfully been able to perform GET operations on my endpoint. However, when I try to perform POST or PUT operations, I get an error.

Steps to Reproduce:

  1. Follow Instructions here to set up Oauth with 2-Legged Authentication: http://drupal.org/node/1871498. Exceptions:
    1. Install the latest version of Oauth 7.x-3.x-dev from May 6, 2013 as that covers the problems mentioned on there and also fixes http://drupal.org/node/1987350
    2. Don't bother setting up a view. Instead, select "node" from the RESOURCES tab on the Service.
  2. Make sure the Drupal User you create as your "Oauth User" has sufficient permissions to create content.
  3. Send a GET command to endpoint/node just to test that it works.
  4. Send a POST command to endpoint/node to try to create a new node.

Expected Result
The services module should create a new node and should return the node id.

Observed Results
It throws an OAuthException. On the client end it says:

Invalid auth/bad request (got a 401, expected HTTP/1.1 20X or a redirect)

In my watchdog logs on the server site it says:

ServicesException::__set_state(array(
   'data' => 'Invalid signature',
   'message' => 'Invalid signature',
   'string' => '',
   'code' => 401,
   ...

Details
I followed this tutorial to get things up and running at first, though I've switched from using a View to using node as the resource: http://drupal.org/node/1871498

Here's my code where I'm trying to make the post:

define('EFEED', 'http://example.com/services/node');

$consumer_key = 'MY_CONSUMER_KEY';
$consumer_secret = 'MY_CONSUMER_SECRET';

$oauth = new OAuth($consumer_key, $consumer_secret, OAUTH_SIG_METHOD_HMACSHA1, OAUTH_AUTH_TYPE_URI);
$oauth->enableDebug();

// First, I try to download the node index. This works without any problems
print "<h1>Downloading a Node</h1>";

try {
  $data = $oauth->fetch(EFEED);
} catch (Exception $e) {
  die($e->getMessage());
}

$response_info = $oauth->getLastResponse();
$nodes = json_decode($response_info);

print "<pre>";
print_r($nodes);
print "</pre><hr />";

// Next I try to post a New Node to the page.
// I have verified that my service is set up properly. When I disable OAuth Authentication on the service
// and set Drupal permissions to allow anonymous users to create content, this code succesfully
// posts a node. But when I turn authentication on, I get the 401 error.
print "<h1>Creating a Node</h1>";

try {
  $oauth->fetch(EFEED, http_build_query(array(
      'body' => 'This is a new node for the sake of testing...',
      'title' => 'New Node',
      'type' => 'page',
    ), NULL, '&'), OAUTH_HTTP_METHOD_POST);
} catch (Exception $e) {
  die($e->getMessage());
}

My Thoughts
POST Works perfectly if I disable Oauth Authentication and give Anonymous users the right to post content. But when I enable Oauth, I get errors. I don't get Errors when I GET, only when I POST.

Since I don't want just anyone to be able to execute POST, PUT and DELETE operations through my API, I really want OAuth authentication to work...

Why would GET and POST authenticate differently? Am I missing a step?

My best theory at this point is that the problem is with the authorization of my user. I recently downloaded the 7.x-3.x-dev release that was updated on May 6 in order to fix http://drupal.org/node/1987350, but that only takes care of part of the problem.

When I try to authorize the user (user/%uid/oauth/consumer/%csid/add-authorization), I don't get those errors anymore, but I still can't authenticate. And the table at user/%uid/oauth/authorizations is still blank.

There seems to be a problem looking up the authorization from the oauth_common_token table or something. When I create the authorization, it outputs a hash saying "The token wVRkW7MnJSzF9ugN7r7aPDQQeQumc5WQ has been updated."

When I look directly at the database using PHPMyAdmin, I can find that token in the oauth_common_token table. In fact, I see several tokens associated with this uid, presumably from my many attempts to authorize this user. But the table at user/%uid/oauth/authorizations isn't able to see them for some reason. Maybe that's what's behind my Invalid Signature problem?

Thoughts?

Thanks.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

seismicmike’s picture

Component: providerui » Code
Assigned: Unassigned » seismicmike

So I did some investigation of this yesterday. It seems that the signature being generated by the client is not the same signature that is built by the OAuthServer.

It seems that the OAuth object that I create in my client request generates a signature using the HMAC-SHA1 method and the DrupalOAuthServer object generates a signature on the receiving end which uses the OAuthSignatureMethod_HMAC object with an "algo" of "sha1". And the keys that are generated don't match. The OAuthSignatureMethod::check_signature function walks through the keys to check them character by character and they don't match. For kicks, I had watchdog log both and sure enough:

Array
(
    [Client Signature] => z+ynLrnzu+f10YnvrIUD/ce5chA=
    [Signature Built by Server] => 9i1rjttgSgXCfj3std+xLN3nf9Q=
)

Don't know why this happens...

Oh there is one other problem too. I figured out why the Authorizations tab is not showing anything. Evidently there are two types of tokens: Request tokens and Access tokens. The token's type is stored as a tinyint in the oauth_common_token.type field in the database. 0 = Request; 1 = Access.

It would seem that both when the Authorizations table is being loaded and when the OAuthServer object is loading the token, it looks for Access tokens, ignoring the Request tokens. I couldn't actually find any instance where Request tokens were looked up. The trouble is, that the Token object has a default value for type of Request that is never changed before the token is written to the database. So no tokens match the Authorizations table's query.

But even when I manually changed the type field on one of my tokens to 1 using phpmyadmin, I was still getting the error in which the signatures built by the client and server don't match.

juampynr’s picture

I found out that I could not perform POST operations either because I had to enable application/x-www-form-urlencoded at the Services Server tab.

Is the encryption method HMACSHA1 enabled in the Services server configuration?

Can you also confirm that the service works as expected by writing a 5 line script using Guzzle? I am using it to sign requests for two-legged authentication from a Symfony website that consumes services from a Drupal + Services site successfully.

It would be great if we get to the bottom of this. Looking forward for your feedback.

seismicmike’s picture

I have created http://drupal.org/node/1994506 for the Token type issue related to the Authorizations table. This issue should focus on the signature matching problem.

seismicmike’s picture

Thanks for the feedback. I also found the problem with the application/x-www-form-urlencoded problem as well, but when I got around it, I was still having the signature matching problems.

The only place I know to set the HMACSHA1 as the signature method is when I edit the context on the OAuth config. I do have it set to HMAC-SHA1 and only HMAC-SHA1 there. The Services module does not seem to provide have its own config for that. I'll keep looking to see if it perhaps does.

I'm not familiar with Guzzle. I'll look into it.

seismicmike’s picture

Just found Guzzle. I'll see if I can build a client using it. I was using the OAuth module directly in straight php. Maybe Guzzle will help.

seismicmike’s picture

juampy,

I have written the following Guzzle code

require_once __DIR__ . '/vendor/autoload.php';
use Guzzle\Http\Client;

$client = new Client('http://example.com');
$client->addSubscriber(new Guzzle\Plugin\Oauth\OauthPlugin(array(
  'consumer_key' => 'NAW3ADrFiPAtXn5jPjBVBwyWf2pWpHec',
  'consumer_secret' => 'F3KT2L9bZmUzQLAckDLNcjVEeMiCjBZF',
  'token' => '3PZ2vEXbvnG8APAyqsWax7aKen6NtjN6',
  'token_secret' => 'AfMbf3U6M2nvTDcEwgwWVybDUtsANDwN',
)));

$list = $client->get('nodes/node');
$list_response = $list->send();
echo $list_response->getBody();
echo $list_response->getHeader('Content-Length');

$add = $client->post('/nodes/node', null, http_build_query(array(
  'body' => 'New body content',
  'title' => 'Title of New Post',
  'type' => 'page',
), NULL, '&'));

$add_response = $add->send();
echo $add_response->getBody();
echo $add_response->getHeader('Content-Length');

The get command returns the list of nodes. The post command returns "Invalid Signature".

If I turn off OAuth and swith authentication to Session and try:

$client = new Client('http://example.com');
$add = $client->post('/nodes/node', null, http_build_query(array(
  'body' => 'New body content',
  'title' => 'Title of New Post',
  'type' => 'page',
), NULL, '&'))->setAuth('OAuth User', 'password');

$add_response = $add->send();
echo $add_response->getBody();
echo $add_response->getHeader('Content-Length');

This works as long as I give anonymous users the right to post content. It doesn't succeed in logging me in as the OAuth User before posting, though. If I remove the "Create content" Permission for OAuth users, then it tells me access denied for user Anonymous.

Nevertheless, I think this demonstrates that the services module is functioning properly, but that the signatures just aren't matching for some reason...

Thanks for your help.

seismicmike’s picture

This may indeed be a problem with the underlying library: http://stackoverflow.com/questions/7093878/php-oauthprovider-library-inv...

seismicmike’s picture

I was able to get the REST API to function properly when using the services_auth_basic module, which allows me to use the username/password method for auth. I will proceed using this for now. I would prefer the higher security of OAuth in the long run, so I will be following this issue, but I know how things can be slow sometimes :).

Let me know if there's any way I can help.

Jaypan’s picture

I am also seeing this issue. I get "Unauthorized: Invalid signature" responses on each request.

Jaypan’s picture

This may indeed be a problem with the underlying library

Looks that way. I just switched to the oauth-php library and I'm now able to POST to the server without issue.

kbadelt’s picture

Priority: Normal » Major

I'm having the same issue. GET works, PUT doesn't.

Certainly not my decision but shouldn't this issue be elevated to 'major' priority? After all, neither POST nor PUT work which seems substantial.

@Jaypan: How did you "switch to the" other library?

Jaypan’s picture

Are you using the PECL OAuth library for PHP? This is the library I was using to connect, and PUT wasn't working. Switching to a different PHP OAuth library oauth-php instead of the PECL library solved the problem. So the problem was in the OAuth library I was using, not in the Drupal OAuth library that this issue queue is referring to.

kbadelt’s picture

Thanks for the quick response, and sorry if this is a beginner question.
I didn't install an OAuth lib with PECL - I thought oauth_common is using the OAuth.php library in oauth/lib? In any case, how do you configure oauth_common to use oauth-php (which I downloaded)?

Jaypan’s picture

First, let's clarify. There are a few parts to OAuth, but the main two are the OAuth server, and the OAuth client. The OAuth module provides an OAuth server. It may also provide an OAuth client, but I'm not using it for that myself.

I was using a completely Drupal-unrelated library for my OAuth client, the PECL OAuth extension. This library had an underlying issue with the PUT issue, when posting from the client, to the OAuth server.

It sounds like you may be using the OAuth module for your OAuth client. If the OAuth library at /oauth/lib is a modified version of that library, you may be facing the same issue. I don't know if that is the case however.

kbadelt’s picture

I'm using the OAuth module with Services as server. Clients are Ruby with the OAuth Gem, and also for testing REST Console within Chrome.

The PUT and POST signature error of this issue appears with both, so the culprit might be the underlying library at /oauth/lib of the Drupal module as Jaypan says. I tried 7.x-3.x and even 7.x-4.x ("hidden" in git) branches of the module, same result. I also tried both 2-legged and 3-legged.

As GET works, I assume my setup is correct (otherwise GET would't authenticate either).

Just for reference here's my Ruby client code:

require 'oauth'

# oauth authentication
@consumer = OAuth::Consumer.new OAUTH_CONSUMER_KEY, OAUTH_CONSUMER_SECRET, :site => WEBAPP_URL
@access_token = OAuth::AccessToken.new @consumer
# so far it works great. @access_token is valid.

# updates[] is hash of field_name => value
response = @access_token.put "/node/#{job_id}.json", updates
# => 401 "Invalid signature"

@Jaypan: did your get your client (using the other oauth-php library) to work with the OAuth module as server?

Jaypan’s picture

@Jaypan: did your get your client (using the other oauth-php library) to work with the OAuth module as server?

Yes, I'm using Services 3, the Oauth module, and the oauth_services (or services_oauth?) module to provide an oauth server. I was able to use the other oauth-php library and successfully connect to the server.

kbadelt’s picture

Thanks, Jaypan. Unfortunately, neither the Ruby oauth Gem nor the REST console mentioned above can PUT or POST to Drupal Services with OAuth. Seems others here have the same issue with other client libraries.

While I was able to make it work using PLAINTEXT signature_method, the default HMAC-SHA1 does NOT work. As PLAINTEXT defeats the security purpose this is no workaround though.

Jaypan’s picture

In that case, it may be that the problem wasn't in my underlying PHP Oauth library, but rather in the module OAuth library. Maybe the new library I'm using just happens to work with the idiosyncrasies of the server library.

On that note, I think the priority of 'Major' sounds correct to me.

seismicmike’s picture

Just checked in on this issue. Jaypan, thanks for the suggestion on the oauth-php library. I will have to check that out when I circle back around to this.

I appreciate the help :)

torotil’s picture

I've had the problem as well. I found a solution: It seems that DrupalOAuthRequest::from_request() includes $_GET['q]' in the list of parameters to calculate the signature. This is definitely a violation of the oauth standard. I'm going to post a patch as soon as I've figured outh that function.

torotil’s picture

Status: Active » Needs review
FileSize
689 bytes

Ok the logic of DrupalOAuthRequest::from_request() is based on the assumption that $_SERVER['QUERY_STRING'] holds the original value of $_GET['q'] (before the request was rewritten using clean_urls). This doesn't seem to hold for newer PHP versions. I've patched the module to use REQUEST_URI instead.

torotil’s picture

Version: 7.x-3.1 » 7.x-3.x-dev
torotil’s picture

Here is an more invasive patch that completely simpifies the method: When parameters aren't passed explicitly then the QUERY_STRING is always set to the original query-string from REQUEST_URI.

torotil’s picture

Sorry attached the wrong file …

ldpm’s picture

I have a similar issue when attempting 2-legged OAuth authorization on a REST Service. All OAuth transactions (GET or POST) were shedding OAuthExceptions with the "The request must be signed" error. In testing, I was using REST Console in Chrome, and the requests were definitely signed. Sending the same query against an unrelated but known-good RESTful endpoint using OAuth, I got the expected Invalid Signature error.

Using countless watchdog entries and this thread, I tracked the problem to the condition in DrupalOauthConsumer::loadProviderByKey, which was returning a hashed key that didn't match the hashed_key in the database. However, I ran a test where the whole loop was on localhost, so presumably client and server were using the same SHA1 hashing algorithm, same timestamp, etc. Not sure why they wouldn't match.

Commenting out the condition comparing the key_hash to "sha1($key)" made everything work. Since this doesn't seem to bypass the actual signature verification steps in oauth_common.inc, I'm not sure what the purpose of that comparison is, anyway. Anyway, I'll follow this in case seismicmike comes up with something.

I'm using:
OAuth 7.x-3.1
Services 7.x-3.5

ldpm’s picture

Issue summary: View changes

Forgot a link.

marcus178’s picture

I'm having the same issue using nodejs oauth as the client, all get requests work fine but post requests all return Invalid Signature.

I bit confused as to what the solution is.

torotil’s picture

marcus178: can you test this with the patch from #24 applied?

swafran’s picture

#10 worked for me. I had the same problem, could GET not POST. Changing client libraries to the one Jaypan suggested allowed me to do both.

tbenice’s picture

Issue still remains even when using the oauth-php library. POST works but PUT doesn't...signatures don't match.

tbenice’s picture

Also, #24 did not help, both PUT and POST requests with body (not query) data return with signatures don't match.

tbenice’s picture

More info,

I discovered that the algorithms are the same for signing requests for the oauth-php library HMAC-SHA1 and the oauth module HMAC (algo=sha1) methods.

Also, the keys match.
However the base_strings do not match:

On the client site (oauth-php):

POST&http%3A%2F%2Fmysite.local%3A8082%2Fcomx%2Forder%2F&oauth_consumer_key%3DiynDzKJuko7RfxDYsfNhuUuXiWfH7Rrx%26oauth_nonce%3D5495d2c1108dc%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1419104961%26oauth_token%3D%26oauth_version%3D1.0%26xoauth_body_signature%3DKFW8Y2RwRv220qwbr1T9QJjoGF8%253D

Which when decoded is:

POST&http://mysite.local:8082/comx/order/&oauth_consumer_key=iynDzKJuko7RfxDY...

On the server site (oauth module):

POST&http%3A%2F%2Fmysite.local%3A8082%2Fcomx%2Forder%2F&oauth_body_signature%3DKFW8Y2RwRv220qwbr1T9QJjoGF8%253D%26oauth_consumer_key%3DiynDzKJuko7RfxDYsfNhuUuXiWfH7Rrx%26oauth_nonce%3D5495d2c1108dc%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1419104961%26oauth_token%3D%26oauth_version%3D1.0

When when decoded is:

POST&http://mysite.local:8082/comx/order&oauth_body_signature=KFW8Y2RwRv220qw...

Not only are the parameters not matching in order, but you can see that the oauth_body_signature has an 'x' in the client side parameter.....investigating....

tbenice’s picture

Looks like the issue is with the oauth-php library sending a xoauth_body_signature to the server which the oauth.php does not recognize as an oauth parameter, thus the base_string(s) don't match and the signatures don't match.

Find/replace xoauth to oauth in the oauth-php library allows POST and PUT to proceed.

Cheers.

brettdonohoo’s picture

Did anyone successfully get PUT working with PECL? I'd prefer not to switch libraries on client side.

brettdonohoo’s picture

Edit: I had made a mistake in debugging. The server base string appears to be encoded properly. This comment can be removed.

Thanks.

mrconnerton’s picture

I recently had this issue and #23 seemed to do it for me when testing with rest console.

JamesK’s picture

Status: Needs review » Reviewed & tested by the community
MustangGB’s picture

#24 also solved this for me.