Ouath required for discogs search and image API calls

This is really unfortunate, and not an easy fix at all.

To be clear, images don't require an API key (which I have). They require OAuth authentication. This is significantly more difficult to implement in the code.

It's entirely possible that the Discogs.com Provider sub-module may end up being dependent upon the OAuth module. This, in turn, depends upon PECL's OAuth extension, which does not come with PHP by default.

Also, OAuth is designed to be set up on a per-user basis. That means that the user gets redirected to a URL, where they log in to Discogs.com, and get some sort of authorization key that the "application" (here, Drupal) stores to use for all users' requests to the API. At least, that's how I understand it... I've never actually dealt with OAuth before.

So, yeah, this really sucks. I'm going to ask about it on the Discogs.com forums, and see what's going on.

I think you are right, I am pretty sure they switched to the OAuth to limit the load on their servers, but I am sure it is a hassle to change your code.

That is exactly why they switched to OAuth. And the hassles won't just be limited to changing my code.

Here's how Discogs.com, a Drupal website, and the Drupal site's users interact:

1. Drupal site administrators download and install the Discogs modules
2. Drupal site administrators MUST register as a developer with Discogs
3. Drupal site administrators get a Consumer Key and Consumer Secret key
4. Drupal site administrators store these keys in their Drupal site
5. Drupal site USERS import releases from Discogs. When that happens:
6. Drupal software sends consumer key and a secret key to the Discogs.com "Request token URL" to get an OAuth request token.
7. Drupal site users are redirected the Discogs.com "Authorize URL" sent with the request token.
8. Drupal site users sign in with their Discogs.com username and password.
9. After login, Drupal site users are redirected back to a URL that must be set up in the Developer Settings on Discogs.com. This URL would likely be the URL on the Drupal site to start importing releases (I guess).

As you can see, this is horrifically complicated, not just from my perspective, but from the perspectives of Drupal webmasters and site users. But that's how OAuth works, and there's no way around it.

From what I can tell, the Drupal OAuth module handles most of the latter steps (those after getting the Consumer Key and Consumer Secret). At least, those that happen without user interaction - site administers would still have to register as a Discogs.com developer, and site users would still have to log in to Discogs.com. I'm not sure what to do about the redirect URL, or if it's part of OAuth or specific to Discogs.com.

I have been following updates to this module for some time.

I see that In the last few months there have been a lot of API announcements, and the Changelog has been fairly active as well.

Assuming that things will soon stabilise, now might be a good time to get some Drupal 7 code working that imports Discogs data from current endpoints. I am thinking that with Drupal 8 coming along, there will be less enthusiasm to write 7 code in the future.

I would be interested to read any ideas on this!

I have a working discogs API integration running in a custom module I wrote for a group of radio stations.

The module grabs information from the station's automation system about what track is playing. There is a node type 'artist' which the drupal system checks the now playing info against. If no artist exists in the drupal app, it makes a call to discogs to retrieve bio and images and creates a new artist node.

I got oauth authentication working using the oauthsimple php library and some example code from the discogs forum.

See

https://github.com/jrconlin/oauthsimple/tree/master/php

http://www.discogs.com/forum/thread/53219ff8a86b6d57a82081f2#53219ff8a86...

The site adminstrator has to manually authorise the site to communicate with discogs. This is done once, thereafter the system is able to make calls to the API using the prior authentication.

Here's my authentication test function. There's some module specific stuff - variable_gets for API key and secret etc, but it might be a good starting point for integration into the discogs API module.

Note I originally wrote my custom module before this discogs module existed. If the discogs API module could be updated to support ouath, and offer some abstract methods, I would happy rewrite my module to use it, and put my efforts into this module!

Anyway, here's my oauth test page callback.

/**
 * Test fetch via oauth.
 */
function radio_playlist_discogs_oauth_test() {
  $output = array();


  // This example uses the OAuthSimple library for PHP
// found here:  https://github.com/jrconlin/oauthsimple/tree/master/php
//
// For more information about the OAuth process for applications
// accessing Discogs API, read:
// http://www.discogs.com/developers
  // Try to load the library and check if that worked.
  if (($library = libraries_load('oauthsimple')) && !empty($library['loaded'])) {
    $oauthObject = new OAuthSimple();
    $scope = 'http://api.discogs.com';

// Initialize the output in case we get stuck in the first step.
    $output['auth'] = array(
      '#type' => 'markup',
      '#markup' => 'Authorizing...<br/>',
    );
    $signatures = array(
      'consumer_key' => variable_get('radio_playlist_discogs_key'),
      'shared_secret' => variable_get('radio_playlist_discogs_secret'),
    );

    if ($oauth_props = variable_get('radio_playlist_oauth_props')) {

      $oauth_props = $oauth_props + $signatures;

      $output['props'] = array(
        '#type' => 'markup',
        '#markup' => 'props are set <br/>',
      );
      // reset the oauth object
      $oauthObject->reset();
      // rebuild it with the URL of the resource you want to access and the token/secret
      $result = $oauthObject->sign(array(
        'path' => "$scope/oauth/identity",
        'signatures' => $oauth_props)
      );

      // Now that we have our signed URL, we can make one more call to the API
      // which will grant us access to an authenticated resource
      // such as http://api.discogs.com/oauth/identity

      $url = $result['signed_url'];

      $ch = curl_init();
      curl_setopt($ch, CURLOPT_USERAGENT, 'YOUR_USER_AGENT/0.1 +http://www.2ch.com');
      curl_setopt($ch, CURLOPT_URL, $url);
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

      //Execute the curl session
      $curl_result = curl_exec($ch);

      curl_close($ch);

      // print the JSON output to the page
      $output['result'] = array(
        '#type' => 'markup',
        '#markup' => 'result is ' . $curl_result,
      );
    }
    else {
      // Check if verifier exists.  If not, get a request token
      if (!isset($_GET['oauth_verifier'])) {
        // To get a Request Token, we make a request to the OAuthGetRequestToken endpoint,
        // submitting the scope of the access we need (api.discogs.com)
        // and also tell Discogs where to redirect once authorization is submitted
        $result = $oauthObject->sign(array(
          'path' => 'http://api.discogs.com/oauth/request_token',
          'parameters' => array(
            'scope' => $scope,
            'oauth_callback' => url('radio_playlist/oauth',array(
              'absolute' => TRUE,
            ))),
          'signatures' => $signatures));

        // The above object generates a simple URL that includes a signature, the
        // needed parameters, and the web page that will handle our request.
        // Using the cUrl libary, we send a GET request to the signed URL
        // then add the response into a string variable ($r)
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_USERAGENT, 'YOUR_USER_AGENT/0.1 +http://www.2ch.com');
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_URL, $result['signed_url']);

        $r = curl_exec($ch);
        curl_close($ch);

        // Then we parse the string for the request token and the matching token secret.
        parse_str($r, $returned_items);
        $request_token = $returned_items['oauth_token'];
        $request_token_secret = $returned_items['oauth_token_secret'];

        // We store the token and secret in a cookie for later when authorization is complete
        setcookie("oauth_token_secret", $request_token_secret, time() + 3600);

        // Next we generate a URL for an authorization request, then redirect to that URL
        // so the user can authorize our request.
        // The user could deny the request, so we should add some code later to handle that situation
        $result = $oauthObject->sign(array(
          'path' => 'http://www.discogs.com/oauth/authorize',
          'parameters' => array(
            'oauth_token' => $request_token),
          'signatures' => $signatures));

        // Here is where we redirect
        header("Location:$result[signed_url]");
        exit;
      }
      else {
        // If we have a oauth_verifier, fetch the cookie and amend our signature array with the request
        // token and secret.
        $signatures['oauth_secret'] = $_COOKIE['oauth_token_secret'];
        $signatures['oauth_token'] = $_GET['oauth_token'];

        // Build the request-URL
        $result = $oauthObject->sign(array(
          'path' => 'http://api.discogs.com/oauth/access_token',
          'parameters' => array(
            'oauth_verifier' => $_GET['oauth_verifier'],
            'oauth_token' => $_GET['oauth_token']),
          'signatures' => $signatures));

        // ... and get the web page and store it as a string again.
        $ch = curl_init();
        //Set the User-Agent Identifier
        curl_setopt($ch, CURLOPT_USERAGENT, 'YOUR_USER_AGENT/0.1 +http://www.2ch.com');
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_URL, $result['signed_url']);
        $r = curl_exec($ch);

        // parse the string to get you access token
        parse_str($r, $returned_items);
        $access_token = $returned_items['oauth_token'];
        $access_token_secret = $returned_items['oauth_token_secret'];


        // We can use this long-term access token to request Discogs API data,
        // for example, the identity of the authenticated user.
        // All Discogs API data requests will have to be signed just as before,
        // but we can now bypass the authorization process and use the long-term
        // access token you hopefully store somewhere permanently.
        $oauth_props = array('oauth_token' => $access_token,
          'oauth_secret' => $access_token_secret);

        variable_set('radio_playlist_oauth_props', $oauth_props);

        // reset the oauth object
        $oauthObject->reset();

        // rebuild it with the URL of the resource you want to access and the token/secret
        $result = $oauthObject->sign(array(
          'path' => "$scope/oauth/identity",
          'signatures' => $oauth_props)
        );

        // Now that we have our signed URL, we can make one more call to the API
        // which will grant us access to an authenticated resource
        // such as http://api.discogs.com/oauth/identity

        $url = $result['signed_url'];

        curl_setopt($ch, CURLOPT_USERAGENT, 'YOUR_USER_AGENT/0.1 +http://www.2ch.com');
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

        //Execute the curl session
        $curl_result = curl_exec($ch);

        curl_close($ch);

        // print the JSON output to the page
        $output['result'] = array(
          '#type' => 'markup',
          '#markup' => $curl_result,
        );
      }
    }
  }
  else {
    watchdog('radio_playlist', 'unable to load oauthsimple libraray', array(), WATCHDOG_WARNING);
    drupal_set_message(t('Unable to load library'), 'error');
  }

  return $output;
}

By the way, you can see the results of my module in action at
http://www.2ch.com

:)

Module no longer functions under Drupal 6, returns "Error importing from Discogs.com"

I am heavily invested in Discogs and Drupal 6, and would be willing to pay for either a fix or upgrade path. PM.

Thanks for all recent responses. I think:

Version 7.x

The current module provides a multi-step form through which a user makes a request for “discography” data. Typically this is a search request to Discogs API. Maybe only a small amount of script changes are needed to make a successful search request with OAuth. At the same time there's a lot of potential for this module to do great stuff, with more changes, along the lines of the radio station module (maybe incorporated into a version 8 module?).

OAuth and OAuth libraries

OAuth has its share of supporters within the Drupal community, so I don't know if people using the Discogs module, will find other uses for OAuth but it's possible.

An OAuth library is used to simplify the process of authenticating. In the example on this page, with $oauthObject as the library object - you get:

$result = $oauthObject->sign(array(
          'path' => "$scope/oauth/identity",
          'signatures' => $oauth_props)
        )

There are various libraries compatible with versions of OAuth such as php-proauth library and php-proauth. The use of a library could break some scripts if there are conflicts between libraries. For reference Discogs used OAuth version 1.0a.

Provide a structured url example

To start with an example structured url would be helpful, maybe added to the help section of the module. The Discogs API provides an example search string without OAuth authentication.

Converted to PHP variables, the string can be divided into parts: a base URL and some query data). The query data can be further broken down into an array of key/value query pairs.

More housekeeping

The OAuth variables can be stored in the Drupal database. As an example do this:

name = Discogs OAuth settings
description = Discogs OAuth settings form
core = 7.x

configure = admin/config/content/discogs_oauth_settings

/**
 * Page callback: Current Discogs OAuth settings
 *
 * @see discogs_oauth_settings_menu()
 */
function discogs_oauth_settings_form($form, &$form_state) {
  $form[' discogs_oauth_settings_consumer_key'] = array(
    '#type' => 'textfield',
    '#title' => t('Consumer key'),
    '#default_value' => variable_get('discogs_oauth_settings_consumer_key'),
    '#size' => 75,
    '#pxlength' => 75,
    '#description' => t('consumer key from your Discogs API application'),
  );	
  $form[' discogs_oauth_settings_consumer_secret'] = array(
    '#type' => 'textfield',
    '#title' => t('Consumer secret'),
    '#default_value' => variable_get('discogs_oauth_settings_consumer_secret'),
    '#size' => 75,
    '#pxlength' => 75,
    '#description' => t('consumer secret from your Discogs API application'),
  );  
  $form[' discogs_oauth_settings_token'] = array(
    '#type' => 'textfield',
    '#title' => t('Access token'),
    '#default_value' => variable_get('discogs_oauth_settings_token'),
    '#size' => 75,
    '#pxlength' => 75,
    '#description' => t('OAuth access token from successful series of requests to Discogs API endpoint'),
  );
  $form[' discogs_oauth_settings_token_secret'] = array(
    '#type' => 'textfield',
    '#title' => t('Access token secret'),
    '#default_value' => variable_get('discogs_oauth_settings_token_secret'),
    '#size' => 75,
    '#pxlength' => 75,
    '#description' => t('OAuth access token secret from successful series of requests to Discogs API endpoint'),
  );  
  return system_settings_form($form);
}

Utilising PHP's CURL support

PHP's curl support is used in making request to Discogs API. It is used in the examples process of OAuth authentication on this page.

Thus:

the script does it's stuff to make a signed request, whether this for testing, finding variables to store, control a request process.

And, as a suggestion, we can start by using curl for sorting out problems with OAuth authentication .before moving onto changes to the current module (which doesn't use curl).

discogs_prov.module

The discogs_prov module has function discogs_prov_discog_search. It has a variable

  // Build the query data array
  $query_data = array(
    'q'        => $term,
    'type'     => $type,
    'page'     => $page,
    'per_page' => $per_page,
  );

used to build a structured URL.

Used at:

l.365 (new)

 $url .= '?' . http_build_query($query_data);

where http_build_query turns the parameter $query_data array into a string appended to a url. Actually looking at the Drupal API there is an equivalent function drupal_http_build_query

Now to encompass OAuth we need to add to query_data stuff / headers. And the data returned by the simpleoauth function sign gets added as key/value pairs.

Other variables

So we have a structured url for this search request, the way the multi-step form works need to consider there are other variables that should encompass OAuth authentication.

Therefore presumably need to have to build an array of query data, of key/value pairs each time (add OAuth to headers) for

l. 157

  $url  = 'http://api.discogs.com/';

l.163 (new)

 $url .= $type . 's/' . $id . '/releases';

l.180 (new)

$url = 'http://api.discogs.com/' . $type . 's/' . $id;

l.92 (new)

$url  = 'http://api.discogs.com/database/search';

l.249 (new)

$url  = 'http://api.discogs.com/releases/' . $id;

drupal_http_request()

If switching from implementation of curl then look at swapping over some curl_set_opt() variables to drupal_http_request options.

Parse returned result

If all goes well a $json_array should created which can be used to create content as before.

Perhaps using the OAuth (+Connector) modules will help. I haven't quite got my head around OAuth for web sites as one user (or OAuth in general, for that matter).

Hi,
Is there anyone that got the module up and running?
#push

#push#push

Hey, it's Karl again. Sorry that I've been so quiet on this issue. I was in college full-time. I just graduated, and hopefully I can start fixing this.

There is actually some good news on this front. Because nobody liked working with OAuth for simple queries, Discogs.com created a new authentication service called "Discogs Auth." Essentially, you have to use HTTPS, and send the key/secret pair (or a token) either in the URL (easiest), or as part of the HTTP headers.

But, either way, this can be sent once the webmaster gets a Consumer Key and Consumer Secret - which is only done once. These can be stored as Drupal variables and sent automatically, so there wouldn't be any monkeying about with users having to be redirected and signing in on Discogs.com itself.

I've posted some questions about this on the Discogs forum (particularly if this will work with images), but if this type of authentication works, then it should be much easier to implement than OAuth would be.

Please don't forget about version 6!

Good news, everyone! Version 7 is working. Off to fix Version 6 right now.

BUT, note that you will still have to get a Consumer Key and Secret from Discogs.com. Yes, this means that you have to register as a developer. There's no way around this.

Patch against 7.x-1.x-dev is attached.

v6 please!

skurkee: It certainly works on my test machine. Did you try clearing the caches (admin/config/development/performance)? If I remember correctly, this will clear out the menu cache as well.

skurkee: It appears that the module won't let you enter the Key/Secret values. This is an issue that is different from OAuth authentication - it seems to be a problem with the Drupal module, not the Discogs API. It's a separate issue, so I created this: #2494843: Not able to enter OAuth keys

Having said that, I can't seem to reproduce this on my test machine. Once I can find the error, I will fix it.

FYI, that latest issue is fixed (I hope). The fix is pushed to dev, and there's a patch on the issue: #2494843: Not able to enter OAuth keys.

It was a stupid mistake on my part - the admin file wasn't checked into the repo.