I recently discovered https://www.drupal.org/about-drupalorg/api and, while writing a script to use the API, I found an issue. If you make a request to https://www.drupal.org/api-d7/node.json?type=project_module the response includes paging data in the form of first, next, previous, last, etc links. These links are in the form of https://www.drupal.org/api-d7/node?type=project_module&page=1 which does not include the format (e.g. /node.json is now just /node). If you try to request this formatless URL, drupal.org issues a 301 redirect to node.json but it drops all query parameters. This causes my script to endlessly call https://www.drupal.org/api-d7/node.json.

There are two issues that I can see here, either of which I think would solve the problem:

1. The restws module shouldn't strip formats in paging data #2253947: format suffix not added to next-first-last page url's
2. Drupal.org shouldn't drop query parameters on 301 redirect

In a quick test of restws, I couldn't duplicate the redirects d.o is doing so I'm posting here to solve that specific problem.

GET https://www.drupal.org/api-d7/node?type=project_module HTTP/1.0
Host: www.drupal.org
User-Agent: get_list_of_modules.php from https://github.com/rocketeerbkw/drupal_function_stats
Accept: application/json

HTTP/1.0 301 Moved Permanently
Age: 0
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
Date: Tue, 28 Oct 2014 08:33:12 GMT
Etag: "1414485192"
Expires: Tue, 28 Oct 2014 08:33:11 GMT
Front-End-Https: on
Last-Modified: Tue, 28 Oct 2014 08:33:12 +0000
Location: https://www.drupal.org/api-d7/node.json
Server: nginx/1.4.4
Vary: Accept-Encoding
Via: 1.1 varnish
X-Cache: MISS
X-Cache-Svr: www7.drupal.org
X-Drupal-Cache: MISS
X-Varnish: 1671098371
Connection: close

CommentFileSizeAuthor
#2 restws-query_string_drop_on_redirect-2364755-2.patch630 bytesrocketeerbkw
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm commented
Project: Drupal.org infrastructure » RESTful Web Services
Version: » 7.x-2.x-dev
Component: Servers » Code

I think this 301 is coming from RestWS too. Specifically, the call to drupal_goto() in restws_page_callback(), http://cgit.drupalcode.org/restws/tree/restws.module#n322.

rocketeerbkw’s picture

rocketeerbkw CreditAttribution: rocketeerbkw commented
Title: Query parameters dropped when d.o API issues 301 redirect » Query parameters dropped when redirected from /node to /node.json
Status: Active » Needs review
FileSize
restws-query_string_drop_on_redirect-2364755-2.patch630 bytes

In order to reproduce this, you must have page caching enabled, then just send a request with the correct Accept header to /node?page=0 and you'll see a 301 redirect to /node.json.

I've attached a patch to make sure any query parameters are kept.

Media Crumb’s picture

Media Crumb CreditAttribution: Media Crumb commented

Is there a reason its redirected to the .json counter part if there is no cookie session set? Since RestWS is all but completely broken with the new version of Drupal and it no longer working with .json this creates yet another hurdle in getting apps to work.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented

Is there a reason its redirected to the .json counter part if there is no cookie session set?

From the code:

        // Consumers should not use this URL if page caching is enabled.
        // Drupal's page cache IDs are only determined by URL path, so this
        // could poison the HTML page cache. A browser request to /node/1 could
        // suddenly return JSON if the cache was primed with this RESTWS
        // response.
        if ($format && !isset($_COOKIE[session_name()]) && variable_get('cache')) {

Drupal’s core page cache isn’t aware of accept headers, so it would get poisoned. If you have an external caching layer like Varnish or a CDN, you can turn off the core page cache. Be sure that the external caching is aware of accept headers, and continues to cache as expected with any different cache-related headers Drupal outputs when the internal page cache is off.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Issue tags: +affects drupal.org
Related issues: +#2926350: Pagination links in Drupal.org api-d7 are broken

  • drumm committed 5ef3160 on 7.x-2.x authored by rocketeerbkw 
    Issue #2364755 by rocketeerbkw: Query parameters dropped when redirected...
drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Status: Needs review » Fixed

Patch works well, committed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.