I've had a problem with using mydropwizard via a proxy (Squid 3.2) where the POST to MYDROPWIZARD_STATISTICS_URL is failing for two reasons.

First is that the client is POSTing raw JSON, but sending it as application/x-www-form-urlencoded. Squid validates the body and notices it's not correctly form encoded, so it refuses to send it. Given it's transmitting JSON, there needs to be a Content-type: application/json request header added.

Second is that the server is issuing a 301 redirect from HTTP to HTTPS for both the endpoints, for every request. We use curl as our drupal_http_request implementation, which doesn't handle a 301 on a POST the way you'd expect (it assumes it was a successful POST, and it needs to GET the Location it was redirected to). This leads to an "ERROR" response as it does a GET to the HTTPS endpoint. Seems like the configuration should just be updated to point to HTTPS in the first place, and avoid all those redirects for every module XML, which slows things down.

Have attached a patch that does both, against 6.x-1.x.

Thanks

Comments

neilnz created an issue. See original summary.

  • dsnopek committed 705e975 on 6.x-1.x authored by neilnz
    Issue #2733783 by neilnz: Missing content-type header on stats POST (and...
dsnopek’s picture

Status: Needs review » Fixed

Thanks for the report and patch! We really should have been using the https:// URL anyway to prevent man-in-the-middle attacks, and setting the content type to the actual content type of payload is obviously good. :-)

So, committed!

Any chance you'd be interested in writing a short blog post about how to setup a Squid proxy for Drupal update status information? We could post it or link to it from the mydropwizard.com blog? I think that'd be super useful information for the community. If so, shoot us an e-mail at support@mydropwizard.com - thanks!

neilnz’s picture

Our use of Squid is as a content filter and outgoing traffic audit for our production environment (imposed) rather than for caching updates. We replicate that in our dev environment, which is traditionally the only place we run update status.

Our organisation (Catalyst) has though set up a caching mirror of Drupal updates in Nginx as a separate thing to this proxy, due to NZ being geographically far from d.o which makes polling all the modules quite slow. Doing so requires ignoring upstream cache-control headers that d.o uses so they can collect site stats (we don't expose our site keys upstream). I didn't build that system, but we could look at extending it to D6LTS updates and maybe I could find out how that works and blog about it if you like?

Our drupal_http_request is based on a backport of https://www.drupal.org/project/chr and a core patch, by the way. Using a proxy for HTTPS in D6 requires moving off the standard DHR due to the lack of support for CONNECT in the stock implementation. I don't really think I need to document the state of the art in D6 technology though :)

Thanks for the quick commit! Will have a look into the caching thing.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

dsnopek’s picture

Sorry for the super late reply!

Our organisation (Catalyst) has though set up a caching mirror of Drupal updates in Nginx as a separate thing to this proxy, due to NZ being geographically far from d.o which makes polling all the modules quite slow. Doing so requires ignoring upstream cache-control headers that d.o uses so they can collect site stats (we don't expose our site keys upstream). I didn't build that system, but we could look at extending it to D6LTS updates and maybe I could find out how that works and blog about it if you like?

That'd be awesome! There's loads of internation Drupal users, so I'm sure others could benefit from setting a caching mirror of updates. :-)