No longer connects to the update service

Thanks for the report!

Can you give days/times when it wasn't working? It could be a server load thing on our end, and if I know the days/times it wasn't working, I can check the logs..

Hrm. I checked a bunch of our clients, and it's working for them -- so maybe it isn't a server thing..

Hm, checking the firewall was a great idea (our firewall is pretty aggressive about blocking IPs that it thinks are trying to hack us) but there isn't a block on that IP. In fact, I can't find that IP in the access logs at all! Our oldest access log goes back to February 16th, so if it ever connected, I should see your IP in there.

Are you sure that that's the IP of your server? And not the IP of your local machine or something? It's the server that's calling out to our update service.

Hm. That IP isn't blocked or in the logs either! Is it possible the server has multiple interfaces/IPs and is using a different IP for outgoing communication?

Or, what about a firewall on your server? Is it possible that it's blocking outgoing requests on port 80/443 except to a list of specific IPs?

Aha! 37.187.137.19 is the one. :-)

I'm able to find successful connections from that IP back to March 29th.

Our firewall is blocking that IP with the log message "Brute force Web Server 26 attacks - Sun Apr 3 12:13:50 2016". I'm hoping there wasn't an actual brute force attack, just too many connections too quickly - our firewall is pretty paranoid. ;-)

I'm removed the block on our firewall. Please let me know if it's working for you now!

Awesome, I'm glad it's working! No worries - I'm happy that you reported the problem, rather than just stopping using the module. :-)

However, the module should not send the request at every cron call, there might need to do something about this.

Could you open a new issue for that? While it is nice to check more often when possible, there should certainly be a limit to that (maybe checking at most every hour, even if cron runs more often?).

Hrm. None of your IPs are blocked on our end. I can even see '37.187.137.19' successfully connecting an hour and 12 minutes ago.. I'm not sure why it isn't working on your end!

Looks like the server not always responds.

I don't see any (important) failures from your IP on our end. All the requests to /update/statistics/v1 were either 301 (redirect to SSL) or 200. There are some request for modules that aren't on Drupal.org, like "DefaultTextForNode", but that shouldn't cause the whole page to not work...

Is there a more detailed error in the log on your end so I have something to investigate?

We've recently started running into the same issue for a few of our D6 sites running this module as well. This started happening on May 31. The one I'm looking into today is up at 198.16.5.106. It runs cron every 30 minutes.

Is there anything else I can send you to help troubleshoot?

Thanks!

@hey_germano: Unfortunately, I don't see anything in the firewall for 198.16.5.106, so it's not that. :-/ I do see POST requests coming in in the access log (including today), and getting a 301 redirects but not POST'ing again. Can you update to the 1.5 version of the module? This recent change should mean it doesn't redirect anymore #2733783: Missing content-type header on stats POST (and redirect issue)

@dsnopek I updated the module on the site I mentioned above (and we tried on another one as well) and we're still seeing the "Unable to fetch any information about available new releases and updates." message. Not sure if this means anything, but I see the same thing on my local copy of the site.

Is there anything else we can check on our end?

@hey_germano: Hm. I don't see any requests from your IP for today at all. :-/

My next guess is that it's maybe something with SSL?

Drupal 6 unfortunately uses stream_socket_client() to make the connection (rather than cURL) so this isn't a perfect test, but what happens when you run this command from your server:

curl https://updates.mydropwizard.com/update/statistics/v1

You should get the string "ERROR" (which is because it was a GET request, rather than POST) but that at least means that it's able to connect. And I should be able to see that in the logs.

If that doesn't get us anywhere, I'll work on a patch for mydropwizard to try and generate more debug info.

@dsnopek Here's the output of that curl command:

curl: (35) Unknown SSL protocol error in connection to updates.mydropwizard.com:443

Thanks for your help!

Not sure if it matters, but I noticed you asked this on the related issue - we're on PHP 5.3.3 here.

@hey_germano: Ok! That narrows it down. It must be that our SSL only supports certain protocols or ciphers, that your server doesn't support, so it can't connect. I'm not 100% sure how to figure out exactly what those are or how to fix it, but it gives a place to start researching!

I have a possible work-around to test! I just setup this domain:

http://updates-insecure.mydropwizard.com/

Try setting these two variables via drush:

drush vset -y mydropwizard_project_url 'http://updates-insecure.mydropwizard.com/update/statistics/v1'
drush vset -y mydropwizard_statistics_url 'http://updates-insecure.mydropwizard.com/files/update/v1'

This will cause the module to not use HTTPS, which could fix the issue! It does open you up to the possibility of man-in-the-middle attacks, however, those are a relatively difficult class of attack to pull off.

If this works, I may add module configuration setting to disable HTTPS without having to mess with variables.

Thanks!

@dsnopek: I had a really good feeling about this one, but no luck.

I tried a curl http://updates-insecure.mydropwizard.com/update/statistics/v1 on the server and just got the "ERROR" return message, though. Maybe now it's not the SSL issue, but the firewall? The IP for the site I'm testing with is 198.16.5.106.

@hey_germano: "ERROR" is what you should get! curl's not doing a POST request and sending all the information that end-point needs, so we're expecting it to be an error. Did you try setting the variables in Drush like in #26? That'd be the real test!

@dsnopek - Ah, sorry that wasn't clear - yes, I did set the variables via Drush first and tried checking for updates again. Still seeing the "Unable to fetch any information about available new releases and updates." message.

I tried changing those variables via Drush on a different D6 site with the same issue, which is on a different server, and I wasn't able to retrieve update info there either. cURL response was the same.

The IP for this other site is 198.16.5.121 and is also on PHP 5.3.3.

#26 has the project getting a statistics URL and the statistics getting a files URL. Shouldn't it be this way around?

drush vset -y mydropwizard_project_url 'http://updates-insecure.mydropwizard.com/update/files/v1'
drush vset -y mydropwizard_statistics_url 'http://updates-insecure.mydropwizard.com/statistics/update/v1'

Even that seems like it might not be right because http://updates-insecure.mydropwizard.com/update/files/v1 "could not be found" when I try to load it in a browser, and the same with /files/update/v1.

But now I think the firewall has locked out our server IP, 178.32.120.70 so I can't test anymore right now. :-)

Also having that problem. Was on 1.4, now 1.5. FreeBSD 10.1, PHP 5.6.22, curl previously reported "ERROR" so apparently not an SSL issue, at least with respect to curl's support.

Oops! @GreenReaper, you're right! I messed those URLs all up. Sorry, everyone! :-(

This is what it should be:

drush vset -y mydropwizard_statistics_url 'http://updates-insecure.mydropwizard.com/update/statistics/v1'
drush vset -y mydropwizard_project_url 'http://updates-insecure.mydropwizard.com/files/update/v1'

... and I tested these values on a dev site, so they should at least be correct this time.

If this actually fixes the problem, I'll build a feature into the module to switch to these URLs, so we won't have to mess with variables via Drush. I just need confirmation that this fixes the problem!

I can confirm that I was able to retrieve updates having set the above URLs.

It is still a little concerning that it worked fine before a few days ago and now does not. I checked the SSL options for updates.mydropwizard.com but they seem reasonable.

That fixed the issue here. Thanks!

Awesome, thanks, Everyone!

I've opened a new issue to add an option to help others more easily switch the URLs:

#2743495: Provide option to disable pulling update information via HTTPS

Sadly I am no longer allowed to re-open issues.. ughh!!

I just tried out this awesome module on my local D6 site. It worked great. Used it with Drush as well, also worked great. That was earlier today. Now it no longer works stating:

Unable to fetch any information about available new releases and updates.

Is there possibly a limit to how many times it can run? I have likely run it 3 times today.

We have no limits on our end, but we do have an adaptive firewall - you could maybe have gotten your IP blocked. What's the outgoing IP of the server?

My update.php worked earlier today, but I hit it several times due to some wrong website php settings. Any chance I got blocked by your firewall? server IP: 96.70.135.107

@joeebel: I checked and your IP isn't being blocked by the firewall.

The most common other problem is issues with SSL in PHP. You can try to work around that by running the Drush commands in the issue summary here:

https://www.drupal.org/project/mydropwizard/issues/2743495