SimplePie is (temporarly) blocked from FeedBurner

This will get resolved for now though, if your having trouble this patch worked for me.

patch72.patch1.14 KBScott Reynolds
Members fund testing for the Drupal project. Drupal Association Learn more


m3avrck’s picture

Great sluething Scott!

One option is to remove database caching and use SimplePie caching but way back when that never really worked well. I wonder it is working well enough to finally remove database caching?

Needs some testing and to be looked into... performance, duplicate feeds, and the like.

gsnedders’s picture

SP's built-in caching does not deal with duplicate feeds/items whatsoever. It just caches the current state of the feed. There is no intention for it to do any more. For anything like SimpleFeed, it really isn't suitable (nor is it meant to be).

As my post on the FeedBurner list says, I expect it's just done through checking for If-Modified-Since/If-None-Match headers, which you really ought to use.

Scott Reynolds’s picture

AHH so a simple

  $feed = new SimplePie();
  // prevent SimplePie from using all of it's data santization since we use Drupal's input formats to handle this
  header("If-Modified-Since:" . $process_feed->checked);
  $success = $feed->init();
m3avrck’s picture

Status: Needs review » Fixed
Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

Code Rader’s picture

Version: 5.x-2.2 » 5.x-3.1
Status: Closed (fixed) » Active

This is still an issue in 5.x-3.1.

I just did a test using curl

$ curl -I
HTTP/1.0 200 OK
Date: Fri, 24 Oct 2008 16:09:04 GMT
Server: Apache
X-FB-Host: app39
Last-Modified: Fri, 24 Oct 2008 16:07:55 GMT
ETag: ijiCil76g2mWD2oRQMhWELWWIKk
Content-Length: 98547
Keep-Alive: timeout=30, max=100
Connection: keep-alive
Content-Type: text/xml;charset=utf-8

$ curl -I -A simplepie
HTTP/1.1 200 OK
Date: Fri, 24 Oct 2008 16:09:27 GMT
Server: Apache/2.0.54 (Debian GNU/Linux) mod_fastcgi/2.4.2 mod_ssl/2.0.54 OpenSSL/0.9.7e
Last-Modified: Wed, 22 Oct 2008 22:23:44 GMT
ETag: "1742ae-182ff-45dc00"
Accept-Ranges: bytes
Content-Length: 99071
Vary: Accept-Encoding
Content-Type: application/xml

If you notice the Last-Modified, it is significantly different. Different enough to effect my feed retrieval. It looks like the only way might be to mask the User-Agent.

Code Rader’s picture

So are we way out of line to do this?

simplefeed.module line 511
  $feed->set_useragent("SimpleFeed/5.x-3.1 (Feed Parser; Allow like Gecko) ");

Also, to get my feeds to load properly now, I had to delete the hash value in the db directly.

Scott Reynolds’s picture

Status: Active » Postponed (maintainer needs more info)

Was simplefeed fetching your feed? Was the feed fetching just behind? If that is the case not sure this change is warranted (and it probably is justification for creating a new issue). I know this has to do with how Google does its caching and it feedburners performance. I believe the thinking would be the following:

Google allows web browsers to pull the most up-to-date version regardless if its cached, and any other user agent needs to pull from cache. The reason is that the other agents usually ping periodically and thus create load on the servers. This is a way to reduce the load.

Code Rader’s picture

It was fetching the feed, but the feed was over 48 hours old. I understand the problem of load, but to have a blog cache that's over 2 days old seems a bit much to me.

Scott Reynolds’s picture

Status: Postponed (maintainer needs more info) » Closed (fixed)

Then I believe this isn't an issue for SimpleFeed. Google is doing its thing, whether we like it or not. Changing the user agent will get you the most up to date, but from all I have read on this issue (and its been a lot...) its a FeedBurner issue. I have seen the SimplePie guys working with them figuring out what is going on. Not something that should be in the D.o issue queue but SimplePie's :

rmccue’s picture

I really, really hate to bump this thread/bug/node/whatever, but why would header() work? header() sends a header to Apache and to the browser, not to SimplePie and through the socket connection to FeedBurner.

You need to look into the SimplePie_File class and send your headers via that. I don't remember how off the top of my head, unfortunately.