Aggregator: Feed URL Length Is A Limitation [#70201]

I cross posted this in Aggregator2 (at http://drupal.org/node/70021) but it's applicable to Aggregator as well ...

Issue: feed URL is capped at 255 characters. I see that this is because the field types VARCHAR (and CHAR?) are limited to that length under MySQL prior to 4.1.3 (?).

I have already run across some feed URLs that don't fit in that field length. For instance, I can't use custom Amazon feeds created here (http://www.onfocus.com/amafeed/) because they run about 350 characters. I'd imagine that with more and more on-the-fly RSS feeds -- such as Google News -- the 255 char limit will become more and more of a problem.

Sorry I don't have a specific suggestion to fix this, but I don't know enough about MySQL field types and indexing, etc, to write anything intelligent!

Comments

Comment #1

beginner CreditAttribution: beginner commented 23 June 2006 at 10:31

Version:	4.7.2	» x.y.z
Category:	feature	» bug

Comment #2

jt6919 CreditAttribution: jt6919 commented 8 August 2006 at 00:42

I have the exact same problem....someone please provide some type of fix to this...

Comment #3

Kjartan CreditAttribution: Kjartan commented 4 October 2006 at 12:59

http://tools.ietf.org/html/rfc2616#section-3.2.1

The HTTP protocol does not place any a priori limit on the length of
a URI. Servers MUST be able to handle the URI of any resource they
serve, and SHOULD be able to handle URIs of unbounded length if they
provide GET-based forms that could generate such URIs. A server
SHOULD return 414 (Request-URI Too Long) status if a URI is longer
than the server can handle (see section 10.4.15).

Note: Servers ought to be cautious about depending on URI lengths
above 255 bytes, because some older client or proxy
implementations might not properly support these lengths.

Looks like we have to move from a VARCHAR field to a TEXT field for the feed urls.

Comment #4

coreb CreditAttribution: coreb commented 15 November 2006 at 17:36

Version:

x.y.z

» 4.7.x-dev

Has this been fixed for 4.7? It appears the fix (as suggested by Kjartan) is a change to the default DB schema, and I don't know how to patch against that.

Comment #5

jt6919 CreditAttribution: jt6919 commented 16 November 2006 at 17:10

no - it hasn't been fixed, you have to figure out how to patch it yourself (the form and the db table)

Comment #6

budda

English

Cheshire

CreditAttribution: budda commented 19 November 2006 at 03:27

I fixed this issue with the feedparser module (still work in progress). The code to update the database to remove the limit is as follows:

      $ret[] = update_sql('ALTER TABLE {aggregator_feed} DROP KEY link');
      $ret[] = update_sql('ALTER TABLE {aggregator_feed} CHANGE url url TEXT NOT NULL');

You'll also need to remove the maxlength value in the aggregator.module form code
Search and destroy the following line:-