field_FIELD_data column is filled with huge amount of unnecessary data [#995244]

I checked database and found 200Kb / 200.000 chars stored in the "data" column in each row of my stored youtube media field.

It seems like a serialized XML response from Youtube, there is something wrong in the fetcher and parser I think.

I also tried to clear the data directly in the database, but after update of the node was data columns filled again.

Export of the "data" column (for single file) is attached.

Comment	File	Size	Author
#2	todrupal.txt	708.85 KB	TolstoyDotCom
	youtube_data.txt	199.16 KB	wojtha

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

wojtha CreditAttribution: wojtha commented 11 December 2010 at 02:23

Title:

Huge amount of stored in field_FIELD_data column

» field_FIELD_data column is filled with huge amount of unnecessary data

Comment #2

TolstoyDotCom CreditAttribution: TolstoyDotCom commented 15 January 2011 at 23:35

File	Size
todrupal.txt	708.85 KB

I can confirm this, and I've attached the print_r output for a node I created using one of my own videos. A spot check of some of the items in the 'raw' array shows things that have nothing to do with my video: I've never heard of those users, my video has nothing to do with computers or XMen, etc. etc. I've never seen anything like that coming from Youtube's gdata XML when searching for other videos, but I haven't looked into where all that data is coming from.

Comment #3

Diamonds0a CreditAttribution: Diamonds0a commented 27 January 2011 at 00:27

Issue tags:

+memory allocation

I just encountered a problem that is related. I fixed the issue without getting rid of the data->raw fields but I would like to point out why this is alarming.

In ./modules/node.module line 704 you can see that node_load() will leave nodes in memory that have been loaded.

function node_load($param = array(), $revision = NULL, $reset = NULL) {
  static $nodes = array();

In my use case, using drupal search which resulted in all youtube videos caused memory usage to go over 128 MB.

Comment #4

aaron CreditAttribution: aaron commented 3 February 2011 at 23:15

Version:	6.x-1.2	» 6.x-1.x-dev
Status:	Active	» Fixed

This is now fixed. The update will clear the metadata from your tables, and they will be refilled with the bare minimum for functionality. Developers requiring all the metadata can check that box in API in the module's settings. (For now, that should be done prior to running the update, as the module doesn't currently handle that very well if checking/unchecking the box on the fly.) Also note that in the vast majority of cases, you probably don't need to store the raw data.

Thanks!