Feeds 6.x-1.0-beta9 causing memory exhaustion errors [#936422]

I am importing a very small feeds item (xml file with 3 items in it), and when Boost is enabled I get some below errors.

An error has occurred.

Please continue to the error page (error page is hyperlinked to the finished page where all nodes are properly created)

An error occurred. /batch?id=27&op=do { "status": true, "percentage": 100, "message": "\x3cbr/\x3e\x26nbsp" }<br /> <b>Fatal error</b>: Allowed memory size of 167772160 bytes exhausted (tried to allocate 77 bytes) in <b>...sites/all/modules/views/includes/handlers.inc</b> on line <b>1209</b><br />

An error occurred.
/batch?id=29&op=do { "status": true, "percentage": 100, "message": "\x3cbr/\x3e\x26nbsp" } Fatal error: Allowed memory size of 167772160 bytes exhausted (tried to allocate 77 bytes) in ...includes/module.inc on line 87 

It seems like it is always a random error line and never the same. I am using Boost and I have about 2000 pages/files in cache. Once I disable Boost and run the feeds, no problems. After I reenable feeds and the cache is cleared I can run for awhile with no errors. The other odd thing is that even with the errors, the feeds import properly with no issues I can see.

I currently have 160 MB of PHP Memory allocated, is this not enough or is it another issue? Thanks!

Comment	File	Size	Author
#4	boost-936422.patch	559 bytes	mikeytown2
#4

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

mikeytown2 CreditAttribution: mikeytown2 commented 9 October 2010 at 09:03

Boost runs all views, checking if the newly created node code live in it. This is done as a shutdown function; thus the import works. I think I fixed the memory leak in the _boost_view_insert() function in the latest dev; could you give that a try and see if you still run out of memory?

Comment #2

tyler-durden CreditAttribution: tyler-durden commented 9 October 2010 at 13:45

I'd like to test the latest Dev, but this only happens on my live site when testing and not any test sites. Both run Boost, but the live site has up to 5000 pages indexed, and the errors seem to happen when these are all cache. Wiping out that cache clears the error.

Is the latest Dev stable enough to use on a live site?

Comment #3

mikeytown2 CreditAttribution: mikeytown2 commented 10 October 2010 at 00:37

I would say yes; also looks like I might want to use $view->destroy();

Comment #4

mikeytown2 CreditAttribution: mikeytown2 commented 10 October 2010 at 05:59

Version:	6.x-1.18	» 6.x-1.x-dev
Status:	Active	» Fixed

File	Size
boost-936422.patch	559 bytes

Get the next dev, should be out in under 6 hours.

this patch committed.

Comment #5

tyler-durden CreditAttribution: tyler-durden commented 10 October 2010 at 13:54

Great, thanks fro the help. I will get this installed and tested tomorrow (Monday) and keep you updated.

Comment #6

24 October 2010 at 14:00

Status:

Fixed

» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Comment #7

tyler-durden CreditAttribution: tyler-durden commented 27 October 2010 at 16:02

Status:

Closed (fixed)

» Active

Sorry, I got side tracked with other issues and I finally got around to updating to 6.19 and the feeds Dev dated 10/20. I am still getting the same errors, even on importing the smallest XML file.

Comment #8

Marko B CreditAttribution: Marko B commented 27 October 2010 at 23:47

having similar problems

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 20 bytes) in /home/maggare/public_html/includes/module.inc on line 462

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 40 bytes) in /home/maggare/public_html/includes/database.mysqli.inc on line 321

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 41 bytes) in /home/maggare/public_html/includes/cache.inc on line 33

this is all on official version, will try dev also.

Comment #9

Marko B CreditAttribution: Marko B commented 28 October 2010 at 00:31

After that i got "The mysqli error was: Too many connections." and drupal is down now.
This all happened on 1.18 version when first time hitting page (crawler not used) and guessing drupal/boost tried to generate node but it failed.

Node has
-node title/body, cck image
-has custom pagers(using views)
-adsense block
-similar entries block http://drupal.org/project/similar

will try to disable this blocks, maybe they cause this errors when generating page.

Comment #10

Marko B CreditAttribution: Marko B commented 28 October 2010 at 10:18

Version:

6.x-1.x-dev

» 6.x-1.18

In the end custom pagers made the mess with boost and made all that errors. Guessing pagers loaded too many nodes, i didnt set limit, was for all the nodes there are, guessing there were alot and boost tried to cache them all?

Comment #11

tyler-durden CreditAttribution: tyler-durden commented 28 October 2010 at 02:21

I think this is a separate issue deepM, your errors don't match mine and I am not using Custom Pagers. I'm glad you figured your issue out though.

More info:
This error does not seem to be causing issues, but I am also only importing 3 nodes from a very small xml file getting this error. On the live system I could be importing 100's, and I have 160 MB PHP mem available. I was also getting "CORE.12345" and other similar files being created when I imported test files with 100+ items. These core files were near 100 MB, and the XML file was around 4 MB. Not sure if this is memory leak issues like mentioned earlier, or some Feeds issue but I don't seem to be getting those CORE files anymore (not even sure what the heck those CORE files are).

Comment #12

tyler-durden CreditAttribution: tyler-durden commented 28 October 2010 at 02:23

Ah wait, reading deepM notes, is there a setting to not add the node to a view/boost at creation? These nodes are in multiple views and at 100's being entered at a time that could be a problem. I don't need them cached at creation, in views or the node itself. Off to research.

Comment #13

tyler-durden CreditAttribution: tyler-durden commented 28 October 2010 at 02:31

"Clear all cached views pages associated with a node on insert"

I had this checked in production but not my test environments, where I was not getting the message (all on the same server). I cleared the boost cache and will try these imports tomorrow as they only happen when boost has been running for awhile.

Comment #14

Marko B CreditAttribution: Marko B commented 28 October 2010 at 10:23

Version:

6.x-1.18

» 6.x-1.x-dev

I would guess the cause is different but the end problem seems the same.

Comment #15

tyler-durden CreditAttribution: tyler-durden commented 28 October 2010 at 18:58

So far so good, I just ran my 3 item xml file with no errors. I will give it another 24 hours and try it again, this is a good enough work around for me.

Comment #16

mikeytown2 CreditAttribution: mikeytown2 commented 29 October 2010 at 20:12

Priority:

Normal

» Major

I need to run this with a php debugger on so I can see whats eating up the ram... out of town so it might be 2 weeks before I get to this.

Comment #17

tyler-durden CreditAttribution: tyler-durden commented 29 November 2010 at 20:01

Status:

Active

» Closed (fixed)

mikeytown2,
I'm going to close this as I am not having any problems and these modules are working properly on my live site. I believe my issue was just dirty xml files I was using for testing, but the xml files are now cleaned up before processing the feeds. I have uploaded up to 1200 nodes at once with n o issues so far.

And thanks for the support.