I just installed "sitedoc" after trying to keep notes about configuration changes manually for several months (which, of course, were always out of date or incomplete), so some automatism promised to help.

However, after installing the "sitedoc" module, enabling most of the option, and running the report for the first time, I got the following error:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 262144 bytes) in /var/www/drupal/includes/path.inc on line 62

It's not the first time I'm seeing errors like this on the site to-be-documented, so it's possibly not specific to the "sitedoc" module. The memory limit in php.ini is currently set to 64 MB and I'm not sure if it's sane to raise it even further. The reason for this might be the size of the site (>20k nodes), but basically this alone shouldn't be a problem for Drupal, I think.

To get "sitedoc" running, I unchecked all checkboxes in ./admin/settings/sitedoc and re-enabled the options one by one, starting at the top ("Include Basic Drupal information?"); checking one section more at a time and everytime running ./admin/build/sitedoc, I could get an almost complete report (approx. 800 kB). "sitedoc" finally died completely when enabling "Include URL Aliases?"; this resulted in a reproducable completely blank browser window without giving an error message.

I think, this might happen in a similar way on any other mid-size to large Drupal site (has "sitedoc" been run on drupal.org?). If this should not be specific for my site, I'd suggest to enable to tun the different reports completely separate, outputting the results in different sections/tabs of one main report. If there's another fix/workaround to get a full report in one run, this'd be nice, also ;)

Regards, -asb


NancyDru’s picture

I'll do some homework and see if there's anything I can do to reduce my memory usage. But all my sites run in 16MB with no trouble. Granted none has 20K nodes, or the potential URL aliases associated with them. I just can't think of why Sitedoc would take that much memory. The report does build up and if you have a lot of stuff in it, it could get fairly large, but I can't believe even that would go over a couple hundred KB at the most. I can believe that the query cache might go bonkers.

Your message does point at URL aliases, but I read the table directly. I don't invoke the path module at all. URL Aliases are known to be problems - especially memory. And if you use the indexing feature in path, it is really bad. You might want to also post this error message in the path (or pathauto) module issue queue.

You can use the archive feature and run parts of the report at a time. (BTW, I'd love to see the results; they might help me figure something out.)

asbdpl’s picture

I installed "sitedoc" on four other, smaller, and younger Drupal sites, where the module worked perfectly fine as it is. The genereated full reports (including URL aliases) is between 400 kB and 2 MB (uncompressed). So most probably it's a either (a) a specific problem of my old/large site, or (b) a general problem of mid to large sites (small site = 0-10k nodes; mid size site = 10k-100k nodes; large site = >100k nodes).

> [...] or the potential URL aliases associated with them.

All nodes at my site should be aliased (imho nothing but sematic URLs makes sense).

> [...] but I can't believe even that would go over a couple hundred KB at the most. I can believe that the query cache might go bonkers.

I can send you the zipped HTML files, if there's any way to attach them to a private message. The unedited reports can be compressed to 50-250k each (and they definitely show a lot of stuff in my sites that needs cleanup ;)

> You can use the archive feature and run parts of the report at a time. (BTW, I'd love to see the results; they might help me
> figure something out.)

On the 20k-nodes-site (started in 2005), even a report containing only the URL aliases dies (same result as mentioned above, blank/white browser window, no error message). The maintainers are aware of memory problems of their module.

Final notice: It is very well possible that "sitedoc" is completely innocent. My current theory is, that Drupal sites tend to corrupt over the time; this might not matter for small projects that run for a few years and then become abandoned, but most of my projects are quite large and long term (my oldest and largest websites are from 1994, and I'm pretty attentive when it comes to data degradation). I suspect, that Drupal lacks mechanisms to keep the database in a sane state, especially regarding to "dangerous waste" from prior installed modules, remains of unused tables and data structures etc. When I migrated the 20k-node-site to another server a few months ago, I couldn't restore the database without using the "--force" option; during the upgrade fron MySQL 4.x to 5.x, I lost most of my site's non-7bit-content (e.g. Greek, Arabic, and Russian quotes, and typographic symbols), and just a few days ago I discovered, that Drupal seems not to support some Polish characters. In the site in question, it is impossible to use the "nodewords" module, and I'm getting continous errors from the unmaintained "amazontools" module ("Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 2897 bytes) in /var/www/drupal/includes/database.mysql.inc on line 188"); also, Cron runs tend to die due to these errors. I tried to use "sitedoc" to cleanup the site, so possibly the errors appearing when using "sitedoc" simply expose problems caused by *other* modules, Drupal core, or my own misconfigurations.

That said, I'll try to help in debugging "sitedoc", if I can help and if "sitedoc" is actually buggy.

Thanks & greetings from Berlin,

NancyDru’s picture

Wow, I knew the report was big, but I had no idea it would be that big! I'll send a private email so you can have my address.

Indeed, Drupal sites can corrupt over time, especially older sites from before developers started paying attention to how to uninstall modules. Even my sites that aren't very old were a surprise when I started writing this module. That's why the "fix it" stuff is in there, even though a few people have said it shouldn't be. Hopefully things will get better as the developers become more aware of this left-over garbage.

Obviously, I don't think Sitedoc is buggy, but there may very well be a scalability issue that I need to find a solution to. Possibly something like writing the interim report pieces to a temporary file or cache and merging it back later. But that will probably make a big impact on the archive function. I'll have to think about it.

I'd appreciate the help in debugging/scaling Sitedoc.

NancyDru’s picture

Assigned:Unassigned» NancyDru
Status:Active» Postponed

I am still trying to figure out this scalability issue.

NancyDru’s picture

I think I've found one small culprit in this: I use much the same method to save the archive file as the file upload system uses. The upshot of this is that the output is duplicated for a short time during the save. I have on my todo list to change this to just a straight fopen/fwrite/fclose technique. I will also look at the possibility of writing each piece out so I don't even have all the generated HTML in memory at any given time. These two things will help a little.

NancyDru’s picture

The small problem in #5 is now fixed.

NancyDru’s picture

Priority:Critical» Normal
NancyDru’s picture

Agon, have you tried this on a 6.x site?

NancyDru’s picture

Status:Postponed» Closed (won't fix)

I have not been able to design a way to make a significant reduction in the memory needed to do the whole report. All I can suggest is to do it in pieces.

pankaj.btech’s picture

Version:5.x-1.0» 6.x-1.x-dev
Assigned:NancyDru» pankaj.btech
Priority:Normal» Critical
Status:Closed (won't fix)» Needs work

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 497977 bytes) in /var/www/vhosts/charleslatrobecollege.vic.edu.au/httpdocs/dev/includes/database.mysql.inc on line 301

NancyDru’s picture

Since you assigned it to yourself, does that mean you are going to provide a patch? If not, please mark it "won't fix" again. I use that status for issues I will try to fix in the future.

bgara’s picture

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 611153 bytes) in /home2/saiskill/public_html/includes/database.mysqli.inc on line 303

NancyDru’s picture

Assigned:pankaj.btech» Unassigned
Priority:Critical» Normal
Status:Needs work» Closed (won't fix)

The ImageApi module demands a minimum of 96MB and no one screams. Hmm.

The worst culprit I have seen is the URL Alias section, and there is little I can do about thousands and thousands of them - if you have them and want that report, it takes memory. I have never seen a report without that section take more than 4MB, even with lots of users.

Frankly, I wouldn't consider putting up a D6 site with less than 128MB any more. Heavier use of CCK (Fields) and Views and images is going to drive this higher and higher.

ricardosilva’s picture

Title:Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 262144 bytes) in ... includes/path.inc on line 6» Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 280968 bytes) in /home1/fatimao1/public_html/pro
Priority:Normal» Critical
ricardosilva’s picture

When i try to use modules folder i receive this message error. I don't know what to do... I try to change memory limit in php.ini to 96M as this information show - "ImageAPI GD Memory Limit 64M
It is highly recommended that you set you PHP memory_limit to 96M to use ImageAPI GD. A 1600x1200 images consumes ~45M of memory when decompressed and there are instances where ImageAPI GD is operating on two decompressed images at once." and give the same error.
Can somebody help me. Is my first aproach to drupal... Thanks

NancyDru’s picture

Start by turning off the URL Aliases section.

karanaso’s picture

Well it may not be a complete solution, but more of a workaround but it DOES THE JOB.
In many posts I have read tha IMAGEAPI GD required at least 96MB to work properly.

So I have tried to change my PHP.ini FILE and set
memory_limit = 32M ; Maximum amount of memory a script may consume (32MB)
but my provider's custom php.ini (godaddy.com) didnt seem to work

So I edited the index.php of DRUPAL (I know I should not have done that) and added the
ini_set('memory_limit', '128M');

I again SAY .... I know I SHOULD NOT HAVE DONE THAT .... but it worked. Since the index.php is the root file of drupal. my ini_set command will execute and cropping via the imageapi gd2 will work with no problems.

Hope I helped in any way ......

NancyDru’s picture

32 MB for Drupal 6? I won't even consider less than 96 and prefer 128.

Christopher James Francis Rodgers’s picture


See Documentation page...

Where you will also find a detailed step-by-step solution to the memory error problem
for both Drupal 6 and Drupal 7 to easily increase the memory allocation to 128M, at...

Drupal 7 Fatal error Allowed memory size of ...


Drupal 7 memory increase to more than 128M,
that may or may not work for Drupal 6, see...

rahulbarge21’s picture

Hello karanaso,

It works... :) I edited the index.php of DRUPAL and added the ini_set('memory_limit', '128M');

I was also facing Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 280968 bytes) in /home1/fatimao1/public_html/pro when i enable module and as well as disabling the module.

It get solve.

Thanks you,
Rahul Barge

teju’s picture

my problem is solved
I added the
ini_set('memory_limit', '128M');
code into setting.php file

Thanks once again.........:)

jeril842002’s picture

Great help.
Thanks a lot

kami786’s picture

Great For help