Hi,
I have been building a Drupal site for the past 2 weeks, based on the latest 5.1 core. I use about 100 modules to add better functionality (some additional views, and other bits and pieces). The whole installation with all the modules is about 25-30Mb in size. I have about 200 tables in one database. I run 1 site from this install.
For the past week, just while testing my site, my account has been suspended temporarily twice by the hosting company because I have been causing high CPU load on their hosting server. All this while testing and tweaking the install, no other traffic. This doesn't seem normal.
CGI/PHP Overload
--------------
19.04% CPU 6.33% MEM 0.7
Top Process %CPU 138 /usr/bin/php index.php
Top Process %CPU 114 /usr/bin/php index.php
Top Process %CPU 80.0 /usr/bin/php index.php
My questions:
1. Can be a memory leak in one of the extensions to cause the high memory usage? If yes how can I or my host debug it?
2. About 100 modules on a core Drual install is unusual or is still OK? Anyone uses more?
3. Can the above issue be caused by my host's server configuration, or is due because my Drupal has too many extensions?
Any feedback, references would be appreciated, as I might need to look for a solution, or different host.
Thanks, Val
Comments
the way to debug this
You'll have to start from a base install, and add in modules, one by one and check the numbers.
100 modules is nuts
WOW!
I didn't know Drupal even had this much modules...
Are you really sure you need all that stuff??
As for your questions:
1) It's common knowledge some modules are not fully compliant with the programming standards Drupal-developers
are supposed to use. I wouldn't be surprised if you got some of those causing the high CPU load on the server.
2) 100 modules? That's waaaaaaay too much in my book.
Not to mention that a site with this kind of configuration will be a freaking nightmare to maintain. Big time!!
3) Probably a mix of either the amount of modules and the quality of code in some modules. (see 1)
My suggestion would be to start all over and first try to work things out with Drupal Core before dragging in external modules.
Drupal Core is in my opinion very (and I do mean very) powerfull.
But yes, it takes some time to go through all the manuals on this site...
Good Luck with your challenge.
Sincerely,
Bart
Hi Bart, Thanks for the
Hi Bart,
Thanks for the detailed response.
1. Agree.
2. I think the extras depend on what you try to achieve. The core is powerful, but lacks quite some functionality, unless you are a programmer and know your ways around. I am not one.
Here are some of the modules I consider necessary, please let me know if this functionality is found in the core module, and I am missing it. There are about 35 of them here, and include multisite option, organic groups with as much functionality I can get, cck, views, community building tools etc.
adsense\
buddylist\
calendar\
captcha\
cck\
cck_taxonomy\
contemplate\
date\
daterange\
ed_classified\
event\
guestbook\
invite\
localizer\
month\
nodefamily\
nodeprofile\
nodeprofile_bonus\
og\
og_calendar\
og_forum\
og_galleries\
og_roles\
og_teampage\
og_vocab\
privatemsg\
subform_element\
token\
tinymce\
viewfield\
views\
views_bonus\
views_fastsearch\
views_fusion\
All the above will depend on some other bits of pieces, which are not big modules (5-10 kb) but add some functionality to the above and the core. Plus I included some "nice-to-haves" as well. There are about 25 of them alltogether.
formfilter\
formtweaker\
jquery_interface\
jquery_update\
jstools\
pageroute\
panels\
taxonomy_batch_operations\
taxonomy_block\
taxonomy_breadcrumb\
taxonomy_context\
taxonomy_dhtml\
taxonomy_fields\
taxonomy_filter\
taxonomy_forceall\
taxonomy_image\
taxonomy_list\
taxonomy_menu\
taxonomy_multi_edit\
taxonomy_redirect\
taxonomy_super_select\
taxonomy_theme\
taxonomySearch\
upload_image\
upload_preview\
usernode\
This puts me at about 60. Yes there are a few modules installed I won't need, but until the site is done, I have them there. Also I am not aprogrammer, new to Drupal and added things which I considered useful. At the end, I will remove things I did not use.
You also mentioned "a site with this kind of configuration will be a freaking nightmare to maintain. Big time!!"
I might be un-experienced, but don't see why this would be an issue. I used in the past loaded osCommerce installations with 50+ addons. Never had a problem, once made it work, never had to touch it. I don't see why this would be different in this case. Drupal is a lot more stable than osCommerce ever was. I setup the site, all the modules work well together and forget it. I backup the database, the file system and don't see why do I have to be in a big "maintenance mode" from now on.
However you are right, in case I will have to upgrade the site Drupal to 6.0 and up, there will be some incompatibility issues. However over time most of the key modules will be ported to the new versions. Drupal 5 came out in January, and now 5 months later there are tons of modules.
If you have any suggestions how I can achieve more with the core, thus have less addons, please let me know.
Best regards
Val
Try a different approach to your question?
Hello Val,
I just found this thread and am disappointed that you haven't had any subsequent responses as you have two very pertinent questions.
I think you should repost reversing the questions (after seaching on recommended module posts). that is a heading like "The best selection of non-core modules" is likely to provoke a lot more feedback, people tend to shy away from anything 'host' related because it's near impossible to determine where the fault lies.
So you might ask something like:
I am a non-programmer, good aministrator and am interested in which of the following groups of modules will provide best additional functionality to the core on 5. I would appreciate people's recommendations, warnings, known functionality overlaps or conflicts AND especially gas-hog modules to be wary of...
I can't help much because this is only my third week into it myself, but DO avoid category.module - it is not the same as the badly named categories (really taxonomy) module supplied with core, not the same as CCK either and you don't need it.
Regards
Zaph
re: cpu load/memory
Hello Val,
I encountered at least several of the issues that u described which served to run up cpu load and exhaust memory. Consider the following suggestions, maybe they will help:
1) With such a heavily loaded site with all the contrib modules, you may consider a dedicated server as a shared setup from a host company stand point will be limited from making many adjustments, just on behalf of your account. It is much easier, just to suspend your account and let the other customers operate without the drag on resources caused by your site. If u look around, u can get a dedicated server for about the same money as it cost for a VPS server setup, which would be the next step up from shared services. The Planet @ theplanet.com --- usually has some nice, affordable entry level, dedicated server packages.
2) If u get a dedicated server, get the max RAM(memory) that u can. This will go a long way to preempting many of your load/memory issues.
2a) Also, u might want to add a swap file(not swap partition, as it is much more involved). A dedicated setup will generally have (1) swap partition as a adjunct to RAM(memory) spill over. It is when this base swap partition itself begins to max out that cpu load and memory+swap maxes out. At this point, the server will slowly become unresponse and eventually just crash. This is why a host will suspend your account before it gets to this point and thereby effect other sites residing the same share environment.
An additional swap file provides the following, now working together: RAM+swap partition+added swap file.
I just added swap file and it finally stabalized my site.
3) I was guilty at first(2 yrs ago now) of browsing the modules listing and being captured by all of the neat things that could be achieved by each contrib module. So like a fat guy at the buffet, I just piled them on, and on. I do realize that in terms of the modules, u need what u need to achieve particular functionality, however, there is a lot of contrib modules which one can do without -- u will have to decide what is cast overboard, in order to save the ship. Too, too many contrib modules can consume an extreme amount of memory. Keep in mind, that even though your site may not be live with the corresponding traffic, each module, through the database tables, are busy querying and querying. This eats memory and drives cpu load, when u reach a memory bottleneck.
Adding the swap file is fairly simple, but detailed, and u can not really make any mistakes with it as it is done at the linux shell(ssh) command line. However, anyone versed in server administration could probably do it for u in an hour or so.
Hope it helps u a bit, hope it helps someone. The forum and all of its participants have really saved my bacon many times and I am grateful for the resource.
Publishing
Thank you
Thank you for the great response.
--------------------------------------------------------------------
NEWCOMERS GUIDE TO AMERICA
http://www.newcomersguideusa.com
--------------------------------------------------------------------