By muckermarc on

Hi,

Have the following warning/errors on two Drupal 6.10 sites, both hosted by the same company on the same physical server:

# warning: opendir(sites/all/modules/fckeditor/plugins/linktomenu/images): failed to open dir: Too many open files in /hsphere/local/home/xxx/includes/file.inc on line 888.
# warning: opendir(sites/all/modules/fckeditor/plugins/linktomenu/lang): failed to open dir: Too many open files in /hsphere/local/home/xxx/includes/file.inc on line 888.

Actually there are 98 individual errors, each one pointing to a different directory in a different module, but all generated from line 888 in file.inc.

The line in question is in the function file_scan_directory():

if (is_dir($dir) && $handle = opendir($dir)) {

So I guess Drupal is trying to iterate through the sub directories in a particular directory and is hitting some kind of limit defined at the webserver level?

This ties in with the pages where I see the error message - usually on the Available Updates page (/admin/reports/updates) and the Modules page (/admin/build/modules), both pages would seem to browse directories on the webserver.

I have a call logged with my hosting company (I'll not be holding my breath) but wondering has anyone else come across the problem, or better still a solution?

Cheers,

M

⋅ Categories: Drupal 6.x

Comments

Didier Misson’s picture

Didier Misson commented

I have the same problem, but I don't find the answer :-(
Probably it's simple, but who can help us ?

Thanks a lot
;-)

Didier Misson’s picture

Didier Misson commented

I find the solution.

En français : http://didier.misson.net/blog/2009/07/06/failed-to-open-dir-too-many-ope...

The problem is a maximum open files limite PER users !
We add several web servers, so several new Apache2 VirtualHost... and Apache2 (www-data) reach the 1024 maximum open files...

you can check with

"ulimit -a" or "ulimit -n" (default is maximum 1024 open files)

you can modify with "ulimit -n 1500" ... but it will be lost after reboot...

To permanently made the modification, add in the file /etc/security/limits.conf :

$ vi /etc/security/limits.conf

#<domain> <type> <item> <value>
#
www-data soft nofile 1600
www-data hard nofile 65535

And of course, restart your web server Apache2 or other.
;-)

So, it was not a Drupal problem, nor a PHP problem...
but I think it's important that Drupal users know how to correct this problem.

antrophos’s picture

antrophos commented

I do run Drupal on the Webspace of my provider. Do I have the right to request those changes?

May be a stupid question but I don't understand the change made in detail.

On the other hand you say drupal has low profile needs. May be you set the possibility somehow to reduce this need or serialize some process?

Didier Misson’s picture

Didier Misson commented

Hello,

I suppose you can not request directly these modification !
If you don't manage the server, contact our provider support with the detail error...
Insist on "too many open files" and ask about the "ulimit"

;-)

Didier Misson’s picture

Didier Misson commented

As explain, I have increase the "ulimit", and modify the value in the "/etc/security/limits.conf"

It's better... but NOT enough !
I even increase the limit to 20000 ...

I always have these messages "Too many open files".
It's not directly a Drupal problem, as I have problem too in Squirrelmail, and my friend has problem too in some of its PHP scripts...

I can solve the problem with an Apache2 restart...
but only solve for some hours ! (even not for a days)...

I don't understand !

lsof -u www-data | wc -l
say me that Apache2 has between 11000 and 13000 open files.
I set the ulimit to 20000 ...
I restart Apache2, and after 10 or 14 hours, problem reoccurs ! "Too many open files"
:-(

Thanks,

unleash’s picture

unleash commented

hi there - hello didier - good day

well i have the same issue here:

i am pretty new to drupal and also [pretty] new to this forum - so do not bear with me. Sorry for the newbie-questions... or just visit the site - http://campus-24.com/?q=stream and http://campus-24.com/ to get a first glance of what happened. with the installation of drupal-commons.
Here the full story! after strugglin several weeks with drupal-commons 6.2.3 (based o november 2011) with issues on htaccess (my admin added and choose some certain options) i finally was able to install the system on the webserver.

Note - yesterday i installed on the local lamp with ease - all went nice and no error at all appeared.
Well today - the drupal-commons was installable on the webserver as well - Note: i gote some errors -which were reported during the intallation - (at each station of installation i got some lines like shown below)
But - nonetheless the installation-process was continued and fulfilled - i was able to choose all options and configuration-steps.
Interesting detail: There were onle 149 tables been created - not 154 like yesterday on the local installation. !?! Interesting - isnt it?!

besides this - the result looks abit funny - what do you think - what can i do with that - can i fix it!?

have a closer look: http://campus-24.com/install.php?locale=&profile=drupal_commons&id=2&op=...
http://campus-24.com/?q=stream
http://campus-24.com/

but it should look like that of course like so:

http://commons.acquia.com/stream
or so http://commons.acquia.com

Question: is this a minor bug - or what can we do here -

btw - see the error reports that shown up during the installation - process. (well nonetheless all went without any break or interruption )

love to hear from you

greetings
unleash

error reports:

    warning: opendir(profiles/drupal_commons/modules/acquia/acquia_search/acquia_search/tests) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/Zend) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/tests) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/contrib) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/SolrPhpClient) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/acquia/acquia_search/apachesolr/drush) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_timezone/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_popup/themes) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_popup/lib) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_popup/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_locale/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_tools/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_php4/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.
    warning: opendir(profiles/drupal_commons/modules/contrib/date/date_repeat/translations) [function.opendir]: failed to open dir: Too many open files in /home/vhost/WWW/campus-24.com/includes/file.inc on line 955.

plz help me - just advice me what to do!

gagarine’s picture

gagarine commented

You never learn than cross-posting is a bad things?
http://drupal.org/node/474152#comment-5440114
http://drupal.org/node/1231106#comment-5440126
http://drupal.org/node/1396990
http://drupal.org/node/825990#comment-5440056

https://interface-network.com - Interface Network is an action and research technology governance agency.

gagarine’s picture

gagarine commented

You never learn than cross-posting is a bad things?
http://drupal.org/node/474152#comment-5440114
http://drupal.org/node/1231106#comment-5440126
http://drupal.org/node/1396990
http://drupal.org/node/825990#comment-5440056

https://interface-network.com - Interface Network is an action and research technology governance agency.

muckermarc’s picture

muckermarc commented

Had to scratch my head to remember how I got this issue resolved.

My sites were hosted on a shared hosting environment, talked to my hosting provider, who worked their magic and the problem went away. My impression was that they had a badly configured webserver, and that other sites on the server (not my Drupal ones) were causing the problems. As it happens, I've moved all my hosting to another provider, and have never seen this problem again (famous last words!), over 2 years and 20 Drupal installs later.

Hope this helps,

M

unleash’s picture

unleash commented

hello dear muckermac

thankx for the reply - sure it has to do with a performance issue - and weak servers do not cope with the load that comes with drupal.

i have a older one - with less ram and so on.

but - my admin had increased the file-handles of the apache. So i will retry it again.

Besides that: i know that i have to replace the old root-server soon.

on a sidenote: i found out that Drupal has a bunch of translations that may cause a serious load...

i did the search with

find -type f -name "*.po"

. well i found approx 930 files.

afterwards i did remove all them with

6.x-2.3 # find -type f -name "*.po" -exec rm -f {} \;

a final serach with that code

find -type f -name "*.po"

gave no results back so every po-file was erased!

Well i will try to install this lean - and leightweight package... I come back and report all the findings

greetings
unleash

jeni_dc’s picture

jeni_dc commented

I was having this same issue with the built in Apache on OS X, but setting ulimit didn't seem to help anything. What did help, though, was deleting some unneeded virtual hosts from my httpd-vhosts.conf.

So if it's a local issue and you're using OS X, give that a try.

joshf’s picture

joshf commented

Thanks! I never would have thought to try this.