The install of drupal crashes if you don't have more than 8M PHP memory_limit. Typically you end up with a WSOD after you've entered your database details. It's caused by the infamous Fatal error: Allowed memory size of 8388608 bytes exhausted. Quite unfriendly for the newbies who won't know what to do or where to look. Many PHP systems are still configured by default with a memory limit of 8M. This is also the case of MAMP.
To avoid this, we could check the size of the memory limit with ini_get('memory_limit') and if it's too low trigger a warning message stating that the installation requires more memory. (12M works in my case)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Gábor Hojtsy’s picture

Status: Active » Closed (duplicate)

Hm, I think that the batch patch at http://drupal.org/node/176003 will easily solve this problem. Can you test that and reopen this issue if it does not solve your problem?

scor’s picture

Status: Closed (duplicate) » Active

I applied the patch and I can briefly see the modules being installed, but then I still get the same white screen.

catch’s picture

Title: Warn users during install when their PHP memory limit is set to 8M » Install fails for php memory limit under 8M

Changing title. If > 8M is necessary, then that's a requirements change that cuts out a lot of shared hosting.

Gábor Hojtsy’s picture

I also hit this requirements problem before and discussed with when testing some patches with Dries. He was surprised that we require more then 8MB. What might be the reason? The set of core stuff to include did not grow that big. In Drupal 5, importing a whole translation (400kb text file all loaded and parsed into a gigantic PHP array) was what went over the 8MB limit.

Gábor Hojtsy’s picture

http://drupal.org/requirements says:

PHP memory of approximately 8MB for a Drupal core installation. In reality you will probably need to use a higher setting depending on your site and contributed modules you are using. A good starting point is 16-24 MB.

Indeed, we say Drupal runs on 8MB. So this would be a requirements change. So can Drupal core run on 8MB? With how many/what modules installed. Is this memory problem temporary (considering core only) in the installer?

scor’s picture

With a limit of 8M, the modules page is also likely to crash, especially with all the core modules enabled.

memory usage of HEAD on average on my machine (MAMP PHP/5.2.3 mod_php, Apache/2.0.59):
5.4M home after installation
7.9M modules page (only default core modules enabled)
--
11.7M modules page with all core modules enabled
7.7M home with all core modules enabled
--
7.3M modules page with no module enabled
5.3M home with no module

on drupal5 the modules page with all the core modules enabled takes 9M.

Gábor Hojtsy’s picture

I wonder what is included on the modules list page which makes it require substantially more memory. With .info files there should not be much of a difference on the modules page.

bdragon’s picture

Assigned: Unassigned » bdragon

I will profile.

bdragon’s picture

Assigned: bdragon » Unassigned

xdebug profile dumped, available at http://drop.rtk0.net/cachegrind.out.4865.bz2

Haven't analysed it yet...

bdragon’s picture

Hmm, xdebug didn't use the right format.. Debugging my debugger... :-/

bdragon’s picture

Much better...
http://drop.rtk0.net/trace.4165204225.xt.bz2

PHP5.2.4 fastcgi + eaccelerator 0.9.5 + xdebug 2.0.2.

agentrickard’s picture

For what it's worth, my basic install (no translations, no install profiles) worked fine with 8MB of memory. Running PHP 5.1.6 on OS X (Apache 1.3)

bdragon’s picture

I think PHP5 is supposed to be more memory efficient than PHP4 in some places... Perhaps the people with OOM problems during install are running PHP4?

agentrickard’s picture

Worth looking into (hence my note)

scor’s picture

@bdragon: for my test I was using MAMP PHP/5.2.3 mod_php, Apache/2.0.59

@agentrickard: are you using APC, Xcache or eAccelerator? they avoid the memory problem, but that's not a fix.

catch’s picture

Any chance this is the install files as mentioned on devel list?

Scor, does this patch make any difference? system.install by itself is 108kb at the moment, minus 4.7 updates it's 93kb. Most others are 3-4kb each.

scor’s picture

the system.install patch doesn't fix it. I still get a WSOD. the tables are created, but the install doesn't finish. in case it can be of any help, in users table, uid 1, name and mail are both 'placeholder-for-uid-1', and pass is empty.
Even if the system.install patch would work, it wouldn't be a fix since the modules page with all modules enabled uses more than 8M memory.

agentrickard’s picture

@scor I am not using any accelerators or opcode caches. Just the Mac PHP distro by Marc Linyage.

Jody Lynn’s picture

Confirming bug - I have always had to increase php_value memory_limit in htaccess in Drupal 5 when running MAMP, and Drupal 6 now WSOD's right away without it. I've wondered why a php_value memory_limit line does not exist out of the box.

agentrickard’s picture

Perhaps this is a MAMP bug. Do we have any proof that this occurs on other LAMP stacks?

nedjo’s picture

We've been discussing the footprint of Drupal core, but we have to consider the fact that many or most installs are or will include contrib modules as well. While this issue may occur on initial install, it is just as likely to occur later, when an admin enables a module that pushes the Drupal install over the available limit. Yes, this is more problematic in install than later, but the issue is essentially the same.

It seems that we need a general method for evaluating the anticipated memory footprint of modules to be enabled. Consider the use case of an install profile that includes views, cck, and several other large contrib modules. The full array of modules to be enabled is available before they are enabled (and, hence, before they are loaded, potentially triggering the out of memory error).

Can we use some proxy measure to anticipate the memory footprint of the code to be enabled? E.g.

* For every required module enabled, we measure the length of the file (number of lines? number of characters? number of non-comment lines?) and test memory usage before and after loading the file, producing a statistic for the local environment.

* Before loading new modules, we use this statistic to evaluate the anticipated additional memory usage of that module (measuring all the files in the module's directory? of course this is inaccurate, since in many cases not all will be loaded at once, but determining which will be loaded when is probably not feasible) and compare that to the available memory limit.

* If the anticipated memory hit approaches the available limit, we fail with an error before enabling the modules.

catch’s picture

Title: Install fails for php memory limit under 8M » Install fails for php memory limit under 8M on MAMP

It does look like MAMP, and in that case people will have access to change the memory limit right? Unless we get some reports from other systems of it failing, then this isn't critical IMO.

Gábor Hojtsy’s picture

Lynn: we can try to do a php_value memory_limit 32M or something along these lines, but there is no guarantee it would work, so we would still need to check for it.

catch’s picture

nedjo - issue #198053 looks like a good start on what you're suggesting, obviously doesn't help at install time.

scor’s picture

Title: Install fails for php memory limit under 8M on MAMP » Install fails for PHP memory_limit under 8M

@agentrickard, if I set memory_limit to 8M in the Mac PHP distro by Marc Liyanage, I get the same WSOD.

the problem appear in MAMP because it has memory_limit to 8M by default.
I tried on 2 other Linux debian servers set to 8M memory_limit, and I get the same problem. This is not a MAMP specific issue.
Also, whatever the server is, the memory used by drupal after the installation is the about same: 8M on the modules page after installation, and 12M when all the modules are enable. So even if you could 'magically' install drupal, you could no display the modules page - that means that the 'Installation failed w/ max execution time of 30 s' issue will not fix this, as the memory problem is not limited to the install process.

@gabor,

we can try to do a php_value memory_limit 32M or something along these lines, but there is no guarantee it would work,

on some shared hosting accounts, you don't have the rights to change that setting in .htaccess, and if you try to do it, you get a Internal Server Error.

@catch, yes that's great, but we need to check that before starting the install, otherwise the user will have no chance to get to the modules page.

scor’s picture

Title: Install fails for PHP memory_limit under 8M » Drupal requires more than 8M PHP memory_limit

let's reflect the last point in the title

catch’s picture

I'm starting to think we should change http://drupal.org/requirements here. Minimum 12MB, recommended 16-32? Does anyone actually run live sites with 8MB memory limit at all?

scor’s picture

for the core alone, 12M is risky (all modules activated can reach that limit). 16M seems a safer minimum requirement.

catch’s picture

Well here's a suggested rewrite for that section of /requirements. We would also need this to be added to the installer requirements check of course.

    * PHP memory of approximately 16MB is required for a Drupal core installation. You may need to increase this amount if using contributed modules.
keith.smith’s picture

As an aside, I installed xampp-win32-1.6.4 on a Windows Vista (I know) system yesterday, and have had no issues with memory. (Clean URLs, yes, but that's another story.)

I notice that PHP.INI has, out of the box with no modifications:

max_execution_time = 60     ; Maximum execution time of each script, in seconds
max_input_time = 60	; Maximum amount of time each script may spend parsing request data
memory_limit = 32M      ; Maximum amount of memory a script may consume (16MB)
catch’s picture

Thinking about it, there's no good reason to put this as a hard requirement for install - we have no idea if hosts are using opcode caches etc. so enforcing a high limit when it may have a good chance of working with a lower one seems like overkill.

Some kind of warning if the memory limit is low < 16MB? but not enforced might work I guess. "Your php memory limit is %memory_limit, consider increasing this to 16MB or above to ensure a smooth installation, see http://drupal.org/requirements for further details".

deavidsedice’s picture

Drupal eats about 8 Mb of PhP memory because PHP stores in memory each file included. When more files are included, more memory you'll need. No matter if that file uses more or less globals, or if it is executed or not. When it is loaded, that data goes to the php memory.

To solve the problem, the best sollution is installing an PHP Accelerator (any php accelerator should work). This reduces the memory used, because all accelerators are caching those files in the same place in the memory, out of PhP.

You can also write an entry in .htaccess to tell Apache & PHP to change the memory_limit setting. (Works in some hostings)

catch’s picture

Component: install system » documentation

Moving this to documentation queue, there's no code in core that deals with memory settings, apart from color.module which does it programmatically based on memory_get_usage - so we should just increase the requirement formally in http://drupal.org/requirements to 16MB or above.

scor’s picture

Component: documentation » install system

@keith.smith: it's normal that you don't have the problem with 32M.

@catch, I don't know if many hosting companies have opcode caching, especially the cheap ones. If we don't know, we have to consider the worst case. It may be too overkill to require 16M, but I think we should at least display a warning during the install if the memory_limit is less than 16M, reminding the user to check the requirement page. If we don't, most people will not read http://drupal.org/requirements and will fail to install drupal.

catch’s picture

scor: I looked through the installer code - and we don't have code to check php version, mysql version and the rest - I'm not sure memory limit is any different. We could put this in install.txt (in which case it's still a documentation issue) - although that itself goes to drupal.org/requirements for just about everything so it might be redundant.

I also couldn't see an obvious place to put an ini_get and a warning in the installer - it would make sense for it to go on the same page as the files/settings.php check - but those are tasks and won't allow you to continue unless they're met (and that page won't even display if they are - so the warning would be missed and it'd then either proceed as normal or WSOD anyway). This was my first or second look at the installer though so I may be missing something.

The closest thing to a hard requirements check along these lines I can think of was lock tables , but it was impossible to run Drupal without it - we don't have a hard limit here (i.e. someone on the 30s max execution time issue has just reported successful install with 8MB memory_limit, lighty etc.).

nedjo’s picture

FileSize
2.03 KB

Here is an untested patch that draws on PhalanX's approach in http://drupal.org/node/198053.

We introduce a memory setting in .info files, representing the amount of memory required by a module (or theme?).

(We could consider adapting our drupal.org packaging scripts to assign modules an inferred memory value if none is set explicitly in their .info file.)

At install time, we read the memory requirements and the available memory and bail with an error message if the modules to be installed would exceed available memory.

We could as in other cases treat system as a special case, including in it a baseline for Drupal's required non-module files.

nedjo’s picture

FileSize
2.02 KB

Fix a couple of typos in the previous patch. Still untested.

scor’s picture

Status: Active » Needs work
FileSize
1.91 KB
1.91 KB

the php, apache and mysql versions are all tested in system.install (among other tests).

deavidsedice reported a successful install on the second trial after failing on the first attempt. he also uses an accelerator.

Here is a first patch which adds the memory_limit test among the other tests. needs more work if we want to make the test non required and display a warning message.

JirkaRybka’s picture

I would rather not make this a REQUIREMENT_ERROR, it might be or not be a show-stopper, so a warning might be more appropriate (for runtime display on the Status report page, I mean). For the installer check, we may well have a special case using if ($phase == 'install'), and in there drupal_set_message() the warning - I believe it'll then show on any installer page following, whichever it is. Note that we can't set the message as error, because that's treated as a general show-stopper inside install.php code.

Sorry for having no time to implement this myself today. All the ideas above are untested.

catch’s picture

Quick +1 for drupal_set_message(), seems like the best option here.

shunting’s picture

Wesley Tanaka has several lower memory module versions on his site here:

http://wtanaka.com/drupal

I don't have the skills to check it out, but others here might find it useful (or already trodden ground, I don't know.)

Also, and definitely FWIW because I don't have the knowledge to help, treating more than 8M is a documentation problem seems a little cavalier, if that suddenly shuts out a large class of users.

Above, Gabor writes:

I also hit this requirements problem before and discussed with when testing some patches with Dries. He was surprised that we require more then 8MB.

Is it possible Dries is surprised because there is a class of users he still expects to be able to reach? With at least core in 8M?

Gábor Hojtsy’s picture

shunting: by reading the docs at http://wtanaka.com/drupal, Wesley simply did what we did in Drupal 6. He moved out parts of Drupal not always required to include files. Drupal 6 does the same. So this will not help us going further.

We covered why Drupal eats lots of memory:

- we have more modules then in Drupal 5, so even through they have less code always loaded, they take up more memory
- the system page loads all .install files to check whether updates are available to run, so it takes more memory there...

As I said, we did all Wesley suggested to move out often not used parts. Unless anyone else comes around with memory improvements, all we can do is document what we have.

shunting’s picture

Gabor:

Understood. I'd still like to understand the "surprise" Dries has above on 8M, though. Seems like a big decision, here.

(Maybe the install files could be loaded periodically, via a cron setting, instead of every time? Might be a worthy tradeoff for the smaller sites on smaller ISPs. Sorry, meddling in affairs of wizards ;-)

catch’s picture

A drupal_set_message as suggested in #39 seems sufficient for the installer.

Would suggest this text:

"Your php memory limit is set to %memory_limit. Consider increasing this to 16MB or above to ensure a successful installation process, see http://drupal.org/requirements for further details".

Khalid's memory usage analysis of 5.x and 6.x suggests about 11.5MB with a basic drupal install for either version once installed. In docs, we should probably change the memory section to something like

Drupal core can run with 8-12MB php memory depending on your configuration. 16-24MB is recommended, especially if using additional contributed modules.

edit: nedjo's patch at #28 looks like a start. But hard-coding memory usage estimates into modules I don't think is a good idea.

If the problem is with loading of .install files: as mentioned, we can shave 15kb off system.install by removing 4.7 updates, although that's not something that should be done just for this reason. All .install files together are somewhere over 200kb so that's be something like 6-7%. I have no idea if it'd make much difference.

In D7 maybe something can be done with splitting updates into .update files or whatever, but I agree we should focus on appropriate documentation and prompting at this point. Maybe .schema files needs to be revisited in D7 as well.

The vast majority of people who use drupal, wether on shared hosting or anywhere else, have to use memory limits of over 8MB - since installing any number of contrib modules enforces this quite quickly. I ran into this once before on a cheap shared hosting account, and the company raised it on request, as will most.

Gábor Hojtsy’s picture

shunting: Dries' surprise was about the *installer* requiring more then 8MB out of the box. A simple blog with blog module, and most Drupal modules disabled should run under 8MB for example. If anyone has bright ideas on enabling the installer to run under 8MB, then please move forward. An obvious way would be to split up the install.php file and move different screens and functions to different include files. Although that sounds a bit scary. IMHO we have a clearer focus in this patch, even if the patch does not solve the installer memory limit, it still solves a few other bugs as noted, so it is worthy to pursue by all means.

Feel free to open a new issue to make the installer 8MB friendly. By the way as far as the future goes, Drupal 7 is planned to require PHP 5.2, which defaults the memory limit to 16MB.

velankar’s picture

A suggestion.

I am not sure whether whatever I am going to suggest is doable in less time or not.

If the install is overflowing the memory limit, then divide the install step into several steps. step1, step2 etc...

At start load one javascript into the browser and let it fire the install url with first step id. (stop 1)

When that step is completed, the page will again load the same javascript and it will fire the next step. ........

Keep on doing this untill all the steps are over.

This will take more time but the memory problem will not come.

I repeat. I do not know how pratical this idea is.

I do something very similar when I want to send a lot of emails; but the server will not allow me in a single stretch.

Gábor Hojtsy’s picture

velenkar: As discussed already, the patch at http://drupal.org/node/176003 does exactly this already (although it degrades nicely to situations when there is no JavaScript), but that is not a solution to the problem, as more and more modules are installed, the memory requirement grows, and reaches / gets close to 8MB once all modules are installed.

scor’s picture

Status: Needs work » Needs review
FileSize
3.9 KB

This patch implements a PHP memory_limit test in system.install along with the other tests (Web server, PHP version etc...), allows the display of warning messages in theme.maintenance and displays a warning message during the install if the PHP memory limit is lower than 16M.

A simple blog with blog module, and most Drupal modules disabled should run under 8MB for example.

Gábor, I measured that the modules page with no module enabled already uses 7.3 MB (see http://drupal.org/node/197720#comment-648567). If you enable the blog.module and comment.module, you'll be very close to 8M. (and that doesn't include menu, path, upload which will be usually used on blogs).

vjordan’s picture

Stephane - works as described for me. The documentation update in #44 would need to be done of course (does this require a further issue to be logged?).

Should the same warning be made available during an upgrade process where the memory limit is below 16M?

scor’s picture

FileSize
4.14 KB

rerolled now that the batch install patch is committed. (note: the batch install doesn't solve the memory issue discussed here).

I don't understand why in theme.maintenance.inc the status messages displayed in green are considered as warnings and have the following title:
'The following installation warnings should be carefully reviewed, but in most cases may be safely ignored'
this doesn't make sense if their are status messages.

vjordan, good point, a similar warning could probably be displayed during the upgrade. We could come up with a more general message that we could reuse in every case, but some people might think they can install drupal and worry about the memory afterwards - which is not true.

moshe weitzman’s picture

I didn't read every post here, but it sounds like we would stay under 8M if we didn't include all the update junk when we install. I call it junk because it of no interest to a new install - all the update functions are ignored. I think they are only used to set the number in schema_version. So one solution is to split hook_install() from the slew of update functions and to deal with schema_version. This requires some more thought, and some more code, so I'll gracefully bow out here :)

scor’s picture

Moshe, the other problem we notice here is not only specific to the install: Drupal is likely to go over 8M on the modules page for example, on a fresh core install (see measures taken in #6). So it might be wise to remind these users having a low memory limit.

moshe weitzman’s picture

"- the system page loads all .install files to check whether updates are available to run, so it takes more memory there..."

Perhaps we can just drop this. Now we have update module, which tells you have to upgrade but uses a totally different mechanism. It compares your .info file information versus published release info from drupal.org.

Gábor Hojtsy’s picture

Moshe: there are two checks here:

- update status checks whether the remote package repository has newer releases, then what you have installed
- the module page check looks for update functions you did not run yet on the local database

For the remote check, we need remote data compared to local data. For the local check, we need the set of names of all update functions in all .install files and that compared to the local database.

Now the local check makes the module page consume considerably more memory, because of loading all the update code, and this just increases a lot with contrib modules being present. I opened another issue for this local issue, as it is marginally related to the 8MB memory limit (as pointed out earlier, we have a memory limit problem regardless of the modules page, but the modules page underlines this with considerably more amount of memory required). The issue: http://drupal.org/node/199460

Let's continue here with the generic 8MB problem.

theborg’s picture

I use MAMP with :

# Apache 2.0.59
# MySQL 5.0.41
# PHP 5.2.3

and I cannot install with 8Mb, following the installation proceess i found that when saving cache_menu in menu.inc at:
cache_set('router:', $menu, 'cache_menu'); produces a big $data variable in function cache_set:
$data = serialize($data); and then stops.

scor’s picture

theborg: can you try the patch #50 and report? (you should see a warning about the memory limit while installing drupal).

theborg’s picture

@scor: tried to reproduce today but with the new head files is working smothly.

catch’s picture

@theborg: #50 should warn you about the memory limit if you have under 16MB - so it should be quite possible to review even if your install works now.

theborg’s picture

Tested and it works!

System:
- Suse 10
- PHP Version 5.1.2
- MySQL 5.0.26

scor’s picture

beside the warning during the install, this patch adds a warning in the admin/reports/status page. see screenshots. (make sure to add php_value memory_limit 8M to the .htaccess file to test this patch).

keith.smith’s picture

FileSize
5.05 KB

Your screen shots (thanks for those, by the way) made me look closer at the text added by this patch.

Specifically:
* it seems unlikely that anything can "ensure a successful installation process". :) But, it will certainly "help prevent memory errors".
* someone just installing Drupal may not know what "Drupal core" is.
* "especially if you using" => minor tweak to "especially if your site uses".

Hopefully, I did not change your intended meaning, but please review carefully -- its been a long day.

JirkaRybka’s picture

Nice :) But IMO the link to drupal.org requirements page should show some sensible label rather than the full URL on UI.

keith.smith’s picture

FileSize
5.13 KB

Very good point. The editing cortex of my brain usually shuts off when I see a URL so I didn't even think about that. New patch attached, that I have not tested.

scor’s picture

FileSize
4.91 KB

thanks keith.smith and JirkaRybka for your reviews ;)

catch’s picture

Well the strings look great to me now, agree with the additional warning on reports, but I don't anything to actually test it on. Looks ready though.

douggreen’s picture

FileSize
4.91 KB

There was a small coding style error in if(

gopherspidey’s picture

I am not familiar enough with drupal to know it there would be an appropriate spot to do a memory check to prevent the "WSOD".

I was thinking of something like this.

if  (memory_get_usage () > ini_get(memory_limit) - 4096) {
  echo "Please increase your memory_limit";
  exit;
}

I think it would help new drupal users.

Pancho’s picture

Just want to point you to #200674. I didn't follow this discussion here in details, but hopefully update.php is covered as well? Probably by dumping the old update_N's, I guess...

vjordan’s picture

Actually, I have read this thread and so far install is the focus, not update (which was mentioned in #49). The patch currently deals only with the install process. Keep #200674 open for now. It can be marked duplicate and closed if a patch comes through here which solves that problem.

catch’s picture

Status: Needs review » Needs work

system.module hunk fails- needs a reroll.

scor’s picture

Status: Needs work » Needs review
FileSize
4.86 KB

rerolled.

catch’s picture

Status: Needs review » Reviewed & tested by the community

Well I tested this on a fresh install with a memory limit of 36M - no warning in install, all green and rosy on admin/reports/status and it told me what I already knew.

theborg has tested with 8M memory and only string/code style changes since then, so this ought to be ready.

scor’s picture

@pancho, vjordan: this patch takes care of the install and the runtime. update is different story, and there is still work being done on it. Let's focus on the install and runtime first, which is now RTBC.
edit: for the update, see http://drupal.org/node/200674

Gábor Hojtsy’s picture

This is in RTBC for some time now, so I'd like to inform you that I discussed this with Dries, and he said that he would like to look into possibilities to let Drupal run under 8MB still, if he has the time. Sadly noone else took that helm I suggested above, so I am not going to commit this until I hear back from Dries.

catch’s picture

re: running under 8MB - at least some people can, so it seems quite borderline.

We can apparently shave about 500kb off memory required for update by removing 4.7 updates according to scor in #200674 - presumably it'd be a similar saving at install time as well.

Putting the .schema definitions back into .schema files would also reduce memory consumption when either just .install or just the .schema files are need, but too late for that at this stage, I missed the issue where that was changed in the first place, and it's not all that much code anyway I guess.

theborg in #55 posted this which might be another thing worth looking at.

following the installation proceess i found that when saving cache_menu in menu.inc at:
cache_set('router:', $menu, 'cache_menu'); produces a big $data variable in function cache_set:
$data = serialize($data); and then stops.

At runtime, only thing I can think of that'd deal with it quickly would be disabling all non-required modules in the core install profile by default so initial memory consumption is the lowest possible. Reckon the warning is preferable to that though.

Dries’s picture

Status: Reviewed & tested by the community » Needs work

I think the approach taken makes sense. It seems like the best we can do, for now, is print a warning message. Personally, I'd prefer to give better instructions in the warning message itself. Linking to an external URL on drupal.org is not my favorite approach. I suggest we just tell them to edit php.ini and to restart Apache? That saves them a lot of reading and we can actually include their php.ini path.

Gábor Hojtsy’s picture

Dries: while we can point to the php.ini for sure, we know that on most shared hosts, people don't have permission to go and modify it. Depending on their server though they might have:

- a httpd.conf level php_admin_value set up which is not possible to modify later in any .htaccess
- a httpd.conf level setting but no .htaccess so they can only modify it with ini_set() - this one we can even try to set ourselfs if less then our requirement
- a .htaccess level setting possibility - this one we can even try to include, but then people with higher requirements would need to modify their .htaccess
- no setting possibility at all (this is at least common to Hungarian shared hosts)
- not even an LA*P server (eh, on Windows PHP settings could even be set in the registry)

So while we can provide basic info on what to do if we assume the user can modify their php.ini, this falls short on most shared hosts, where this exact memory problem would show up most. There is mostly no way to modify their php.ini there.

IMHO we can provide this basic info *and* link to a page with more details and user comments, so actual use cases are covered later. The thing is that we most probably won't modify the included instructions, but new use cases will evolve and those can be documented by ourselfs or our users.

NancyDru’s picture

On my current host, there is NO way to change my memory limit - and I know of other hosts that do the same. As a matter of fact, this host (and others) don't even have that option compiled into PHP, so you can't even use the "memory_usage" techniques. Remember that not everyone has a need for, or can afford, VPS or dedicated hosting.

However, if I remember correctly, PHP 5 increased the minimum to 16MB. I personally will not even think about a host that is not on PHP 5. So I would definitely state that PHP 5 is greatly preferred for D6, and remind people that it will a requirement in D7.

darumaki’s picture

I'm using php5 and my will only work with 128m and that's with not too many modules, not sure where the recommended 12-16m came from. Drupal is a hog and requires much, it would be better to be lite and fast then to have too many gigets running behind the scenes. Performance is more important than gigets.

NancyDru’s picture

@darumaki - are you sure you're not including MySql and Apache in that figure? Even my biggest site runs fine in 16M.

hass’s picture

@Gabor: php.ini values are not in the Registry on Windows... there is no difference to *nix systems. e.g. "C:\Program Files\PHP\php.ini"

Gábor Hojtsy’s picture

hass: http://hu2.php.net/manual/en/configuration.changes.php specifically "Changing PHP configuration via the Windows registry". I am not saying *you* have your php.ini directives in the registry, I am saying that some Windows based hosts might use this method to limit server level settings. Or they might not. I have no idea about how widely this usage is accepted, but it is definitely a PHP feature.

hass’s picture

Hui... i was not aware about this feature. Again learned something new :-)

darumaki’s picture

not sure what you mean - the 128m is in my php.ini and yes i do use mysql and apache, when i try to change the 128 to 16 drupal goes blank

shunting’s picture

Is there a way we can determine whether the installation is on a shared host? Maybe php.ini not present or present with wrong permissions?

If so, we could adjust the error message accordingly. "It seems likely that you are on a shared host. Please supply your administrator with this error message." Put it on their desk...

Gábor Hojtsy’s picture

shunting: the user/group with which Drupal runs could very well be different to what user/group the user herself is when she uploads files. Although we can get what this later user is by examining file ownership, we cannot check whether this user has permission to modify the php.ini, or at least I have no idea.

catch’s picture

I don't think we should try to do anything clever with this - too many configurations to deal with.

How about

Your PHP memory limit is set to %memory_limit. Consider increasing this to %memory_minimum_limit to help prevent memory errors in the installation process. If you have access to php.ini, find the line memory_limit %memory_limitM and change this to memory_limit %memory_minimum_limitM. See the <a href="@url">Drupal requirements</a> for more information.

Also the requirements page should mentioned contacting your hosting provider if you can't change the memory limit yourself (if it doesn't already).

NancyDru’s picture

You do realize that if the memory option is not compiled into PHP, that you can't even find out what the current setting is, let alone change it. And some web hosts, such as my current one, will not change it because it would change it for everyone on the server. So, in some cases, you can only make an educated guess at the current setting based on the PHP version.

Gábor Hojtsy’s picture

I think it is fine to provide info on the php.ini, keeping in mind that it is a common way to deal with PHP settings, but giving a link to more docs as well, because admittedly there are other common ways ranging from "not possible" through "contact your server admin.." to "on your web control panel....".

catch’s picture

Status: Needs work » Needs review
FileSize
5.05 KB

Don't have time to do a proper re-roll now (happy new year!), but here's an untested text editor version of #71 with the wording from #87.

keith.smith’s picture

Status: Needs review » Needs work

Happy new year catch! (I've got another five hours to go!)

The string beginning with Your PHP memory limit is set... has two instances of See the Drupal requirements for more information..

catch’s picture

Status: Needs work » Needs review
FileSize
4.98 KB

oops. Here's one without the beer-induced silliness. Enjoy the last hours of 2007.

On the plus side, this still applies with a bit of fuzz.

hass’s picture

It's minor, but shouldn't there a period, colon or exclamation mark on the end of the sentences!?

st('The following installation warnings should be carefully reviewed') : st('The following installation warning should be carefully reviewed');

st('The following installation warnings should be carefully reviewed, but in most cases may be safely ignored') : st('The following installation warning should be carefully reviewed, but in most cases may be safely ignored');

catch’s picture

FileSize
4.98 KB

Yes.

keith.smith’s picture

hass: This string:

st('The following installation warnings should be carefully reviewed, but in most cases may be safely ignored') : st('The following installation warning should be carefully reviewed, but in most cases may be safely ignored');

could use a period but it doesn't look like this patch is actually modifying that string.

(Off topic for this issue, but can I just point out that this string has bigger issues than a period. We should carefully review, but yet, ignore, the same warning?)

catch’s picture

FileSize
5.08 KB

Keith's right, and #94 doesn't change that string at all, it just happens to appear in the patch file.

Here's a new one to remove the fuzz and offset. Ought to be ready now with a bit of luck.

Gábor Hojtsy’s picture

Status: Needs review » Needs work

I note two things from the latest patch:

- "find the line memory_limit %memory_limitM and change this to memory_limit %memory_minimum_limitM" is bogus
- why only check for the memory warning in the installer? Why not go through the requirements problems and set all REQUIREMENT_WARNINGs as dsm() warnings? That would allow any contrib install profile to emit warnings here (eg. if they also have pressing needs)

Otherwise it looks good.

Gábor Hojtsy’s picture

Anyone taking the remaining issues up?

catch’s picture

Status: Needs work » Needs review
FileSize
5.1 KB

If contrib modules have pressing needs, shouldn't they be setting them as hard requirements rather than warnings? To me this is a special case, and allowing contrib install profiles to choose between failure and warning seems like both a feature and something which could introduce bugs later if they make the wrong choice. I'm not against it, but I can't think of a situation it'd be especially useful.

I changed the php.ini string, and added it to the runtime warning as well. This is it:

If you have access to php.ini you can usually change this setting there.

Short and generic as possible, not 100% happy with it but it's all I've got for now. I put 'usually' because as nancyw describes there are always potential edge cases.

Gábor Hojtsy’s picture

Status: Needs review » Needs work

I don't see how would iterating through $requirements to drupal_set_message() descriptions where severity == warning hurt any contrib. Can you provide any examples? These are only install phase requirements which are clearly marked as REQUIREMENT_WARNING, so if there would be any error, it would be in the contrib module, right?

Maybe this sounds like a feature, but IMHO we are better off adding a generic warning setter here, then hard-wiring a special case. It is probably around the same amount of code as well, maybe one more line.

catch’s picture

Status: Needs work » Needs review
FileSize
14.51 KB
4.9 KB

Gabor: the only example I was thinking of is if someone incorrectly sets a warning instead of an error in an install profile allowing someone to install without meeting the requirements, I guess developers should just not do that.

This deals with your concern I think, and it definitely looks cleaner.

It now does exactly the same as REQUIREMENT_ERROR due to the beauty of ctrl-c and ctrl-v, and I tested with system.module set to recommended 200M in order to get it to show up on my server. Due to the way the message is constructed, it tells me the memory_limit twice. I think that's probably OK, but otherwise the string needs changing again to take this into account, since we should keep the same construct of (currently using !item !version) for consistency between the two.

Gábor Hojtsy’s picture

Good. This just comes down to some nitpicking I guess. We should either reword the message to accomodate the currently using %version present after the message, or not set the value key on the install case, so this will not show up there. Looks like the runtime case also repeats the memory limit in the message, but is more fittingly worded to make the repeat text look adequate. Otherwise the patch looks good to me for commit. Will try to get Keith here as well :)

catch’s picture

FileSize
14.21 KB
4.82 KB

Runtime warnings don't have the (currently using item! version!) appended to $message for memory limit, so it's not duplicating there.

Here's one with the first reference to memory limit removed. Has the advantage of the warning being one sentence shorter as well.

edit: screenshot.

Gábor Hojtsy’s picture

Status: Needs review » Fixed

As a last point, I looked into whether PHP provides any info on where the php.ini is located, so we can inform the users as Dries noted. While it does provide this info in phpinfo() as we know, it does not provide an API function for this (which I could find), so we could only get phpinfo() into a buffer, and then grep out the location. I am not sure it is worth it here.

Anyway, as the latest patch looked nicely solving the issue to me, I committed it, so we can take on the update memory requirement issue as well. If someone feels strong about any follow ups, feel free to post them.

asimmonds’s picture

get_cfg_var('cfg_file_path'); should return the location of php.ini

Gábor Hojtsy’s picture

Priority: Critical » Normal
Status: Fixed » Active

Hm, indeed. It would be great to have the path in the warning message output then as Dries suggested.

adrian’s picture

We could move the schema version to the .info files, which could help us not need to load the install files to detect upgrades, and we could more easily operate without loading any of the module code for when we need to upgrade / install.

keith.smith’s picture

The attached patch, on my system at least, produces a message as shown in the screenshot below.

Please note that I have no mad PHP skillz. But, its a quiet Friday afternoon and I thought I would give this a shot.

catch’s picture

No time to test this minute, but it all looks sane and the addition to the warning looks great.

keith.smith’s picture

Status: Active » Needs review

Thanks catch!

(and I forgot to change status)

scor’s picture

FileSize
3.87 KB

%memory_minimum_limit is still required for the runtime message.

scor’s picture

asking users to edit php.ini sounds like a joke on shared hosts - which is where this issue is the most likely to happen. Shared hosts is also where you find many less experienced users.
alternative solutions should also be mentionned, such as:

  • .htaccess - in which we can add php_value memory_limit 16M commented by default, so that normal installations are not affected by it
  • settings.php - where we can also add ini_set('memory_limit', '16M'); which would be commented by default

as pointed out by gabor in #77, these might not work in some restricted environments, but they are easier and more realistic than editing php.ini. (Note that at least one of these 2 solutions works in the 3 different shared host environments I use, but not php.ini).

NancyDru’s picture

I know on my current host that I cannot do it in settings.php. I have not tried it in .htaccess but I suspect it won't work there either.

Gábor Hojtsy’s picture

Status: Needs review » Needs work

I would do

if ($ini_path = get_cfg_var('cfg_file_path')) {
  ...
}
else {
 ...
}

This is more in line with the "positive" approach of Drupal, and inits the $ini_path variable to use later (without a repeated call to get_cfg_var()).

vjordan’s picture

Status: Needs work » Needs review

As pointed out in #77, and others, there are many configuration possibilities to successfully get to the minimum memory limit. It seems impractical to document the range of alternatives in the warning message as they are highly dependent on the user's set-up. Furthermore it might hinder rather than help the novice user to get this message at that time. I think we need a message more indicative (like #103) and less prescriptive (like #108).

We also need http://drupal.org/requirements to have more detail to ensure the various options for adjusting the memory limit (php.ini, .htaccess, settings.php, contacting one's hosting provider) are clearly explained. I'll have a go at this, although keith.smith is likely to have the last word, and best words, when it comes to making this lucid for users.

keith.smith’s picture

Status: Needs review » Needs work

vjordan: go for it -- as you correctly point out, there's no good way to cover all bases here. I have no real good ideas about what this message should say.

But, I think the patch is CNW based on your comments and Gábor's in #114.

scor’s picture

Given the range of possibilities which need to be documented, I think it's worth having a dedicated handbook page on how to increase the PHP memory limit. I looked around on d.o and found a couple of other pages, but the were all tailored for earlier version of Drupal. I have created http://drupal.org/node/207036 which is located in 'Troubleshooting FAQ > Webhosting issues' for now, but could be moved as a child of System requirements later on. The other advantage of having a dedicated book page is that we can welcome user input and improve it. Feel free to edit and append more information.

catch’s picture

I agree with scor on this - the original intent of the wording was to have the notice as uncomplicated as possible to avoid having to deal with every edge case (and these will change release to release anyway, like PHP5 having 16M as default). So a dedicated handbook page, under system requirements (but maybe also linked from troubleshooting) sounds great.

vjordan’s picture

I've logged a separate issue to work on the documentation on drupal.org (link scor's handbook page into the requirements page).

Back to the patch for this issue. The consensus appears to be pointing towards not referring to php.ini and its path. This is back to the patch in #103, I think.

catch’s picture

Note that in #76, Dries specifically asked for a mention of php.ini in the warning. I put this in the #103 patch, but personally think that it should probably just link to drupal.org/requirements or at least some centralised place where it can be explained (because as pointed out, only people who already know what to do will understand the existing message). If we want the information somewhere local as well as drupal.org, should we consider putting it in INSTALL.txt and referencing that in the message instead?

scor’s picture

#103 has already been committed. What needs to be done now is to change the message and include a link to http://drupal.org/node/207036.
Shall we also include the path to php.ini in the warning message as mentioned in the handbook page:

The location of php.ini depends on your server. Drupal may be able to give its location.
vjordan’s picture

INSTALL.txt already contains the line:
For more detailed information about Drupal requirements, see "Requirements" (http://drupal.org/requirements) in the Drupal Handbook.

It seems reasonable to expect this will ensure the proactive users will read the requirements page and get the information they need before install. The error message will point the keen, but less prepared users, to the information they need when installing. And the not-so-proactive ones are going to post a support request anyway!

Gábor Hojtsy’s picture

Let's fix #114 and get this committed! Telling users the actually used php.ini is good information we can provide, as Dries pointed out. Otherwise there is a huge maze of configuration possibilities as we already explored and discussed. We will not document all this, but point to drupal.org's requirements page, which will lead users forward.

scor’s picture

Status: Needs work » Needs review
FileSize
3.82 KB

fixes #114 and some minor typos and logic.

keith.smith’s picture

FileSize
3.82 KB

Adds a period at the end of a comment.

scor’s picture

FileSize
4.82 KB

#93 and #94 introduced a period at the end of a string in the handling of warning messages, which shouldn't be used if we want to remain consistent with other message handlers. A colon : is appended to the string by the maintenance theme. As a result we end up with .: at the end of it (see http://drupal.org/files/issues/memorylimit_warning.png). This patch removes the period.

catch’s picture

Status: Needs review » Reviewed & tested by the community

Correctly found my php.ini on a new install (with the requirement hacked to make it show up). My only issue is it now uses the phrase memory limit four times in a very small space, but that's what we're talking about, so RTBC.

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Fixed

Committed, thanks.

Gábor Hojtsy’s picture

FileSize
4.42 KB

Just noticed that we introduced using t() instead of $t() with this patch and that some of my code reuse improvements were not applied, so I went ahead and did some code cleanup here. We should only start filling in text for the php.ini help for example, if we actually have something to report, we should not use $requirements['php_memory_limit']['description'] a dozen times, but build up a $description and then use that as other requirements checks do. I tested this cleanup and all was nice. This will also make the update memory limit patch neater a bit in my hopes.

DRagonRage’s picture

* For every required module enabled, we measure the length of the file (number of lines? number of characters? number of non-comment lines?) and test memory usage before and after loading the file, producing a statistic for the local environment.

not the size of the module file but the size of virables it generates so we take all the $_GLOBALS or al the $_ virables and sirealize them into a file or a just:
echo strlen(serialize($_GLOBALS))/1024)."Kb is used by this module";
or:
echo strlen(serialize($modules->moduleX))/1024)."Kb is used by this module:moduleX";

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.