Deprecated - Performance and scalability

Report originally posted here: http://spiders-lair.com/node/23 (which has the advantage of attaching a file which this forum doesn't).

I'm reposting this on this forum "for the record" as pointed out by "Khalid B" on the devel list

In a recent blog by Dries and in a follow post to the devel list, it was expressed that some optimisation within Drupal's database abstraction could be looked at along with SQL optimisation.

So, I decided to look into these possibilities. Given that the abstraction layer was most used (all SQL queries must pass through it) I decided to start there, with db_query().

We all know the param syntax for db_query(), a "tokenised" query string followed by one or more values to fill the tokens (or an array containing the same). So, I decided that would be where I concentrate my first efforts.

Looking at just how the tokens (placeholders, call them what you like), preg_match_callback() is used to call a helper "callback" function. This callback function is first pre-initialised with the array of parameters. Then, each time preg_match_callback() calls it, it passes off to a lower level function the actual escape sequence requirements.

All this is required to ensure Drupal doesn't "leak" an SQL injection attack by ensuring all parameters are escaped appropriately.

My first thoughts on how this all worked were "preg_match_callback" is a great filter mechanism but it has a performance penalty. First of all, the regex passed, define('DB_QUERY_REGEXP', '/(%d|%s|%%|%f|%b)/');, just says to me that using the PCRE library to do this is using a "sledgehammer to crack a nut". The PCRE does an awful lot of "stuff". Secondly, the callback mechanism from PCRE/PECL space to PHP userspace is a performance degrading operation. This normally isn't noticed, but given the large number of SQL queries generated to render a page, it all starts to add up.

Hello All,

Has anybody done any scalability tests of Drupal being accessed through dialup speeds? With a fresh install of 4.72 I seem to have a very noticeable lag in page load. No template changes, only one "test" page created but still the site takes a very long while to load up. Does this mean Drupal is not made for third world users? I was planning to use it for a huge organisation with target users not using high speeds. Any recommendations, experiences?

Thanks,

For table rows an odd/even style is often used. A repeated test for odd-/evenness would seem time consuming and different runtime environments have different optimisations. In PHP the following two functions have been performance tested rigorously:

Would pathauto scale to ~ 20 millions nodes? Anyone tried it with a substantial number of nodes?

I am trying to optimize some of my install's mySQL queries and I notice that 39 queries ( as shown thru the devel module ) are repeated 3 times. For example, on my front page I have a story, and this query is run three times:

SELECT SUM(total_clicks) FROM shared_clicks WHERE nid = 14063

I am in the process of optimizing performance on a dedicated server Debian + Apache 2.0.58 + PHP 5.1.4 + MySQL 4.1.10 + Directadmin + 1G RAM +Pantium 4 3Ghz.
I disabled eaccellerator and zend optimization on the test because I used XDEBUG.

The Problem: My tests show very fast SQL while the page rendering time without cache in drupal is high (~2000 msec)