Hi,

is there any reason, why xmlsitemap_generate_chunk uses
'ORDER BY' => 'ORDER BY x.language, x.loc'
and not something simple like
'ORDER BY' => 'ORDER BY x.id'
?

Later option would help me being more performant and use existing indexes. But maybe I am missing something?

related: #1003058: MySQL slow queries

Comments

Anonymous’s picture

Status: Active » Postponed (maintainer needs more info)

You should have two keys 'language' and 'loc', do you have those? Did you miss running update.php?

The reason for the ordering is for those who need separate sitemap.xml per language.

digi24’s picture

Status: Postponed (maintainer needs more info) » Active

Thanks for your reply earnie. My question is not really a bug report, I am trying to find out, whether some performance optimisations might break anything, it is difficult to notice with a large site and sitemaps.

Re: the keys
Yes those keys are present and they are being used. My problem is that I am seeing a "using filesort". (for explanation see http://s.petrunia.net/blog/?p=24)

This occurs even when using a minimal query on a large sitemap table:
SELECT x.loc, x.lastmod, x.changefreq, x.changecount, x.priority, x.language FROM `prefix_xmlsitemap` x WHERE x.access = 1 AND x.status = 1 ORDER BY x.language, x.loc LIMIT 0,30

The EXPLAIN output is as follows:

+----+-------------+-------+------+-------------------+-------------------+---------+-------------+--------+-----------------------------+
| id | select_type | table | type | possible_keys     | key               | key_len | ref         | rows   | Extra                       |
+----+-------------+-------+------+-------------------+-------------------+---------+-------------+--------+-----------------------------+
|  1 | SIMPLE      | x     | ref  | access_status_loc | access_status_loc | 2       | const,const | 520852 | Using where; Using filesort |
+----+-------------+-------+------+-------------------+-------------------+---------+-------------+--------+-----------------------------+

Now, this could maybe be solved by further increasing mysql caches etc. but I guess I would run into other performance problems sooner or later as I am using other modules similar to domain_xmlsitemap that implement hook_query_xmlsitemap_generate_alter. These modules typically add a JOIN on the id and joining on one column an sorting on another is often a performance problem in mysql from my experience or at least a pain in the *** to get the keys right. I played around a bit an noticed, that ordering by id seems to solve my performance problem even when joining other tables.

I am not using sitemaps by language, so I am not familiar how it is implemented by this module, but I do not really see the need for special sorting, as the the languages can be taken care of in the WHERE conditions, running one context at a time.

digi24’s picture

Issue summary: View changes

added related issue

yhager’s picture

Issue summary: View changes

The filesort killed my server. I have 2.3M nodes, and it generated about 70k IOPS (normal is under 1k). My VPS provider didn't like that :)

As a temporary change, I removed the language from the sort and it fixed my issue, however, a proper solution is needed (probably another index).

Subscribing.