Language not considered in page caching--users get wrong page

I enabled "normal caching" on my test site. I then created two countries 'ca' and 'us' and languages 'fr', 'en', 'en-CA', 'en-US' and 'fr-CA'. I enabled country based language negotiation and enabled the country switcher and language switcher blocks.

First I created a node with multiple translations and used the country / language switcher blocks to switch between different translations. It all worked fine. However this is probably because each translation is a separate node, so the nid is different for each translation, so there's no caching involved.

I then imported the fr.po translation file into 'fr-CA'. I viewed the 'ca/admin' page and it correctly appeared in English. I then used the language switcher block to switch to 'fr-CA' and the interface text was correctly translated into French. The language switcher blocks adds a GET param to the URL (?language=fr-CA) however, so I repeated the same test but instead of switching languages on the page I was interested in, I had a separate tab open on a different page and switched languages there. Then on the tab I was interested in, I just refreshed the page and the language was correctly changed from English to French.

So can't reproduce this issue yet...

stella: were these tests done as an anonymous user?

catch: no they weren't. I've since repeated the tests. I've noticed as anonymous and authenticated user, that if you view the registration / profile edit forms and switch languages, not all of the visible text is translated. Things like breadcrumbs and form labels are, but, for example, the Legal terms & conditions aren't (with Legal multilingual patch applied).

I've also noticed on node/add/ the text for the content type names are cached. So for example 'Poll' always appears even though I am viewing the site in French (Sondage) or German (Umfrage). However, I've tested this on a lot of other pages and everything is translated...

Again no problem when viewing nodes.

However, I then disabled the country code module, cleared the cache, and the same behaviour was exhibited. I then disabled caching, cleared the cache and again the same thing happened. It may be that I need to start over with a clean installation.

I can reproduce like this:

I have countries set up for US and France. Page caching is turned on, with a 1 minute minimum cache time.

1) As admin clear cache.
2) In your settings.php add $conf['country_code_ip_address'] = '62.23.8.157';. This will hard code a french IP address.
3) Log out.
4) Browse to the site root (/). You should now see the home page as France. This is the page that will be cached as the site root (/). This simulates a french user being the first user after a cache refresh to hit the home page.
5) Remove the IP address setting from settings.php
6) In a new browser, pull up the home page. I now the the cached french home page, even though my IP should be detected as a US ip.

Here's a patch to get around this. Using the above technique, I am appending a country_code_cache variable to the REQUEST_URI in hook_boot(), before the page caching happens. I originally wanted to use the language code, but language isn't available until after page caching occurs.

I decided to use country_code_cache as the variable name to try to avoid any collisions that may occur. I'm not sure if this is really a problem though.

This seems to be a core issue. I did the following test on a clean Drupal 6 install, with only locale and translation turned on. I have 2 languages set up, English (default) and Spanish. Page caching is set to "normal", and language negotiation set to "path with language fallback".

I used 2 browsers to cleanly represent 2 separate users. I set Firefox to use Spanish as the first language, to simulate a native spanish speaking user. I used Safari with no changes to represent an english speaking user.

The domain I was testing on is simply drupal6 so the site root is http://drupal6/.

1) While logged in as admin, clear the cache.
2) Log out. Since I was in Firefox, I ended up at http://drupal6/es
3) Go to the site root (without the language prefix). The page is still in spanish.
4) Now, in Safari pull up the site root. The page is in spanish.

I've updated #282191: TF #1: Allow different interface language for the same path which looks like the most relevant.

None of those issues quite dealt with the specific bug outline by Roger, so I've opened a new core bug (copy pasted steps to reproduce) here - #337378: Interface language is cached for anonymous visitors.

Should also have mentioned Roger's patch fixes the bug from my testing. I don't see the core issue being fixed any time soon, and this seems like a reliable option until then.

OK, so we don't have the full language system, but since this is only for salting the cache keys, I'm wondering if we could use the first 2/3rds of http://api.drupal.org/api/function/language_from_browser/6 - then replace the call to language_list() with a variable_get('language_list', '') - which we populate in a #submit, or cron, so it's available during hook_boot(). No patch yet, since I'm not yet sure how the combination of language and country is best handled, or for that matter whether it's viable to use such thin negotiation like that.

We'd get cached records for those languages, but presumably the actual cached page would be determined by our normal language determination since the page is cached after all that has run. My hope then, is that at worst we'd get some unwanted cache misses, and maybe some duplicate cache records - but correct content displayed - since every country-language combination would be assigned a different REQUEST_URI. Obviously not ideal, but the patch should be small enough that it might be worth trying out before doing more fundamental changes.

The page cache is only set by anonymous users though. http://api.drupal.org/api/function/page_set_cache/6

So in that case, unless anonymous users can set a language preference, then the language would be determined by either 1. language detection/default settings 2. The path of the page. Possibly a combination of the two. Assuming our handling of language detection in hook_boot either uses, or mirrors what core does, then I think it ought to work. It's not ideal of course, but for consideration anyway.

Here's an initial patch. It turns out the only non-trivial functions we need in hook_boot() are menu_get_object() and node_load() - so country_code_init_language() changes to country_code_boot_language(), and we run it in hook_boot only if caching is enabled and the user is anonymous - then append the language and country code to REQUEST_URI using Roger Lopez's method above. Otherwise the code runs in hook_init() as usual. Copied language_from_browser verbatim to save loading the whole language system (for now at least) since it's only a small function.

Loading enough code to actually run hook_boot and menu_get_object() doesn't seem like a viable option - since we'd have to load all the modules implementing hook_nodeapi('load') too. So I went with a function_exists and a direct database query as mentioned by nedjo above. There's a @TODO to remove this when the core bug is fixed - since those bits aren't exactly pretty.

This means that anonymous users ought to get consistent language determination - and apart from the direct node table query, the methods used to determine this for anonymous and authenticated users are identical. We're doing a bit of extra work for each anonymous page request, but I've tried to minimise this as much as possible. Posting this up for a look at the approach, not done thorough testing yet.

Edit - forgot to mention two things:

1. $country comes up as undefined variable - because it never gets set, and as far as I can see it's a bug in the existing code, maybe that hunk isn't getting run in normal usage? I'm not really clear where that's supposed to be coming from either. There's some horrible commenting out of that hunk for now and a big @TODO

2. drupal_get_normal_path() isn't available during hook_boot() either - not started on that bit yet.

so this is still needs work for a bit.

drupal_get_normal_path() requires path.inc - so I've added a require_once() for that - although I'm pretty sure it won't work if drupal is installed in a subdirectory. Not yet looked at what's supposed to be happening with $country.

Just a quick thought. Admittedly, I haven't thought this through completely.

It seems like we could save ourselves a lot of grief is we go with a system that maps each path to a language in a 1:1 relationship, and force redirects in situations where this wouldn't match.

- no prefix - after country and language are determined, force a redirect to the appropriate page. This would avoid caching of a non-language prefixed page.
- country prefix, no language - in the case where this country has a single language, serve the page; otherwise redirect as above.
- country prefix and language prefix - serve the page as usual.

In any of the above situations would serve a cached page in the requested country/language combination. For non-cached pages,

This assumes that countries with a single language can use only a single prefix, like /us/node/123. Countries with multiple languages would be forced to use a 2 prefix path like /ca/fr/node/123.

Subscribing