I see a lot of "Page not found" errors in dblog related to JavaScript translation files. It's about those files that need to be generated by the locale.module [see locale_update_js_files()].

Looking in source of the page, in <HEAD> I see the JS file requested in line: <script type="text/javascript" src="/files/languages/ro_e8ed5acc13877f103dc82336f22ac598.js?3"></script> but in the specified folder the file is not present. This is happen only for the Romanian language. The site is multilingual (EN, IT, RO). In italian version I got no error.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

David Koudela’s picture

Version: 6.6 » 6.8

I had installed version 6.6 including Czech localization and everything was OK.
After update to 6.8 I often get this error message in web logs:
Type: "page not found"
Place: https://www.my_own_web.cz/sites/default/files/languages/cs_212be125b34b8...

I have already the localization file, but it has different name:
sites/default/files/languages/cs_212be125b34b88632a34700d1c2cae3a.js

How can I repair it? Where can I find the file name stored?

plach’s picture

Version: 7.x-dev » 6.x-dev
FileSize
3.16 KB

Your problems are due to the fact that the Locale module is unable to rebuild the Javascript translation files: if one of these gets accidentally corrupted or lost there is no way to tell Drupal to rebuild it.
The patch should fix the problem: apply it, clear the cache (Site configuration > Performance > Clear cached data), visit a page which includes the missing file(s) and you should find it again in the languages directory.

If you are unfamiliar with patching or do not wish to patch the core, you can find a module which will let you rebuild the files manually here:

edit: (updated link) http://download.psegno.it/files/drupal/locale_rebuild_js.zip

plach’s picture

Title: JavaScript translation file not created but requested » Locale is unable to rebuild lost Javascript translation files
Version: 6.8 » 7.x-dev
Status: Active » Needs review
Issue tags: +JavaScript
FileSize
3.28 KB

This is the 7.x patch.

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

FileSize
3.21 KB

fixed patch

plach’s picture

Status: Needs work » Needs review
light9’s picture

Version: 6.x-dev » 7.x-dev

plach thank you so much! Your answer was very helpful for me.

plach’s picture

@a.horoshavin: did you test the patch or the module?

Agence Web CoherActio’s picture

Hi plach

Your mini-module above enables to rebuild the file properly (BTW the patch was not working for me on D6.8).

However, locale module is not working properly on my configuration :
- file system is private method
- file folder is outside the www folder
- Drupal is in sub-directory of www (called 'my_directory' in my case).

In that case, if I set up the file system path in absolute path, locale module breaks because the page tries to call "/my_directory/absolute_path/languages/fr_xxx.js" which it doesn't exsit.

The only way to fix this is to set the file system path as relative path to Drupal base directory: "../../files". Strangely, the page calls "/my_directory/../../files/languages/fr_xxx.js" which seems to be fine (at least not logging a page not found).

Is that a correct set up ?

Thanks

Laurent

Leeteq’s picture

Subscribing.

plach’s picture

Issue tags: +i18n sprint

I think this is another issue, I will try to investigate.

realOFF’s picture

I've tried the Locale Rebuild JS Module on Drupal 6.9.
I run it a couple of times but it seems it didn't work because every time I entered the Module page, I saw the same values in the table.
Now, I don't see the Module tab in the Language Settings and I can't even enter to the Edit page of the any language.
Any ideas?

Thanks in advance.

PS: Now I see I'm also getting error 404 on /path/to/your/sites/stylesheet.css?0. Maybe this is of any help.

plach’s picture

@LaurantB: as I supposed the problem you are reporting is caused by another issue, see #181003: private download method and dynamically generated CSS and JS for details. Unfortunately JavaScript translations won't work with the private file-system setting enabled until that one is fixed, that's why the patch is not working for you. Moreover I am afraid your soultion won't work because all the paths you suggest will be passed through file_create_path and rejected, as they actually point to locations outside the file directory.

@realOFF: the values in the table are supposed to remain unchanged, for the other problems I'm afraid I can't help you: that module is an as-is solution while this issue gets fixed in core; if it does not work for you, please try the patch.

kkretsch’s picture

Just one fix for the module from #2, you missed the curly braces in locale_rebuild_js.admin.inc around the table name, so with a fixed table prefix the first sql query throws an error. Line 107 should be:

$query = 'SELECT language, javascript FROM {languages} WHERE language IN ('. $placeholders .')';

at least now it worked for me.

plach’s picture

@kkretsch: thanks, I fixed that, but keep in mind that the module is just an interim solution while the patch find its way to the core...

Status: Needs review » Needs work

The last submitted patch failed testing.

David Koudela’s picture

I have used the module, but the problem still remains and nothing happened.
I still get the "page not found" errors.

Is there any procedure how to remove all correct/incorrect translation files from Drupal and install them again?
This would be sufficient for me now.

I accept that I will lose some of my manual translations.

plach’s picture

@David Koudela: apply the patch in #2, follow the instructions, delete (backup them!) the language files.
The correct files should get rebuilt and you should not lose you translations.

David Koudela’s picture

Ok, I did it:
- I applied the patch to the file and I copied it to the directory
- I removed the language files
- I cleaned the cache

But I still get "page not found" errors:
Type page not found
Date Tuesday, 24 February, 2009 - 10:42
User
Location https://www.koudelovi.cz/sites/default/files/languages/cs_f8cc24f295b687...
Referrer https://www.koudelovi.cz/cs/admin/reports
Message 500.phtml
Severity warning
Hostname
Operations

Any ideas what to do or how to better analyze the problem?

plach’s picture

Please, check one thing: in the languages directory do you have any cs_*.js files, apart from the missing one?

David Koudela’s picture

I have following file:
sites/default/files/languages/cs_f8cc24f295b6870eee00c27dc541587e.js

So it seems to me correct, isn't it?

I have checked also permissions - all directories have at least 755 (rwxr-xr-x), the file has 664 (rw-rw-r--)

plach’s picture

In this case I'm afraid neither the patch nor the module will help, they just rebuild the file, if it is actually there you are probably experiencing some kind of server misconfiguration. Are you using private download method?

dagmar’s picture

@David Koudela: Maybe files directory should have 775 permissions. Apache will create the js file. So, it must have write permissions to create the file.

This bug is also present in Drupal 6.10. Do we have to change the version of this issue?. Maybe it can be fixed in 6.11 version.

David Koudela’s picture

I assume you mean following settings: administer >> site configuration >> file system and check box "Download method"
My Download method is public.

plach’s picture

@dagmar: the issue the patch solves is present both in D6.x and in D7.x, to have the patch automatically passed to the testbed we have to keep the version to 7.x.

@David Koudela: I am sorry, this sounds to me as a server configuration issue.

David Koudela’s picture

@dagmar: Ok, I have set the permissions of directories to 777 for a moment to be sure, that there is no problem with the permissions.
Then I have deleted the cs_*.js language file and I have tried to access some language local pages, but I still get the same errors :-(.
The language file is re-created again.

I would say that it is right time to uninstall the local language with its translatations, make a clean-up and install it again.
But I do not know how to do it without damaging the system :-).

David Koudela’s picture

Well, I did a little progress.
I realized that I did not understand the message of the Drupal log.
The "missing" part was a page 500.phtml somehow requested by: cs_f8cc24f295b6870eee00c27dc541587e.js

I have created empty file 500.phtml at the root directory of my system and the problem disappeared.

I still do not understand why and where it is needed - search of Drupal files and database found nothing :-(
It seems to me a bug, but I can live with it.

Thanks for your help.

plach’s picture

Status: Needs work » Needs review
FileSize
5.28 KB

Rerolled the patch and added a simpletest for the JavaScript translations.

dagmar’s picture

Excellent. Then, the patch for D6 is still #2? Or you are going to do a re roll for 6.x too?

plach’s picture

FileSize
3.16 KB

Here is the 6.x version.

Frank Steiner’s picture

Status: Needs review » Reviewed & tested by the community

Works fine for our site!

dagmar’s picture

Status: Reviewed & tested by the community » Needs review

The patch should be tested in Drupal 7.
Sorry but, I don't think that your site is in drupal 7 :)

I have tested the 6.x version and can confirm that works fine too.

@Frank Steiner: If you have tested this patch for Drupal 7 please, forgive me, and change the status again.

Greetings
Mariano

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

Status: Needs work » Needs review
FileSize
5.07 KB

rerolled

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

Status: Needs work » Needs review
FileSize
5.45 KB

rerolled again

plach’s picture

FileSize
5.04 KB

fixed and encoding problem in the previous patch

cburschka’s picture

Status: Needs review » Needs work

The test-case code has blank lines that contain trailing indentation.

I'm not sure 'SELECT min(lid) AS lid FROM {locales_source} WHERE location LIKE \'%.js%\' AND textgroup = \'default\'' is proper - aren't strings containing single quotes supposed to be double-quoted rather than backslashed? I could be wrong there.

plach’s picture

FileSize
5.18 KB

@Arancaytar: thanks for the review, I removed all the trailing spaces. The query should be fine: the backslash quote is a PHP-level quoting not a SQL-level one, however I copied it from line 909 of the original locale.test.

Edit: sorry, I misunderstood your comment about quotes, however an identical line is already committed.
Edit (2): I checked the coding standards and it seems there are no strong directives to follow about single quotes vs double quotes.

jandd’s picture

Status: Needs work » Needs review

I can confirm that the patch in http://drupal.org/node/338630#comment-1307862 fixes the issue for Drupal-6 and would really like to have it integrated with the next 6.x release

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

Status: Needs work » Needs review
FileSize
5.11 KB

rerolled

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

Status: Needs work » Needs review
FileSize
5.44 KB

fixed the simpletest

Status: Needs review » Needs work

The last submitted patch failed testing.

lilou’s picture

Status: Needs work » Needs review

Setting to 'needs review' - testbot was broken.

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

Status: Needs work » Needs review

retry

Status: Needs review » Needs work

The last submitted patch failed testing.

plach’s picture

Status: Needs work » Needs review
FileSize
5.64 KB

rerolled

catch’s picture

Status: Needs review » Needs work

few code style things.

$dir .'/'. $language->language .'_'. $data_hash .'.js'
needs extra space for concatenation

+ $status = ($status == 'deleted') ? 'updated' : ($changed_hash ? 'created' : 'rebuilt');

Maybe spit the ternary into two?

 +      // Let slip through.

No clear what's slipping through what.

+  function testJavaScriptTranslation() {

This needs phpdoc.

db_fetch_object(db_query should be db_query()->fetchObject();

Should be db_query()->fetchObject();

plach’s picture

Status: Needs work » Needs review
FileSize
3.28 KB
5.95 KB

Implemented suggestions from #51

Status: Needs review » Needs work

The last submitted patch failed testing.

altavis’s picture

Thank you!

Patch from #52 has solved my problem. I haven't got any *.js file (or even language/ dir to be more precise) - probably lost them when moving from local to server. Anyway it seems to work nicely with D6.13.

momper’s picture

subscribe

ball.in.th’s picture

My language javascript file is missing too! The D6 patch in #52 fixed the problem for me. Thanks a lot. ^ ^

dagmar’s picture

Status: Needs work » Needs review
FileSize
5.95 KB
+  $dest = $dir . '/' . $language->language . '_' . $data_hash .'.js';

Should be

+  $dest = $dir . '/' . $language->language . '_' . $data_hash . '.js';

Status: Needs review » Needs work

The last submitted patch failed testing.

dagmar’s picture

Status: Needs work » Needs review
FileSize
5.95 KB

rerolled

GiorgosK’s picture

I used to get the warning every time I would "clear cache"
but now it does not appear after applying
patch from #52 for Drupal 6

talino’s picture

Private Download Method used here too. I have a .JS file in my files/languages folder (outside HTTP root). I tried applying the #52 patch (locally) with 'patch -p0 < locale-338630-52.D6.patch' (from the site root), and got this in Terminal:

patching file includes/locale.inc
Hunk #1 succeeded at 2166 (offset 4 lines).
Hunk #2 succeeded at 2184 (offset 4 lines).
Hunk #3 succeeded at 2220 (offset 4 lines).
Hunk #4 succeeded at 2240 (offset 4 lines).

Things haven't changed much since then. I still have the JS file being asked for in the source HTML. Cleared the cache etc, a bit stumped. This is the first time I apply a patch. Anything I'm missing?

plach’s picture

@talino: see #9-#13

talino’s picture

Thanks, I understand it's incompatible with Private Downloads, however currently the link is still in the source and explicitly displaying the full path to the private files folder (and the host's entire folder structure). This is a major security concern, enough for me to take the site offline. What am I supposed to do now? The site uses French po files for interface string translation but is not multilingual. And I can't use Public Downloads. Do I need to take down a month of work because of a link to a missing JS file?

Thanks.

talino’s picture

Apologies. Please delete my#63. Haven't clicked far enough. Got the patch and responding there (http://drupal.org/node/296831#comment-969355).

arhak’s picture

a little module for D6 which
Provides locale admin pages with buttons to rebuild/invalidate js translations.

is just an unobtrusive way to solve this unusual issue

- enable the module
- navigate to admin/settings/language#js-translations
- expand "JavaScript translations" fieldset
- use either "Invalidate js translations" or "Rebuild js translations" buttons

marcushenningsen’s picture

This small module works for me. I don't really like your custom module package name (arhak - tweaks), though, but that's a minor issue and I just changed it to my likings (Multilanguage =))

arhak’s picture

@#66 sorry about that
that is the way I mark the modules which doesn't have a project page in drupal.org

it should go under Multilanguage indeed

I haven't created a project for this neither for other modules because I won't be able to use CVS (explained at Looking for a CVS committer) and that workaround shouldn't be provided as a patch, since this issue has its own patch going on

koorneef’s picture

Thanks for your mini-module !
It solved my JS language problems. Hope you get CVS access soon.

sinasalek’s picture

subscribed

CKIDOW’s picture

great! thx!

Greeting

CKIDOW

Status: Needs review » Needs work
Issue tags: -JavaScript, -i18n sprint

The last submitted patch failed testing.

Status: Needs work » Needs review

fuerst requested that failed test be re-tested.

Status: Needs review » Needs work
Issue tags: +JavaScript, +i18n sprint

The last submitted patch failed testing.

fuerst’s picture

Status: Needs work » Needs review

Sorry, did not intend to test the failed patch again..
BTW: Modul from #65 works well, thanks!

arhak’s picture

@#68 now I have a partner who makes commits on my behalf
do you think such a tiny module should have its own project?
for feature request/bug fixes?

sinasalek’s picture

#65 works for me too.

arhak’s picture

@ #66 marcushenningsen, #68 koorneef, #74 fuerst, #76 sinasalek
does this worth to be in its own project?
or would be better leaving it here "as is" at #65?

arhak’s picture

BTW: if #65 would be a module I would add it a feature to re-import translations, since Drupal doesn't have a way to "upgrade" translations

sinasalek’s picture

I think it can be part of the i18n module since it's about translation. But if its maintainers didn't like the idea then a separate module might be the only option because this patch/module i believe is most have for almost all non English/Multilingual sites.

dagmar’s picture

Priority: Normal » Critical
Status: Needs review » Needs work

Wait, we cannot test a third party module to solve a core issue. If this bug is still present in Drupal 7 (maybe it was fixed weeks ago) we should provide a patch to fix the core, and once it was committed, make a port to D6.

I'm using #52 in my drupal 6 sites and every times that cron runs, I get a lot of message in my dblog report

locale	11/27/2009 - 08:46	Parsed JavaScript file misc/textarea.js.	Anonymous
locale	11/27/2009 - 08:46	Parsed JavaScript file misc/drupal.js.	Anonymous
...

I'm not sure if this is the expected behavior, cron runs every three hours, why my js files are parsed if they probably doesn't change.

I'm changing the status of this issue to critical since it seems not be enough important to core maintainers and people that build sites with several languages needs solution for this issue.

We need that somebody confirms that this issue is still present in Drupal 7, if not we should back to D6 version.

arhak’s picture

@#79 I wouldn't like to speak lightly, but it seems to me that the maintainer of i18n doesn't want to make his module more complicated

no new features allowed?
http://drupal.org/node/578360#comment-2220526
http://drupal.org/node/582438#comment-2220622

maybe an i18n_bonus module? ("i18n_bonus", "i18n_extra", or what?)

sinasalek, make me a proposition about where to bundle all i18n support we can bring, lets talk about it through mail

arhak’s picture

@#80 sorry for dirtying this thread talking about an "alternative" module, it just seems like this patch stopped being attended
I'm in favor of this patch getting done
(don't know what about it in D7)
in the meantime, just referencing an "alternative" solution
now debating if we should open a module to get some fixes like this collected and discuss on its issue queue (letting this patch thread be)

sorry for the inconvenience

Heine’s picture

@81, @82, this really should be solved in core.

arhak’s picture

@#83 we all agree
"alternative" & "in the meantime" were already mentioned above

EDIT:
PS: to get off this thread I opened a forum topic Gathering more i18n support

sinasalek’s picture

It seems that everyone is agree that this patch should be in core. So if there is any hope of getting it into the core i think maybe a separate module is better , this way we can show the popularity of it to core maintainers. Otherwise IMHO http://drupal.org/project/i18nui by @Jose Reyero is an appropriate place for this module, since i18n maintainer does not accept it.
I invited Gábor to this thread , he was a core maintainer once and he is now responsible for localization server. Maybe he show us the right direction for getting this into the core.

Gábor Hojtsy’s picture

I was asked to look at this issue, but I found it quite hard to parse the problem and the proposed solution here. Can someone who's already active here summarize the issue and the proposed solution, so I can help out?

plach’s picture

Assigned: Unassigned » plach

@Gábor Hojtsy: I am providing a summary and a rerolled patch ASAP, meanwhile you can have a look to #2.

plach’s picture

Status: Needs work » Needs review
FileSize
6.06 KB
3.14 KB

Rerolled #59. This should be ready to go. If someone can test it, confirm it works, and set it to RTBC, we should be able to get this in core soon. The D6 version is almost identical with the exception of simpletests, so backport shouldn't be a problem.

The priority guidelines say this isn't a critcal issue.

The problem we are facing here is that the Locale module is unable to rebuild the Javascript translation files: if one of these gets accidentally corrupted or lost there is no way to tell Drupal to rebuild it.

Currently _local_rebuild_js creates the javascript file only if the hash code of the newly gathered string translations is different from the existing one. The patch simply adds a check on the file existence, this way if the file has been somehow lost it gets rebuilt even if the hash codes are identical.
A simpletest is added in the 7.x version.

To test the patch: apply it, add a language, create a translation for the "Unspecified error" string and visit the "admin/reports/dblog page", delete the javascript translation file, clear the cache, visit "admin/reports/dblog page" again and you should find the rebuilt file the "languages" directory.

@dagmar:

I'm using #52 in my drupal 6 sites and every times that cron runs, I get a lot of message in my dblog report

There is something that's clearing your caches (at least the ones triggering the parsing of javascript files) at each cron run, this should have nothing to do with this patch: I tested it on a plain 6.x installation and a cron run doesn't trigger a new parsing.

plach’s picture

There were some inline comments and PHP docs not wrapping correctly.
Code is identical to #88.

Gábor Hojtsy’s picture

I've looked at the patch, and it seems to be good to solve the issue, and doing what it says :) I did not try to run it, but it looks quite good.

sinasalek’s picture

Thanks Gábor for the comment , i hope that this time core maintainers accept.
@plach Thanks for re-rolling the patch, i'll test D6 version and report back.

dagmar’s picture

Priority: Critical » Normal

Sorry for change this to critical. Now that, Heine and Gábor Hojtsy has seen this issue, I'm changing the priority to normal again.

@plach: Maybe the cache for block is triggering the rebuild of the cache. Can this be the reason of my rebuild messages?

fuerst’s picture

@#77: I see this module as a welcome work around. Fixing the bug in core is the prefered way though. So there is no need to make it a Drupal project I think.
@#78: Open Atrium (openatrium.com) has some mechanism to upgrade translations. May be worth to look there before doing own work.

arhak’s picture

@#93 thanks fuerst for pointing that out

@#93#77 it seems that this patch will move on again, then that little module has no reason to become a project

@#93#78 wasn't aware about Open Atrium, will have to take a look at it

plach’s picture

Status: Needs review » Reviewed & tested by the community

Tests pass, Gabor gave his ok, and the D6 patch (which is identical to the D7 one except for tests) has been tested and confirmed to work.
However I feel uneasy to set my own patch to RTBC: can't someone find 10 minutes to test it and confirm this works even on D7?
Please keep in mind that the sooner the D7 version goes in, the sooner we can have a backport for D6.

plach’s picture

Status: Reviewed & tested by the community » Needs review
sinasalek’s picture

I think i'll have time to test it tommorow. both d6 and d7

sinasalek’s picture

I'm unable to reproduce the issue on my local Drupal installation!! it might have something to do with windows or the other modules. I'll try again on Linux and fresh install. it might takes a little bit more time

sinasalek’s picture

I'm unable to reproduce the issue on my local Drupal installation!! it might have something to do with windows or the other modules. I'll try again on Linux and fresh install. it might takes a little bit more time

arhak’s picture

I'll test the D6 patch today (don't know when the D7 patch)

How to reproduce the issue:
- fresh Drupal install
- enable locale module
- (lets pick Spanish language for instance)
- add some .po files in that language (e.g. es.po enabling dhtml_menu module would be enough, since it has Spanish translation)
- add the new language admin/settings/language/add
- visit some page in Spanish (dhtml_menu menu is present on every page)
- check directory default/files/languages was created and a translation file as well (e.g. es_[MD5].js)
- manually delete the .js file
- refresh the page in Spanish and check the file was/wasn't re-created

arhak’s picture

D6 not working:
- patch cleanly applied
- bug reproduced following steps @#100

@plach
Perhaps I'm making a dummy question, but all changes in the patch are made to function _locale_rebuild_js.
Is the bug actually in there?
Wasn't that function working properly?
Did you reviewed the mini-module @#65?

The mini-module @#65 uses _locale_rebuild_js as it is.
It does NOT introduce any new code. Just a callback and a link to manually invoke that function on demand. (and has received positive feedback)
The problem seems to be (IMO) to detect when the file is missing (note that the hash will be the same known in DB)

am I wrong?

arhak’s picture

I just re-read #88 and the issue is well described

this way if the file has been somehow lost it gets rebuilt even if the hash codes are identical

I'm not seen this happening, since _locale_rebuild_js is not getting called (as I recall, haven't debug it recently)

BTW: without the patch, clearing the cache also fixes the problem, the only reason for another solution (like mini-module @#65) is that "clear cache" can't be afforded by sites with heavy traffic

If testing the patch requires clearing the cache (as I did when testing it on D6) the file will be successfully rebuilt, but then being manually deleted once again brings the same issue back. (and clearing the cache again doesn't makes sense, since it works too without the patch)

PS: forgive/correct me if I'm wrong in something

plach’s picture

I tested everything again and I can confirm the patch works both on D6 and on D7.

These are the steps to reproduce the bug:

  1. fresh Drupal install
  2. enable Locale module
  3. add a new language (e.g. French)
  4. configure language negotiation to use path prefix (for D7 set content language to URL and interface language to Content)
  5. enable the (content) language switcher block
  6. translate the string "Unspecified error"
  7. use the language switcher to change to French
  8. visit the "Recent log entries" page
  9. delete the file named fr_[MD5].js in the files/languages directory
  10. refresh two times the "Recent log entries" page
  11. check that you have a page not found error for the deleted file
  12. clear the cache
  13. check the "Recent log entries" page again and you should have another page not found: the file has not been rebuilt

To test the patch start from the situation above and:

  1. apply the patch
  2. clear the cache
  3. visit the "Recent log entries" page again
  4. you should have no more page not found errors and a warning starting with "JavaScript translation file ..." saying the JS file was lost: the file has been rebuilt

@arhak:

I reviewd #65: it works without touching _locale_rebuild_js because it resets the hash values stored in the DB. This way the check the patch fixes passes and the file gets actually rebuilt.

_locale_rebuild_js is called everytime the cache is cleared, however it does not rebuild the file unless some string (hence the overall MD5) has changed.

I think clearing the cache is ok, also to rebuild lost CSS or JS aggregates you have to do that.

arhak’s picture

both patches work
(as described @#103)

tested (before and after patching) on D-6.14, D-6.15, D-7.x-dev(2009.12.15 tarball)

D6 patch cleanly applies
D7 patch applies with three offset warnings (trivial)

arhak’s picture

on the side notes:

1- I'm not aware about the code style guidance for fallingthrough cases, but for case 'rebuilt': falling through case 'created': I would prefer and ending comment different from // (re)creation.
I suggest to use something like // fall through or // NO break; as the last statement

2- consider the unclear logic around $status

2a-- variable being first set to 'deleted' to be corrected later on for 'updated'.
Maybe this is the better way, (since at the deletion point there is no way to ensure that file_unmanaged_save_data will succeed), but $status = 'deleted'; might be commented about it

2b-- uniformity when testing casuistic

      if ($status == 'deleted') {
        $status = 'updated';
      }
      else {
        $status = $changed_hash ? 'created' : 'rebuilt';
      }

consider an elseif to avoid mixing if-else with ternary operator

      if ($status == 'deleted') {
        $status = 'updated';
      }
      elseif ($changed_hash) {
        $status = 'created';
      }
      else {
        $status = 'rebuilt';
      }

3- (D6 code is frozen, but for D7) the whole logic of function _locale_rebuild_js should be cleaned up (would that be possible?)
an overall view:

function _locale_rebuild_js($langcode = NULL) {
  ...
  if (!empty($language->javascript) && (!$data || $changed_hash)) {
    ...
  }
  ...
  if ($data && ($changed_hash || !file_exists($dest))) {
    ...
  }
  ...
  if ($status && $changed_hash) {
    ...
  }
  ...
  switch ($status) {
    ...
  }

testing several times the $status variable is not a big deal
my concern is around these combinations:
- (!$data || $changed_hash)
- $data && ($changed_hash || ...)
- $status && $changed_hash

PS: last, but not least, this patch fixes the bug by means of clearing the cache, if a production site wouldn't afford this cost the module @#65 keeps being a best (though workaround) approach, since manually intervention would be required anyway, I would prefer not to lose all the cached data

plach’s picture

@arhak: Thanks for reviewing and testing:

1. That was not a two-lines comment, I didn't mean to have an inline comment saying just "(re)creation"; perhaps the one in the attached patch is more clear.
2. I added some inline comments and introduced an elsif branch as suggested.
3. If you wish to clean up _locale_rebuild_js please open another issue. Let's not hold up this bug fix.

About cache clearing, as I said before, I think it's ok as a core solution: if you can't afford to clear caches you'll have to resort to manually clear the variables entry in the cache table and reset the javascript_parsed variable; but this is a nice task for a contrib module as yours. In core we have no such fine grained cache handling: if you wish to rebuild CSS/JS aggregates you have to clear all caches.

arhak’s picture

Status: Needs review » Reviewed & tested by the community

marking it RTBC as per #95 (and my own tests)

on the side notes:
- D7 patch now cleanly applies (against 7.x-dev 2009.12.15 tarball)
- patches keeps being the same since #89 (just got more comments and an elseif instead of ternary operator)

@plach: thanks a lot for taking the time to fix this
forgive my misleading comments #101, #102, #105, and those related to the mini-module @#65

plach’s picture

@arhak: thanks for helping, hopefully the bot will agree on the RTBC status :)

plach’s picture

+++ includes/locale.inc	18 Dec 2009 10:07:04 -0000
@@ -2926,22 +2927,38 @@ function _locale_rebuild_js($langcode = 
+      // If the file did not exist previously and the data has has changed we

double has here :(

This review is powered by Dreditor.

Attaching the fixed patches.

andypost’s picture

+1 to commit, tested on d7-cvs and 6.15

Yas375’s picture

FileSize
14.27 KB

After applying a patch to D6 and clear cache (with devel module) I recieve error: Fatal error: Unsupported operand types in /var/www/publish/stereo/htdocs/modules/locale/locale.module on line 522

And I have to revert chacges to make my site working again

Yas375’s picture

I have changed permissions for my includes/locale.inc and it's ok now.

yas@serv ~/stereo $ chmod 777 includes/locale.inc
yas@serv ~/stereo $ patch -p0 < locale-338630-109.D6.patch

webchick’s picture

Status: Reviewed & tested by the community » Needs work
+++ includes/locale.inc	18 Dec 2009 17:13:18 -0000
@@ -2974,6 +2992,10 @@ function _locale_rebuild_js($langcode = 
+      watchdog('locale', 'JavaScript translation file %file.js was lost.', array('%file' => $language->javascript), WATCHDOG_WARNING);

This is a warning level. What is the reason such a JS file might be lost? Can we give users some direction on what to check so that it doesn't happen again?

+++ modules/locale/locale.test	18 Dec 2009 17:12:20 -0000
@@ -292,6 +292,62 @@ class LocaleTranslationFunctionalTest ex
+    $query = db_select('locales_source', 'l');
+    $query->addExpression('min(l.lid)', 'lid');
+    $result = $query->condition('l.location', '%.js%', 'LIKE')
+      ->condition('l.textgroup', 'default')
+      ->execute();
+    $url = 'admin/config/regional/translate/edit/' . $result->fetchObject()->lid;
+    $edit = array('translations['. $langcode .']' => 'string translation');
+    $this->drupalPost($url, $edit, t('Save translations'));
...
+    locale_inc_callback('_locale_rebuild_js', $langcode);
+    $file = db_select('languages', 'l')
+      ->fields('l', array('javascript'))
+      ->condition('language', $langcode)
+      ->execute()
+      ->fetchObject();
+    $js_file = 'public://' . variable_get('locale_js_directory', 'languages') . '/' . $langcode . '_' . $file->javascript . '.js';
+    $this->assertTrue($result = file_exists($js_file), t('JavaScript file created: %file', array('%file' => $result ? $js_file : t('not found'))));

The rest of this test is commented really well and is easy to follow, except for this chunk here which is the most complex. Could you just toss another couple inline comments before each of these that explain what's going on?

Also, could someone just confirm quickly that Yas375's results are an outlier, and not something we need actively be concerned about?

This review is powered by Dreditor.

plach’s picture

Status: Needs work » Needs review
FileSize
6.83 KB

Comments added as suggested. D6 version is untouched.

This is a warning level. What is the reason such a JS file might be lost? Can we give users some direction on what to check so that it doesn't happen again?

I ain't sure it's correct, but the reason I used a warning is this shouldn't happen in production environments. The only situations in which I had this problems were while deploying sites and not having the rights to write in the 'files' directory. Thus I've been forced to rebuild the file through Drupal.
I can't imagine any (non-malicious) reason why those files would disappear in a production site.

Also, could someone just confirm quickly that Yas375's results are an outlier, and not something we need actively be concerned about?

I tested the patches again and had no errors. It seems Yas375 had troubles with applying the patch.

arhak’s picture

Status: Needs review » Needs work

What is the reason such a JS file might be lost? Can we give users some direction on what to check so that it doesn't happen again?

Hard to know:
- huge chance if a DB backup is imported to other site which doesn't have those files generated
- site moved (without some files)
- manually deleted
- changing file system setting (?)
- applying patches to support private & public download methods

Also, could someone just confirm quickly that Yas375's results are an outlier, and not something we need actively be concerned about?

- Can't reproduce it
- Can't see how it might be possible with/without patch or even with a partial patch applied (assuming the missing file permission allowed only part of the patch), since that line does $parsed += locale_inc_callback('_locale_invalidate_js');
-- the $parsed variable is properly initialized $parsed = variable_get('javascript_parsed', array());
-- function _locale_invalidate_js also takes care of its local variable $parsed which should always return an array as well
-- the guess I can figure out would be having a corrupted variable javascript_parsed which would cause a wrong initialization

arhak’s picture

Status: Needs work » Needs review

cross-posted with #144

Status: Needs review » Needs work

The last submitted patch, locale-338630-114.patch, failed testing.

arhak’s picture

Status: Needs work » Needs review

[22:45:36] Command [php -l -f ./modules/locale/locale.test] failed with output: No syntax errors detected in ./modules/locale/locale.test.
[22:45:36] Encountered error on [syntax], details:
array (
'@filename' => 'locale.test',
)

Re-test of locale-338630-114.patch from comment #114 was requested by arhak.

plach’s picture

Status: Needs review » Reviewed & tested by the community

Green. Setting back to RTBC.

webchick’s picture

Version: 7.x-dev » 6.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Ok, thanks. Committed to HEAD!

Marking down to 6.x.

sinasalek’s picture

Congratulation everyone :) @arhak @platch

arhak’s picture

Status: Patch (to be ported) » Reviewed & tested by the community

D6 is also RTBC since #95 & #104

EDIT: last D6 patch @#109 http://drupal.org/files/issues/locale-338630-109.D6.patch (which keeps being the same since #89 with insignificant minor changes)

arhak’s picture

@plach: thanks for the effort of being re-rolling these patches during a whole year
@others: thanks for testing and feedback

plach’s picture

Wow, it's been tough. It seems we need Gabor again :)

realOFF’s picture

Congratz! and Thank You!

arhak’s picture

hello?!
a RTBC patch, already committed to HEAD waiting for D6 commit @#123, @#109

webchick’s picture

Status: Reviewed & tested by the community » Patch (to be ported)

arhak:

1. That attitude does absolutely nothing but sap will and energy from people who you need to help you. Stop.

2. We need to keep 7.x and 6.x in sync, so the 7.x patch that was committed needs a back-port, including the minor differences. Rolling that would be a constructive use of your time.

arhak’s picture

@#128
1. by no way I meant a bad attitude :(
2. when did the patches went out of sync? ... oh, you mean D7#114 vs D6#109 ... I apologizes

arhak’s picture

Status: Patch (to be ported) » Needs review

@#128 yet... I just diff them and they are sync

#114 just changed LocaleTranslationFunctionalTest
remaining "minor changes" are synchronized up to #109 as they were committed to HEAD

PS: was the attitude problem around the "hello?!" part?, deeply sorry for that

plach’s picture

Status: Needs review » Reviewed & tested by the community

As arhak said, patches #109 (D6) and #114 (D7) are in-sync. The only differences are due to the string-concatenation standard change and the presence of simpletests in the D7 version.
Moreover the D6 version has been successfully tested multiple times (#110, #112, #114), thus setting back to RTBC.

webchick’s picture

Cool, my mistake. Thought there was a comment change, too.

plach’s picture

@webchick: the comment change was in the simpletests :)

W.M.’s picture

Hello!

So this patch (#114) solves the problem in the same manner as the tiny module posted at #65 ?!

I get all those locale related errors logged in that I want to get rid off...

Thanks for help!

arhak’s picture

@#134
their goals are the same, the approaches are different

patch's pro-cons:
- (pro) the patch works automatically when clearing the cache (the mini-module has specialized buttons for it)
- (pro) the patch detects which are the missing files (the mini-module blindly attempts them all)
- (con) the patch requires to clear the entire cache (the mini-module only works on the JS files)

get all those locale related errors logged in that I want to get rid off...

while the amount of generated logs differs, it depends on the situation which alternative generates more logs, but both generate "parsed" and "created" log entries (since it is in core)

probably you might get advantage of the module log_clear (included in Utility package) which allows you to selectively drop a sub-set of logs (according to those being filtered)
then you could filter by type "locale" and the clear those selected logs

PS: Note that patch @#114 was already committed to D7, its D6 backport is still waiting @#109

nonsie’s picture

+1
Looking forward to having #109 in D6 soon

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Fixed

Thanks, committed to Drupal 6.

sinasalek’s picture

Great! Thanks everyone , Gábor

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

mafi-a’s picture

Version: 6.x-dev » 6.16
Status: Closed (fixed) » Active

I just updated from 6.14 to 6.16 and the described issue is still there...

My workaround from privious versions doesn't work anymore:
I added this hook to locale.module:

function locale_file_download($file) {
  $path = file_create_url(variable_get('locale_js_directory', 'languages'));

  if (strpos(file_create_url($file), $path) === 0) {
    return array('Content-type: application/x-javascript');
  }
} 

Has anyone else problems with the latest core version?

plach’s picture

@mafi-a: Did you clear the cache?

mafi-a’s picture

Status: Active » Fixed

I dug deeper into the code and realized that my problem isn't really with locale but more about the file system

file_create_path(variable_get('locale_js_directory', 'languages'));
returns the wrong path.

I set the issue back to fixed. This should be discussed elsewhere

arhak’s picture

Status: Fixed » Closed (fixed)

actually, it was closed (since it was committed already)

W.M.’s picture

So what's the solution ?!

Gábor Hojtsy’s picture

@Geir19: Upgrade to Drupal 6.16.

Jaapx’s picture

Have the same problem of a lot of file not found entries in wachdog
- D6.15, #145 was not the solution for me, so D6.16 produced the same problem
- Dutch language
- Private files enabled (and working fine)
- Report in wachdog: http://(mydomain)/(directory for private files)/languages/nl_ed3ec6b53945993503faf34345039f3e.js?o
- Must be: http://(mydomain)/system/files/languages/nl_ed3ec6b53945993503faf34345039f3e.js?o

arhak’s picture

it is not intended to work with private download
it will be supported in D7
D6 is code freeze, i.e. no new features in
here you can find a core hack which address your case #572516: make private download method support css/js aggregation, color module and js translations

Jaapx’s picture

Arhak, thank you for your answer. I missed this discussion in my search.

ducdebreme’s picture

Subscribing

Michsk’s picture

I got this error, also for the dutch language, in D6.16. I thought these patches were committed?

arhak’s picture

it was committed, check if you are using private download method (read #147)
or else, almost sure you should open a new issue, since I doubt that the reason might be on this fix
if so, drop here a link for others having similar issues to follow it

Bilmar’s picture

I am also still experiencing this issue. I'm using Drupal 6.16 and English + Japanese languages.
Is this a known issue?

arhak’s picture

please, check whether you're using private download method (read above #147, #151)

Bilmar’s picture

Thanks for the quick reply arhak

I haven't played with any settings but found at www.example.com/admin/settings/file-system
Download method: Public - files are available using HTTP directly.

Is this the setting you are asking? So does this mean it should be working?
I enabled the module at http://drupal.org/node/338630#comment-2035162 but it doesn't seem to help =(

arhak’s picture

@#154 yes, that the setting I was talking about, and it is public therefore it should work
could you please check your directories' structure and permissions
you should find sites/default/files/languages with javascript files in it

- did you cleared the cache?
(this fix is intended to work when clearing the cache, read #135)

- what are the permissions of that directory and its respective files?

- what happens if you rename the languages directory to something else?
lets say "languages2", to simulate a deletion and check if it gets re-created

Bilmar’s picture

Every time I flush the cache I am seeing the warning message on the first page I go to next. When I clear the cache as admin then in a different browser go to any page as a non-admin user (authenticated role) the warning message shows as well.

- the Languages directory at sites/default/files is set to 0777
- some files within are set to 0666, 0664 and 0777
Should I change the languages folder permissions or any files within?

I made a backup of the Languages directory and deleted it. I flushed the cache on my website and got the warning message as before. I checked sites/default/files and there was a new Languages directory. The Files folder is set to 0777, the Languages folder is set to 0777, and the one js file within is set to 0664.

Please let me know if any further information is required to better troubleshoot.
I really appreciate your help!

arhak’s picture

I pretty sure we should be moving this to a new issue and leave its reference here

according to what you describe at #156 this patch is working as expected
- what languages do you have enabled?
- what the language prefix of the newly created JS file? (first letters before the underscore)
- what happens when you visit the site in other languages? (it is expected to appear a warning and also a new JS file to be created for such languages)

once the site have been visited for every language, no more JS file should be created and no more warnings either
if your site isn't being moved, those JS files should live there until you enable new modules causing more JavaScript translation to be imported (in which case, they might be re-created and log message should appear again for each language)

when are the undesirable/unexpected warning arising then?

arhak’s picture

according to #152 your JS file should start with "jp" for Japanese (for English there won't be any translation, since it is the source language)

how many often do you see the warnings?
are those warnings for the same warned files? e.g. misc/drupal.js, misc/jquery.js, misc/collapse.js, etc

if so, then the new issue should be related to "why are those JS files getting lost over and over?"
note that this issue is about to get them back and it is working

also note webchick at #113

What is the reason such a JS file might be lost? Can we give users some direction on what to check so that it doesn't happen again?

arhak’s picture

if the answer isn't mentioned at #115, then it should be investigated to satisfy webchick's question at #113
also note plach at #114

I can't imagine any (non-malicious) reason why those files would disappear in a production site.

EDIT: links

Bilmar’s picture

Hello,

I have Japanese enabled and it's set as the default with no path prefix and English with path prefix /en
Japanese: www.example.com
English: www.example.com/en

The file being created in the sites/default/files/languages folder is ja_........js which looks to be the language code shown at www.example.com/admin/settings/language

I am always seeing the same warning shown below, and it shows on the first page I view right after flushing the cache.

warning: file_get_contents(sites/all/themes/example/js/jquery132min.js) [function.file-get-contents]: failed to open stream: Permission denied in /srv/www/example.com/public_html/includes/locale.inc on line 1708.

line 1708 looks to be the $file line below, which is which a larger function

  // Load the JavaScript file.
  $file = file_get_contents($filepath);

undesirable/unexpected:
It is ok if the admin sees this warning, but authenticated users are also seeing this warning if they happen to view a page before the admin views a page right after the cache is flushed, which makes users of the site think something is wrong with the site =(

I'm sorry for any troubles and really appreciate your help on this.
If there is any other info I can provide please let me know.

Thank you

arhak’s picture

It is ok if the admin sees this warning, but authenticated users are also seeing this warning if they happen to view a page before the admin views a page right after the cache is flushed

this would be another issue
I recall users complaining about it being a security issue unveiling filesystem directory tree
but not addressed here

sites/all/themes/example/js/jquery132min.js
this seems to be the problem, and also unrelated to this issue
it seems you have no "read" access to that file, check whether Apache's user (www-data) has read access to it (if in doubt, give it a 664 to test if the warning goes away)

finally, the conclusion keeps being the same, this issue is working properly (as expected) and therefore, closed
it would be more useful to open new issues for troubleshoting those issues, and if you think it might help others, drop their respective links over here for other to follow up

Bilmar’s picture

Thank you very much arhak for your kind explanation
I will do the tests as you recommended and if it does not work i will open a new issue
Regards

clashar’s picture

subscribe

janhajk’s picture

@Laurent_ #9
Found a solution for your Problem:
http://tech.janschaer.ch/content/drupal-problem-mit-language-files-dejs
Translate with google it's in German

ninebark’s picture

I have Drupal 6.20 which incorporates the fix discussed here. I see this error message on every page on the French side of my site:

The selected file /tmp/fileroEf4K could not be uploaded, because the destination languages/fr_fe5252174f66c5935e23acb7707e47cd.js is not properly configured.

If I delete the .js file or the /languages directory and then clear the cache, neither the file nor the directory is recreated. I tried installing the locale_rebuild_js module, but the check boxes next to both the English and French languages on admin/settings/language/rebuild_js are greyed out, so I cannot manually rebuild them either. Both /files and /files/langauges have permissions 777.

Any ideas?

arhak’s picture

did you tried #65

ninebark’s picture

@arhak - Thank you, but this solution didn't work for me either.

I've had to resort to hiding the error message.

div.messages.error {
	display: none;
}
Loquere’s picture

Version: 6.16 » 7.10
Status: Closed (fixed) » Fixed

I also had this problem. There were thousands of log messages about js language files which can't be created. Also my temp folder was growing really fast.
None of the suggested solutions worked for me. I've tried to solve this issue for days... and finally managed to fix it.
Here is what I did:
In includes/locale.inc file there is _locale_rebuild_js() function. In that function I've changed the line

if (file_unmanaged_save_data($data, $dest)) {
to
if (file_put_contents($dest, $data)) {

And now the new language js file is saved successfully to the languages folder.
The problem was that the file_unmanaged_save_data() function first saves the file to temp folder and then tries to move it to destination folder. And that's what caused this problem: the files were saved with apache ownership and the script couldn't move the file because it was with user ownership.
That might not be a professional solution. I am not Drupal expert. But that solved my problem.

oriol_e9g’s picture

Version: 7.10 » 7.x-dev
Status: Fixed » Needs work
fuerst’s picture

Version: 7.x-dev » 6.16
Status: Needs work » Closed (fixed)

When looking at file_unmanaged_save_data() and related functions (file_unmanaged_save_data(), file_unmanaged_copy() and drupal_chmod()) I can not see anything that will change ownership. I rather suggest you may check the ownership/permissions of your destination directory. Looks like the Apache user can not write there.

BTW: Set issue Version and Status back to 6.16/closed (fixed) because #168 seems not to be related to the issue.