Import interface translation files in install time

OK, installer states are in, so here is a patch to introduce language importing functionality to the installer. The rationale here for core is that people will be able to use Drupal in their language from the start. We search translations files all around the enabled modules and themes, and the idea is that core translations packaged for Drupal 6 will not have a monolithic PO file, but smaller files distributed around the module and theme folders. The very good result is that we only import what we need (what's enabled), so the site will be more performant, unused strings will not pollute the database! Not to speak about the fantastic user experience.

We have experience with such functionality with Drupal 5, because I developed the autolocale module and localized install profile with Jakub Suchy, which does the above. Our problem was that there was no batch API to use, so the import was timing out. Now Drupal 6 includes a batch API, so we can run imports in multiple progressive HTTP requests. It also looks very cool on the interface (see the video on the installer states issue referenced above)!

After thinking hard about where we should put importing, it turned out that installer states need some adjustments to accommodate locale imports, and this might make the code a bit more readable too (installer states are more visible in the code). The dilemma was about where to put the locale importing. The most simple would be *before* the configure form (and thus before the profile defined forms). That would leave the state machine already in core nearly untouched. Unfortunately it does not work generally because profiles will be able to turn on/off modules and themes with their forms, and we would need a complete language import *after* the profile is done with all adjustments. So we need to go to the profile states after the configure form and then to the locale import and the finished page.

Unfortunately this has the consequence that the profile cannot choose between going to the stock finished page or printing a custom finished page anymore. Sure, we can add a new callback for profiles to be able to generate a finished page, or return a finished message. That is not there yet (needs discussion here), so this feature is temporarily removed for now.

Included in this patch (in order of appearance in the patch):

INSTALLER CHANGES:

- Now profile PO files are searched under a /po/ subfolder in the profile, to be consistent with modules and themes.
- A reserved task list is defined and the profile final function is only called when none of those is identified (now profile tasks are added between the installer defined screens, and the exact state route of the installer changes depending on what is available/required, so we can only see that if an unkown task is identified, that is for the profile to handle).
- An importlocale task is added which only fires anything if a foreign language is handled. If we find .po files to import, then a batch is set and is run (which results in multiple HTTP requests hit with the "importbatch" task). If no .po files are found for enabled modules/themes, the finished page is shown right away.
- As mentioned above, the finished task is now mandatory and it is not possible to add in a custom message for the profile. We can add this back with a custom callback, or by invoking profile_final() with the 'finished' task :)

LOCALE.INC CHANGES:

- The locale_add_language() function was revorked to really serve as an API function (do not emit messages to the page, allow for setting the language enabled and default, allow for adding a language from the predefined list). It now also properly handles the language count.
- RTL languages got the LANGUAGE_RTL constant for their direction value
- New functions are added to support the running of the batch. locale_batch_installer() and _locale_batch_build() are separated because we will also need runtime batches later (when people enable modules or themes we need to import translations, but that is for a later patch). Here we have callbacks for the batch operations as well as the finished callback, which moves on to the next task in the installer.

Note that language files are searched in modules and themes, in their "po/" directories. The standard $langcode.po files are searched as well as anything ending in .$langcode.po (note the dot before the langcode). This allows for segmentation of translation work for big modules, so the translators are more comfortable with translating smaller files. Files in the Hungarian sample package I will upload next are named like node-module.hu.po and so on.

TODO:
- discuss how the finished page should be handled, if profiles would like to customize/replace it
- update the packaging scripts to package Drupal 6 core translations in the new format (this is for me, Google pays for me to do this, so it will be done :) note that contrib translations are already packaged fine for this to work (but will get improved packaging anyway :)

Reviews please!

Here is data to test the above patch:

- Update to latest Drupal 6-dev.
- Apply the patch above.
- Uncompress the attached zip to the *same* folder as your Drupal root.

Hungarian translation files for the default profile as well as modules will fall into place, and you will be able to use the Hungarian language to install and use Drupal right away. (Note: this is a Drupal 5.1 translation, so the translation is understandably not complete neither for the installer nor Drupal 6-dev itself).

Please test/review!

Reviewed the patch with Dries. Includes the following modifications:

- updated to latest Drupal 6.x-dev
- renamed 'locale' and 'profile' tasks to 'locale-select' and 'profile-select', so we can have other 'locale-...' tasks ('locale-import' and 'locale-batch' are the ones included)
- renamed 'profiletasks' to 'profile', which is the entry task to the custom profile tasks
- upon Dries' request, added a screen to inform users that translations of the interface might be available; this screen is only printed if only one language is available and the default profile is used (in which case it is likely that the user has no clue that the installer might be capable of importing the translations right away)

Tested with latest Drupal 6.x-dev (including FAPI3 changes) and it works fine. You can test with the Hungarian and Czech translation packs provided.

Will post an update to INSTALL.txt, including the configure form changes, once this gets committed.

Yes, the changelog got modified. Rerolled patch against latest Drupal 6.x-dev (but no code changes). Seems like this is ready to go in.

Dries, I cannot reproduce (2). Done a fresh CVS head checkout of Drupal, applied the patch, extracted the CZ package, run the installer, and all went fine, locale module enabled, strings are in place both in the installer and on the initial Drupal screen.

Added a documentation issue here to update the install.txt (it contains two other related updates, so this is why I have the separate issue): http://drupal.org/node/144256

I don't know why this was set CNW, it seems it is fixed.