Hi,

I am trying to instal Drupal 7 RC1. I have downloaded Czech translation of Drupal Core (drupal-7.0-rc1.cs.po) from http://ftp.drupal.org/files/translations/7.x/drupal/drupal-7.0-rc1.cs.po I get the following error during installation

Notice: Undefined index: drupal-70-rc1cs in locale_add_language() (line 459 of /example/includes/locale.inc).
Notice: Undefined index: drupal-70-rc1cs in locale_add_language() (line 460 of /example/includes/locale.inc).
SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column ‚language‘ at row 1

It prevents me from finishing the installation process.

If I choose default language English (built-in) everything works fine and I am able to finish the installation.

I thought that somebody should know that :)

Database system MySQL
Database system version - 5.1.53-0.dotdeb.0-log
PHP - 5.3.3-0.dotdeb.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

meba’s picture

Title: Drupal 7 RC1, localization » Data too long for column "language" during installation
Component: database system » language system

You did not rename drupal-7.0-rc1.cs.po to cs.po in profiles/standard/translations as written in README, that's what is causing the issue.

I would almost say it's a non-issue, doesn't require fixing. Gabor?

Gábor Hojtsy’s picture

Status: Active » Closed (works as designed)

Yes, looks like you did follow only part of the instructions.

plach’s picture

Component: language system » locale.module

@Gábor Hojtsy:

Do you think it is possible to have a more user-friendly failure here?

See also: #988262: Could not complete installation in French.

Gábor Hojtsy’s picture

@plach: Yes, the docs were introduced with #882164: Adding a language help page during installation is incorrect and I said there that it would indeed be best to modify the pattern handler instead to import files with arbitrary prefixes and still identify the proper language. It is definitely possible, but there was not much support for that. Support and help is welcome to make it work better :)

plach’s picture

I guess it's too late for introducing a simple error message saying the file pattern is not recognized, right?

Gábor Hojtsy’s picture

I still think we should fix the behavior (it will not go against string freeze :), but people were strong to document how it works instead. We should just fix how it works. Drupal elsewhere will recognize anythingyouwant.$langcode.po just not in the install profile, really. The fact this made a bit harder in the installer is that it send around the filename as the translation language code, so we'd need to pass both names around somehow.

MPeli’s picture

Thank you for your respose. What README file do you exactly mean?

Content of README file (profiles/standard/translations):

// $Id: README.txt,v 1.1 2010/09/22 01:49:17 webchick Exp $

This directory should be used to place downloaded translations
for installing Drupal core.

Gábor Hojtsy’s picture

@MPeli: unfortunately @meba said README incorrectly. The INSTALL.txt has installation instructions; here is a relevant quote:

2. Optionally, download a translation.

   By default, Drupal is installed in English, and further languages may be
   installed later. If you prefer to install Drupal in another language
   initially:

   - Download a translation file for the correct Drupal version and language
     from the translation server: http://localize.drupal.org/download

   - Rename the downloaded file to your language's ISO code (e.g., de.po or
     fr.po) and place it into your installation profile's translations
     directory. For instance, if you are using the Standard install profile,
     move the .po file into the directory:

       profiles/standard/translations/

   For detailed instructions, visit http://drupal.org/localize
MPeli’s picture

@Gábor Hojtsy I should have read the instructions. Appreciate your help. Martin.

plach’s picture

Gábor Hojtsy’s picture

Status: Closed (works as designed) » Needs work
FileSize
1.93 KB

I think this patch solves this particular problem. Not sure its good though in the light of #1007488: Drupal cannot be installed in a non-predefined language which is a different error, and highlights we need to identify the language even earlier (so we can eg. display a meaningful name in the radio button group for it). This attached patch will avoid the SQL error reported in *this* issue, but will still look odd in the language radio group (it will still display the filename instead of a nice language name). Also totally untested, but trying to bootstrap this since I only have little time. Let me look at another possible approach and post that too.

Gábor Hojtsy’s picture

Title: Data too long for column "language" during installation » Drupal should not require .po file rename in installation
Priority: Minor » Major
Status: Needs work » Needs review
FileSize
5.76 KB

Ok, I validated the attached patch by creating the following files in my profiles/standard/translations:

cs.po
drupal-7.0.cs.po
drupal-7.0.de.po
hu.po
drupal-7.0.hk.po

cs.po and hu.po were obviously identified. The others were not, but offered as distinct languages. I did the following in this (much better approached) patch to make the others work:

1. Added a langcode key to the .po file names found in the installer, which only contains the langcode.
2. Checked the locale we got against that langcode.
3. When listing the language names for selection, used the langcode to look up items from the predefined list and used the langcode to key the items, so we carry on the langcode NOT the filename (this is 100% as intended by Drupal 6 and 7). Now as demonstrated, we can have multiple files with the same language suffix (cs.po and drupal-7.0.cs.po above). So keying by language code also gets us rid of duplicates.
4. On the locale UI .po import, we need to import all the files for that language though, so I put in a file_scan_directory(). Note that if multiple files provide the same translations, later found ones will overwrite the previous ones. We really cannot do much about this (without actually letting the user select from filenames, which could expose multiple selections for the same language and can be very confusing).
5. Found hopefully all places where the required rename was documented and removed those.

Now my D7 shows Czech, German, Hungarian and hk. hk is an unknown language to Drupal and I can reproduce the cross-linked #1007488: Drupal cannot be installed in a non-predefined language with that (but that is a different issue, not to be solved here, it appears with just using hk.po only, not related to how the file name is structured).

Note that I'll not have much time to work on this, so hope someone (plach? :)) will be able to pick this up. Thanks! We should drive this in before Drupal 7.0. Too many people got burned by the file rename requirement as per the issue queue.

meba’s picture

Status: Needs review » Reviewed & tested by the community

OK, tested and looks good. What I did:

- Tried drupal-7.0-rc1.cs.po and drupal-7.0-rc1.hu.po with unpatched Drupal. Result: Doesn't recognize the language.
- Patched and tried both files, language was recognized and installation finished.
- Tried copying both to cs.po and hu.po (keeping the complex files), language was recognized (no duplicates) and installation finished.

Issue summary

Drupal install system requires users to rename downloaded language files ("drupal-7.0.cs.po") to {langcode}.po, which is very confusing and there are number of reports by people who forgot to do that. Result is an SQL error. Gábor's patch resolves this issue by parsing filenames correctly. Drupal then offers users the correct language, no matter what is the file name ({something}.{langcode}.po).

RTBC if testbot agrees...

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Well fixing this total WTF definitely seems worth breaking string freeze for.

Committed to HEAD. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

David_Rothstein’s picture

Version: 7.0-rc1 » 7.x-dev
Priority: Major » Normal
Status: Closed (fixed) » Needs review
FileSize
1.18 KB

As far as I can tell, the committed patch doesn't work in the case where the profile selects the language (without user intervention), or other similar scenarios.

I think we need the attached follow-up.

Gábor Hojtsy’s picture

Well, Drupal would still work with the committed patch for preselected languages if the preselected language has the simple language code based filename. Not if it has other file name components. For that (supporting of which was the whole point of the above patch), your attached patch is required. So looks like your suggestion is a good application of the changes to the rest of the install system.

I did not try it, so not feeling eligible to mark it RTBC.

Gábor Hojtsy’s picture

Version: 7.x-dev » 8.x-dev

It still looks good :)

kscheirer’s picture

#16: language-po-rename-994500-16.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, language-po-rename-994500-16.patch, failed testing.

Sutharsan’s picture

Version: 8.x-dev » 7.x-dev
Status: Needs work » Needs review

This issue no longer applies to Drupal 8. In Drupal 8 only language codes are used where in D7 the language name was used. For anyone else who want to get to the bottom of this in D8: D7's install_select_locale() is converted to D8's install_select_language(). D7 install_find_locales() is converted into D8 install_find_translations().

Putting the version back to 7.x in case we want to apply the change there.

izodor’s picture

11: language-flexible.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, 11: language-flexible.patch, failed testing.

  • webchick committed 52ba5cc on 8.3.x
    #994500 by Gábor Hojtsy: Fixed Drupal should not require .po file rename...

  • webchick committed 52ba5cc on 8.3.x
    #994500 by Gábor Hojtsy: Fixed Drupal should not require .po file rename...

  • webchick committed 52ba5cc on 8.4.x
    #994500 by Gábor Hojtsy: Fixed Drupal should not require .po file rename...

  • webchick committed 52ba5cc on 8.4.x
    #994500 by Gábor Hojtsy: Fixed Drupal should not require .po file rename...

  • webchick committed 52ba5cc on 9.1.x
    #994500 by Gábor Hojtsy: Fixed Drupal should not require .po file rename...