Updated: Comment #11

Problem/Motivation

Exported translation files are not gettext-compatible when containing untranslated strings.

For reproducing it:

  1. Install in a non-English language.
  2. Translate any string.
  3. Export a PO file (checking "Include untranslated strings")
  4. Import the file again, a exception should not be thrown.

Proposed resolution

Empty translations must be between quotes.

Remaining tasks

Add tests.
Quote empty translations when exporting.

User interface changes

None.

API changes

None.

None.

Original report by [username]

When importing a .po file on admin/config/regional/translate/import, once the importation is completed, I get this message:

An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /drupal/batch?id=5&op=do_nojs&op=do
StatusText: OK
ResponseText: Recoverable fatal error: Argument 2 passed to Drupal\Component\Utility\String::format() must be of the type array, null given, called in /Users/outimunter/Sites/drupal/core/lib/Drupal/Component/Gettext/PoStreamReader.php on line 522 and defined in Drupal\Component\Utility\String::format() (line 89 of /Users/outimunter/Sites/drupal/core/lib/Drupal/Component/Utility/String.php).

The translation is anyway imported and I can go ahead by clicking on the error page link.

The steps to produce this are:
- the site is installed in French
- on admin/config/regional/translate/import I import a .po file
- the options about treating the strings and overwriting can be checked or unchecked
- once the importation is completed, the error message appears

Comments

gábor hojtsy’s picture

Title: AJAX Recoverable fatal error while importing a translation on the UI » PoStreamReader argument 2 passed to String::format() must be of the type array, null given error

More specific title. Ajax recoverable errors can be many things.

penyaskito’s picture

Assigned: Unassigned » penyaskito
Issue tags: +D8MI, +sprint
penyaskito’s picture

Outi, I tried with basque and french with the official .po files for core and could not reproduce your issue (drupal-8.0-alpha2.eu.po & drupal-8.0-alpha2.fr.po).
Which po files did you import? could you upload them to the issue?

penyaskito’s picture

Just talked with Outi: she translated some strings in the interface, exported the PO file and then tried to import it again.
Doing the same I can reproduce it.

Uploading exported/imported po file for the reference.

penyaskito’s picture

StatusFileSize
new895.01 KB
penyaskito’s picture

Title: PoStreamReader argument 2 passed to String::format() must be of the type array, null given error » Exporting translations does not correctly generate empty strings

For empty strings that are exported , there are not quotes

msgid "Include customized translations"
msgstr 
msgid "Include untranslated text"
msgstr 
msgid "The strings have been saved."
msgstr 

So when parsing the file we report the error: Recoverable fatal error: Argument 2 passed to Drupal\Component\Utility\String::format() must be of the type array, null given, called in /Users/outimunter/Sites/drupal/core/lib/Drupal/Component/Gettext/PoStreamReader.php on line 522 and defined in Drupal\Component\Utility\String::format() (line 89 of /Users/outimunter/Sites/drupal/core/lib/Drupal/Component/Utility/String.php).

If we run gettext-lint to the exported file, we see that is not valid:

apt-get install gettext-lint
POFileStatus ~/Downloads/fr.po 

returns:

$ POFileStatus ~/Downloads/fr.po 
<po-file-status>
  <file name="/Downloads/fr.po">
    <error>/Downloads/fr.po:23:6: syntax error
/Downloads/fr.po:25:6: syntax error
/Downloads/fr.po:27:6: syntax error
/Downloads/fr.po:29:6: syntax error
/Downloads/fr.po:31:6: syntax error
/Downloads/fr.po:33:6: syntax error
/Downloads/fr.po:35:1: syntax error
msgfmt: found 7 fatal errors
</error>
  </file>
</po-file-status>
penyaskito’s picture

As a confirmation, I checked an exported file with gtranslator (PO editor for gnome), and it crashed. Fixed the file manually and it worked.

penyaskito’s picture

Status: Active » Needs review
StatusFileSize
new2.63 KB
new1.99 KB

Adding test coverage checking that the empty strings are properly quoted if non-translated strings are selected for exporting.
Quoting is done in PoItem itself.

yesct’s picture

+++ b/core/modules/locale/lib/Drupal/locale/Tests/LocaleExportTest.phpundefined
@@ -83,7 +83,7 @@ function testExportTranslation() {
-      ->setString('February')
+      ->setString('March')

Why are you changing from February to March? Is it because the language in the test used to not have a translation for February, and now does... but it still does not have translation for March?

sutharsan’s picture

Status: Needs review » Reviewed & tested by the community

Looks good.

penyaskito’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new1.79 KB
new1.16 KB
new1.31 KB

tl;dr: February is not that bad...

Long version:

While testing this I wanted to test two different cases:
1. Importing a file with an unstranslated string (February)
2. Creating the string in locale storage (March)
but I noticed that in 1), nothing is imported and no string is saved into locale.storage, so it was not really needed.

yesct’s picture

This looks better. :)

I'll try to reproduce.

Status: Needs review » Needs work
Issue tags: -D8MI, -sprint, -language-ui

The last submitted patch, po-export-2029505-10.patch, failed testing.

Anonymous’s picture

Issue summary: View changes

Update summary.

penyaskito’s picture

Issue summary: View changes

core gates

penyaskito’s picture

Status: Needs work » Needs review
penyaskito’s picture

Issue tags: +D8MI, +sprint, +language-ui

#11: po-export-2029505-10.patch queued for re-testing.

gábor hojtsy’s picture

Status: Needs review » Needs work
+++ b/core/modules/locale/lib/Drupal/locale/Tests/LocaleExportTest.phpundefined
@@ -168,4 +168,11 @@ function getCustomPoFile() {
 
+  function getEmptyPoTranslation() {
+    return <<< EOF
+msgid "February"
+msgstr ""
+EOF;
+  }

This lacks code comments as to what it is used for.

outi’s picture

Status: Needs work » Needs review

I tested the export and import on a French and an English installation and it seems to be ok now.

outi’s picture

I tested the export and the import on a French and on an English installation, and it seems to work now.

penyaskito’s picture

Renamed the function to getUntranslatedString for readability and added docblock.

gábor hojtsy’s picture

Status: Needs review » Reviewed & tested by the community

Looks all good now then :)

yesct’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new1.95 KB
new1.31 KB
new678 bytes
+++ b/core/modules/locale/lib/Drupal/locale/Tests/LocaleExportTest.phpundefined
@@ -168,4 +168,14 @@ function getCustomPoFile() {
+   * Helper function that returns a .po fragment file with an unstranslated string.

unstranslated
should be untranslated.

Also put it below 80 chars.. and added the @return.

penyaskito’s picture

Status: Needs review » Reviewed & tested by the community

It was already RTBC by Gábor, and YesCT improved docs.

yesct’s picture

Issue tags: +RTBC July 1

This issue was RTBC and passing tests on July 1, the beginning of API freeze.

catch’s picture

Status: Reviewed & tested by the community » Needs work
-    $output .= 'msgstr '. (isset($this->_translation) ? $this->formatString($this->_translation) : '');
+    $output .= 'msgstr '. (isset($this->_translation) ? $this->formatString($this->_translation) : '""');

While we're here let's make the string concatenation follow coding standards.

yesct’s picture

Issue tags: +Novice

Here is the reference to check:
https://drupal.org/coding-standards#concat

penyaskito’s picture

Assigned: penyaskito » Unassigned

I'm not working on this at the moment, so any help is appreciated.

gábor hojtsy’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new726 bytes
new2.02 KB

Included the whitespace fix that @catch requested. Also I checked the plural formatting as well, and it covers empty strings proper, so no such bug evident there.

    foreach ($this->_translation as $i => $trans) {
      if (isset($this->_translation[$i])) {
        $output .= 'msgstr[' . $i . '] ' . $this->formatString($trans);
      }
      else {
        $output .= 'msgstr[' . $i . '] ""' . "\n";
      }
    }

Back to RTBC as per above given the minor whitespace change only.

yesct’s picture

Issue tags: -Novice

yep. looks good.

gábor hojtsy’s picture

#27: po-export-2029505-27.patch queued for re-testing.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed b2b2a44 and pushed to 8.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

gábor hojtsy’s picture

Issue tags: -sprint

Remove sprint tag.

gábor hojtsy’s picture

Issue summary: View changes

Summary update