Problem/Motivation

Plurals are not exported correctly when exporting source translations.

STR:

  1. Install the "Interface Translation" (locale) module.
  2. Go to /admin/config/regional/language and add a language. I chose Spanish.
  3. Go to /admin/config/regional/translate/export
    • Choose "source text only, no translations"
    • Click export
  4. Open PO file, you'll find:
msgid "@count contextual link"
msgid_plural "@count contextual links"
msgstr[0] ""

As the source language is English and the header says "Plural-Forms: nplurals=2; plural=(n > 1);, it should be:

msgid "@count contextual link"
msgid_plural "@count contextual links"
msgstr[0] ""
msgstr[1] "" <<<<<<<<

Reported originally by Dale Eggett @ Lingotek

Proposed resolution

Investigate

Remaining tasks

Investigate. Create patch.

User interface changes

None

API changes

None

Data model changes

None

Issue fork drupal-2918489

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

penyaskito created an issue. See original summary.

penyaskito’s picture

Maybe related? #2882617: String version of plural formula is not available, exported .po files contain an incorrect default.

I'm not sure if the PoDatabaseReader::loadStrings should be modified to create as many translations in the array as needed, or if PoItem::formatPlural should take care of this.

None of them have access to the plural formula AFAIK.

Feedback appreciated!

gábor hojtsy’s picture

Issue tags: +D8MI, +language-ui

Hm, are translations properly exported when you choose that method (not the template)?

penyaskito’s picture

If they are untranslated the same issue occurs.

If they are translated (e.g. translated them to German -1 plural-), it works as expected.

Interestingly, I tried with Polish (2 plurals) and Slovenian (3 plurals). I translated only some strings (1/2 plural for Polish), (2/3 plurals for Slovenian) and it worked well.

Created a new clean site again, and started over. Without any translations it doesn't work well for Slovenian or Polish.
Translated only 2nd plural for Slovenian, works well.
Translated only singular for Polish, works well.

So the problem is only when there is no translations at all for this string.

penyaskito’s picture

The reason that behavior happens is that saving the translations in the form saves the empty translations:

      $new_translation_string_delimited = implode(LOCALE_PLURAL_DELIMITER, $new_translation['translations']);

in \Drupal\locale\Form\TranslateEditForm::submitForm

gábor hojtsy’s picture

Not clear if this is broken only for saved translations or stuff that was never yet touched too? The code cited from the save seems like it would add the needed amount of delimiters(?)

mglaman’s picture

I reported this to LingoTek. It was an export of untranslated strings I wanted to upload so we could get a translation of strings in our Twig templates.

penyaskito’s picture

Status: Active » Needs review
StatusFileSize
new2.97 KB

It's broken for stuff that was never yet touched, as the form submit handler fixes it for us by saving the space delimiter the right amount of times.

Here I worked on a fix. Depending on the locale.plural.formula service in PoItem won't be acceptable here I guess, but as far of my manual tests have gone this works for the case at hand.
Any ideas of how to make this without depending on the service are much than appreciated.

penyaskito’s picture

Crossposted with both of you, sorry for the bad patch naming!

I confirm @mglaman originally discovered the issue I reported. Hoping we can give him attribution once this is solved.

mglaman’s picture

StatusFileSize
new118.92 KB

Manually tested the patch and can confirm it fixes the issue.

Before patch

msgid "Host"
msgid_plural "Hosts"
msgstr[0] ""

After patch

msgid "Host"
msgid_plural "Hosts"
msgstr[0] ""
msgstr[1] ""

The following UI screen for export. And I was able to upload to LingoTek without errors in parsing the .po

Items which have been translated, as stated, do not have this problem.

penyaskito’s picture

@Gábor do you think it would be acceptable the conditional check for the locale module?

penyaskito’s picture

StatusFileSize
new1.63 KB
new4.6 KB

Added a test for exporting the POT template with a plural string.

The last submitted patch, 12: 2918489-locale-plurals-12.only-tests.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mglaman’s picture

Status: Needs review » Reviewed & tested by the community

Now there are tests, and this fixes the bug. +1 and RTBC from me.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. Sorry for the following comments - it's one of the tricky things about the component space and the fact that they are supposed to be decoupled from \Drupal\Core\*.
  2. +++ b/core/lib/Drupal/Component/Gettext/PoItem.php
    @@ -70,6 +77,13 @@ public function getLangcode() {
    +    if (\Drupal::moduleHandler()->moduleExists('locale')) {
    +      $this->_nPlurals = \Drupal::service('locale.plural.formula')->getNumberOfPlurals($langcode);
    +    }
    

    Hmmm... a component probably should not be reaching out into \Drupal::service land. I think we should add a param to this method to set the number of plurals. Or add a new method to set it. Tricky.

  3. +++ b/core/lib/Drupal/Component/Gettext/PoItem.php
    @@ -187,7 +201,12 @@ public function setFromArray(array $values = []) {
    +        $this->setTranslation(explode(LOCALE_PLURAL_DELIMITER, $this->_translation));
    

    We need a followup to decouple this component from LOCALE_PLURAL_DELIMITER - or to move the definition of this to the component. The dependencies are wrong way around.

  4. +++ b/core/lib/Drupal/Component/Gettext/PoStreamReader.php
    @@ -526,12 +526,18 @@ public function setItemFromArray($value) {
    +    if (isset($value['msgstr'])) {
    +      $item->setTranslation($value['msgstr']);
    +    }
    +    else {
    +      $nPlurals = \Drupal::service('locale.plural.formula')->getNumberOfPlurals($this->_langcode);
    +      $this->setTranslation(array_fill(0, $nPlurals, NULL));
    +    }
    

    As above - calling out to \Drupal::service is not great here.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

andypost’s picture

Assigned: Unassigned » andypost
StatusFileSize
new4.66 KB

It makes sense to check all string/array flip/flops and decide about home for delimiter definition
Faced in #1637662-14: Clean up the Gettext PO parser

Surely component's space should not use services and common.inc constant

PS: reroll

andypost’s picture

Assigned: andypost » Unassigned

Fix for 15.3 in #2911606: Replace usage of the deprecated LOCALE_PLURAL_DELIMITER constant so this issue partially blocked on it

It looks totally wrong that PoItem should know language and number of plurals - it is used to store strings only and language mostly unused

I think it needs deeper refactor to store string & translation as arrays and implode/explode them where it actually expected.

Core using delimiter in following places
- .po files reader/writer - conditionally flipping source and translation depending on number of plurals - #2918489: Plurals are not exported correctly when exporting source translations
- storing plurals in config - also using flip/flop on export and to create translations- #2545730: Misuse of formatPlural() in Numeric field prefix/suffix
- UI elements and controls to edit translations

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

penyaskito’s picture

Closed #2882667: [PP-1] Interface translations export contains incorrect plural formula in favor of this one even if that's older, as this one has a patch and some more discussion. Please credit @idebr here.

penyaskito’s picture

Status: Needs work » Needs review
StatusFileSize
new4.58 KB

Re-rolled this by hand, not sure why those tests failed in #19 but they are working for me locally.
Fixed underscored variables, did that for consistency but that was already fixed everywhere.

penyaskito’s picture

Issue tags: +Global2020

Tagging for the DrupalCon Global 2020 sprint

hedrickbt’s picture

Testing the patch during DrupalCon Global.

penyaskito’s picture

RE: #19: I agree that PoItem shouldn't know about language, but I'm not that sure about plurals number. A PoItem must be responsible of knowing the number of plurals (msgstr) to generate even when the translations are not set.
It should get that value injected somehow and not obtain it by itself, but I don't think that logic should be moved anywhere else.

hedrickbt’s picture

Issue summary: View changes
hedrickbt’s picture

Issue summary: View changes
hedrickbt’s picture

Issue summary: View changes
hedrickbt’s picture

Issue summary: View changes
hedrickbt’s picture

Issue summary: View changes
hedrickbt’s picture

I have tested #24 via a local ddev environment using the 9.1.x branch. I am also seeing the additional 'msgstr[1] ""' items for plurals. There were no other changes to the pot file - except the POT-Creation-Date and PO-Revision-Date lines, which I would expect.

Not sure if this should be moved to RTBC due to the additional discussion.

penyaskito’s picture

StatusFileSize
new8.53 KB
new10.32 KB

I think this solves everything in #15, but I would love to hear if anyone has a better idea than mine.

This patch adds a PoPluralCountAwareInterface, and makes PoStreamReader, PoStreamWriter and PoItem implement it. I would have added those methods to their interfaces, but that would make a BC break.
So the responsibility of looking at the plural count goes to \Drupal\locale\Form\ExportForm, which can depend on that service without adding the dependency from the component to the locale module.

But in this way we are moving that responsibility to any callers, which means that e.g. \Drush\Drupal\Commands\core\LocaleCommands would need to find the plurals count too and inject them. I guess potx module will need to do the same. So I would love to hear if there's anything better than this, but I think this needs to be good enough for the component sake.

penyaskito’s picture

Thanks @hedrickbt for the updates to the issue summary, fixing my vague steps to reproduce and testing the patch, and sorry we crossposted :-/

Would you like to test the new patch and ensure we are not breaking anything else? Thanks again!

penyaskito’s picture

StatusFileSize
new1.27 KB
new10.32 KB

Fixing the parameter name and other coding standards complaints.

andypost’s picture

StatusFileSize
new3.04 KB
new10.34 KB

Fix interface docblock and added types to new interface

samiullah’s picture

StatusFileSize
new13.62 KB

@andypost I tried to manually apply the patch on 9.1x. After site refresh I am getting website encountered error.

samiullah’s picture

Dblog showed the php error:

andypost’s picture

It's strange, it works for me

gábor hojtsy’s picture

Crediting @idebr as per #23.

penyaskito’s picture

Status: Needs review » Reviewed & tested by the community

Reviewed #37 patch and the interface is there, and in the correct place. The name and filename have the same upper case/lower case (I was wondering if that could be an error only affecting MAMP or alike).

Tested #37 with simplytest.me. Patch applied correctly. No errors navigating the site.
Did the steps to reproduce from the issue summary and the result PO file was the expected one.

The only thing I found is, for Russian, the @count[2] param which I'm not sure if it's correct:

msgid "@count item, skip @skip"
msgid_plural "@count items, skip @skip"
msgstr[0] "@count элемент, пропустить @skip"
msgstr[1] "@count элемента, пропустить @skip"
msgstr[2] "@count[2] элементов, пропустить @skip"

But I don't think we are introducing that here.

penyaskito’s picture

if I go to https://localize.drupal.org/translate/languages/ru/translate?project=dru... I see that the translations are saved as that, so it looks it's ok

andypost’s picture

In related it was fixed to handle this suffixes

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/locale/src/PoDatabaseReader.php
    @@ -34,6 +34,13 @@ class PoDatabaseReader implements PoReaderInterface {
    +  private $pluralCount = 2;
    

    Why private?

  2. +++ b/core/modules/locale/src/PoDatabaseReader.php
    @@ -62,6 +69,20 @@ public function setLangcode($langcode) {
    +  /**
    +   * {@inheritdoc}
    +   */
    +  public function getPluralCount() {
    +    return $this->pluralCount;
    +  }
    +
    +  /**
    +   * {@inheritdoc}
    +   */
    +  public function setPluralCount($pluralCount) {
    +    $this->pluralCount = $pluralCount;
    +  }
    +
    

    This are not inheriting anything.

  3. PoDatabaseWriter is not implementing these methods. Should the writer and reader interfaces extend the new PoPluralCountAwareInterface? That feels disruptive. The only thing in cotrib I found extending either interface is http://codcontrib.hank.vps-private.net/search?text=PoReaderInterface&fil... - I guess that should change to.
penyaskito’s picture

RE: 1. Why private?

For consistency. The other properties there are already private. If we want to make them protected, we could do that, but IMHO it's better having consistency inside the same class.

RE: 2,3: PoDatabaseWriter is not implementing these methods. Should the writer and reader interfaces extend the new PoPluralCountAwareInterface? That feels disruptive.

I would totally do that, or even not having a separate interface, but didn't want to do a BC break. If you think that's low impact and acceptable, I would do it.
Or should we just change the {@inheritdoc} there?

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

gábor hojtsy’s picture

I am not surprised that there is not wider impact. IMHO we can coordinate with locale_translation_context to fix on their side as well.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

hmdnawaz’s picture

StatusFileSize
new9.63 KB

Rerolled the patch 37.

hmdnawaz’s picture

StatusFileSize
new10.41 KB
gábor hojtsy’s picture

Issue tags: +ddd2022

While #2882667: [PP-1] Interface translations export contains incorrect plural formula was closed as a duplicate, this does not deal with the export of the plural formula itself in the .po file. Which is also still broken. I think that would be fine to be in scope for #2882617: String version of plural formula is not available, exported .po files contain an incorrect default though.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

robincs’s picture

StatusFileSize
new10.31 KB
new3.76 KB

Recap from DrupalCon Lille 2023:

The functionallity of the latest patch #37 looks fine. If the reroll of the patch passes, we put this back to Needs Review.

andypost’s picture

Moved last patch to MR and fixed following to make the test case to pass

- https://www.drupal.org/node/3168858
- https://www.drupal.org/node/3129738

andypost’s picture

NW to add

  1. +++ b/core/lib/Drupal/Component/Gettext/PoItem.php
    @@ -49,6 +49,13 @@ class PoItem {
    +   * @var int
    ...
    +  protected $pluralCount;
    
    +++ b/core/lib/Drupal/Component/Gettext/PoStreamReader.php
    @@ -62,6 +62,13 @@ class PoStreamReader implements PoStreamInterface, PoReaderInterface {
    +   * @var int
    ...
    +  protected $pluralCount = 2;
    
    +++ b/core/lib/Drupal/Component/Gettext/PoStreamWriter.php
    @@ -35,6 +35,13 @@ class PoStreamWriter implements PoWriterInterface, PoStreamInterface {
    +   * @var int
    ...
    +  protected $pluralCount = 2;
    
    +++ b/core/modules/locale/src/PoDatabaseReader.php
    @@ -34,6 +34,13 @@ class PoDatabaseReader implements PoReaderInterface {
    +   * @var int
    ...
    +  private $pluralCount = 2;
    

    should be typed to int, so default value will be 0

  2. +++ b/core/lib/Drupal/Component/Gettext/PoPluralCountAwareInterface.php
    @@ -0,0 +1,28 @@
    +   * @return self
    ...
    +  public function setPluralCount(int $pluralCount);
    
    +++ b/core/lib/Drupal/Component/Gettext/PoStreamReader.php
    @@ -126,6 +133,21 @@ public function getHeader() {
    +  public function setPluralCount(int $pluralCount) {
    
    +++ b/core/lib/Drupal/Component/Gettext/PoStreamWriter.php
    @@ -75,6 +82,21 @@ public function setLangcode($langcode) {
    +  public function setPluralCount(int $pluralCount) {
    

    Probably should return :self as of PHP 8.0 https://php.watch/versions/8.0/static-return-type

hmdnawaz’s picture

Patch for Drupal 10.3

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.