Problem/Motivation

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Problem/Motivation

In #2945351: Write test coverage for the integration with content moderation module we created a basic test coverage dealing with the most common scenarios. However, all of them were covering only one-language scenarios.

Proposed resolution

Create more tests that cover also multilingual use cases.

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#141 paragraphs_2951436_moderation_139.patch135.58 KBmiro_dietiker
#139 paragraphs_2951436_moderation_139.patch135.58 KBmiro_dietiker
#129 interdiff-2951436-127-129.txt7.96 KBjohnchque
#129 paragraphs-paragraphs-translatable-revision-2951436-129.patch137.1 KBjohnchque
#127 paragraphs-paragraphs-translatable-revision-2951436-127-interdiff.txt7.45 KBBerdir
#127 paragraphs-paragraphs-translatable-revision-2951436-127.patch134.35 KBBerdir
#124 interdiff-2951436-123-124.txt640 bytesjohnchque
#124 paragraphs-paragraphs-translatable-revision-2951436-124.patch130.78 KBjohnchque
#123 interdiff-2951436-119-123.txt8.56 KBjohnchque
#123 paragraphs-paragraphs-translatable-revision-2951436-123.patch130.78 KBjohnchque
#121 paragraphs-paragraphs-translatable-revision-2951436-119-interdiff.txt1.96 KBBerdir
#121 paragraphs-paragraphs-translatable-revision-2951436-119.patch127.74 KBBerdir
#119 err-forward-translation-revision-2961399-69-interdiff.txt2.66 KBBerdir
#119 err-forward-translation-revision-2961399-69.patch29.15 KBBerdir
#116 interdiff-2951436-114-116.txt17.02 KBjohnchque
#116 paragraphs-paragraphs-translatable-revision-2951436-116.patch127.58 KBjohnchque
#114 interdiff-2951436-112-114.txt2.99 KBjohnchque
#114 paragraphs-paragraphs-translatable-revision-2951436-114.patch116.7 KBjohnchque
#112 interdiff-2951436-110-112.txt9.55 KBjohnchque
#112 paragraphs-paragraphs-translatable-revision-2951436-112.patch114.06 KBjohnchque
#110 interdiff-2951436-108-110.txt15.59 KBjohnchque
#110 paragraphs-paragraphs-translatable-revision-2951436-110.patch104.73 KBjohnchque
#108 interdiff-2951436-106-108.txt23.83 KBjohnchque
#108 paragraphs-paragraphs-translatable-revision-2951436-108.patch89.15 KBjohnchque
#106 interdiff-2951436-104-106.txt367 bytesjohnchque
#106 paragraphs-paragraphs-translatable-revision-2951436-106.patch64.91 KBjohnchque
#104 paragraphs-paragraphs-translatable-revision-2951436-104-interdiff.txt9.14 KBBerdir
#104 paragraphs-paragraphs-translatable-revision-2951436-104.patch64.97 KBBerdir
#101 interdiff-2951436-100-101.txt4.47 KBjohnchque
#101 paragraphs-paragraphs-translatable-revision-2951436-101.patch57.33 KBjohnchque
#100 interdiff-2951436-98-100.txt2.22 KBjohnchque
#100 paragraphs-paragraphs-translatable-revision-2951436-100.patch53.88 KBjohnchque
#98 interdiff-2951436-96-98.txt2.55 KBjohnchque
#98 paragraphs-paragraphs-translatable-revision-2951436-98.patch53.68 KBjohnchque
#96 interdiff-2951436-94-96.txt3.5 KBjohnchque
#96 paragraphs-paragraphs-translatable-revision-2951436-96.patch54.16 KBjohnchque
#94 paragraphs-paragraphs-translatable-revision-2951436-94-interdiff.txt1.49 KBmaximpodorov
#94 paragraphs-paragraphs-translatable-revision-2951436-94.patch52.97 KBmaximpodorov
#71 paragraphs-paragraphs-translatable-revision-2951436-71-interdiff.txt1.52 KBBerdir
#71 paragraphs-paragraphs-translatable-revision-2951436-71.patch54.66 KBBerdir
#69 paragraphs-paragraphs-translatable-revision-2951436-69-interdiff.txt1.21 KBBerdir
#69 paragraphs-paragraphs-translatable-revision-2951436-69.patch54.66 KBBerdir
#55 paragraphs-paragraphs-translatable-revision-2951436-55-interdiff.txt3.39 KBBerdir
#55 paragraphs-paragraphs-translatable-revision-2951436-55.patch54.35 KBBerdir
#54 paragraphs-paragraphs-translatable-revision-2951436-53-interdiff.txt3.86 KBBerdir
#54 paragraphs-paragraphs-translatable-revision-2951436-53.patch54.04 KBBerdir
#53 paragraphs-paragraphs-translatable-revision-2951436-52.patch51.56 KBBerdir
#52 paragraphs-paragraphs-translatable-revision-2951436-52-interdiff.txt7.5 KBBerdir
#51 paragraphs-paragraphs-translatable-revision-2951436-51-interdiff.txt1.9 KBBerdir
#51 paragraphs-paragraphs-translatable-revision-2951436-51.patch34.99 KBBerdir
#50 paragraphs-paragraphs-translatable-revision-2951436-50-interdiff.txt1.67 KBBerdir
#50 paragraphs-paragraphs-translatable-revision-2951436-50.patch34.78 KBBerdir
#48 paragraphs-paragraphs-translatable-revision-2951436-46-interdiff.txt720 bytesBerdir
#48 paragraphs-paragraphs-translatable-revision-2951436-48.patch33.85 KBBerdir
#46 paragraphs-paragraphs-translatable-revision-2951436-46-interdiff.patch1.63 KBBerdir
#46 paragraphs-paragraphs-translatable-revision-2951436-46.patch33.85 KBBerdir
#44 paragraphs-paragraphs-translatable-revision-2951436-44-interdiff.txt14.41 KBBerdir
#44 paragraphs-paragraphs-translatable-revision-2951436-44.patch32.21 KBBerdir
#40 paragraphs-paragraphs-translatable-revision-2951436-40-interdiff.txt19.65 KBBerdir
#40 paragraphs-paragraphs-translatable-revision-2951436-40.patch20.68 KBBerdir
#31 paragraphs-paragraphs-translatable-revision-2951436-30-interdiff.txt3.1 KBBerdir
#31 paragraphs-paragraphs-translatable-revision-2951436-30.patch11.11 KBBerdir
#29 paragraphs-paragraphs-translatable-revision-2951436-28-interdiff.txt737 bytesBerdir
#29 paragraphs-paragraphs-translatable-revision-2951436-28.patch13.14 KBBerdir
#26 paragraphs-paragraphs-translatable-revision-2951436-26.patch13.73 KBBerdir
#24 2951436_24.patch16.86 KBmtodor
#24 2951436_interdiff_24_6.txt20.97 KBmtodor
#19 paragraphs-translatable-revision-2951436-19.patch1.6 KBBerdir
#6 2951436-add_test_content_moderation_translation.patch7.61 KBdubcanada
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

marcoscano created an issue. See original summary.

marcoscano’s picture

Title: Improve test coverage with content moderation in multi-lingual scenarios » Improve integration with content moderation in multi-lingual scenarios

@Berdir reminded us that we know there are issues with content moderation in multi-lingual scenarios, so we should probably make this issue about ensuring that works, apart from writing the tests.

Related: Content translations can be moderated independently

miro_dietiker’s picture

Title: Improve integration with content moderation in multi-lingual scenarios » Fix integration with content moderation in multi-lingual scenarios
Priority: Normal » Major

At least major now that content moderation is landed.

And Berdir said it's completely broken now. :-)

dubcanada’s picture

Is there anything I can help with to get this fixed?

Berdir’s picture

Hi

As a starting point to move this along, it would be useful to have failing test.

We recently committed a content moderation integration test for nodes, paragraphs and paragraph library items. I would recommend that you copy that and start extending it to cover translations as well.

That will show exactly what is working/broken and will make it easier to fix it.

dubcanada’s picture

I am a bit new to writing tests for Drupal 8 and paragraphs/content moderation/translations. I started with a basic failing test (hopefully).

miro_dietiker’s picture

Properly parentising.

miro_dietiker’s picture

Priority: Major » Critical

And promoting according to the roadmap.

dubcanada’s picture

I'm working on trying to get the tests to be written properly and pass. There is only a few things left.

Will add another patch soon, just super busy atm.

dubcanada’s picture

After playing around with it a bit more, nothing works when you turn on multiple languages and try and translate. Works fine for the default language. Form doesn't even show up on the translation edit/add page you just get

Fields that apply to all languages are hidden to avoid conflicting changes. Edit them on the original language form.

Using the asymmetrical translations patch you can bypass that error, but then have issues with drafts not overwriting published values when moved from draft to published.

If you turn off the

Fields that apply to all languages are hidden to avoid conflicting changes. Edit them on the original language form.

the fields show up, and work with the following issues.

When you create a new draft from the edit screen it adds a new revision, that can be seen in the database and on the latest revision tab. When you go to convert it from a draft to published the live version gets changed to English and you lose all your changes.

miro_dietiker’s picture

@dubcanada Great to see progress here. When you're on a topic for multiple days, please also provide WIP patches so we can provide early feedback.

bastnic’s picture

Just reporting a situation I encounter:
- moderated & translatable entity
- contains a paragraphs field, which is set to not translatable as its the paragraphs field themselves that need to be translatable as per the documentation
- the checkbox "Hide non translatable fields on translation forms" is forcely checked because "Moderated content requires non-translatable fields to be edited in the original language form."
- when I try to translate my content, I do not see the paragraphs field as it's hidden by the previous option.

Right now I've no idea on what level to fix it: paragraphs / content moderation / translation.

dubcanada’s picture

Yah not being familiar with Entity/Paragraphs/Content Moderation I could use some direction from @Berdir or @plach on where to start fixing stuff.

From my best guess it seems that we would need to turn off the Fields that apply to all languages are hidden to avoid conflicting changes. Edit them on the original language form. #access => FALSE using a hook and then fix the paragraphs on save to make sure the revisions are properly accounted for. It seems content_moderation nukes everyone of the revisions but the English somewhere along the line.

But my entity revision knowledge is not very good...

I'll try and get a WIP patch posted soon, but I'm super busy atm.

AMDandy’s picture

Our site has exactly the same issues that @dubcanada describes. I will gladly test any fixes against our content.

dubcanada’s picture

I have no time atm and won't until at least May so I will be unable to work on this anymore.

As was said above I think someone needs to write a simple failing test where a paragraph is added with content moderation and you try and translate it.

gg4’s picture

Can anyone provide a more concrete issue summary of the open issues involved here?

AMDandy’s picture

Two scenarios.

Synchronous translation (the default)

  • Create Paragraph in default language and save draft
  • Translate the page
  • Paragraphs field does not show in translation because it is not translatable.

Asynchronous translation (requires patch)

With paragraph container set to be translatable:

  • Edit default language and publish. Text in example paragraph is "text"
  • Create translation and publsh. Text in example paragraph is "translation text".
  • Edit default language and change paragraph text to "text 1.1" save as draft.
  • Edit translation and change paragraph text to "translated text 1.1" save as draft.
  • Go back to default page and refresh. Text has been reverted to "text".

If this isn't what you're looking for please let me know how I can better explain the issues we're experiencing.

bnjmnm’s picture

I created a module that provides a stopgap solution for this issue https://www.drupal.org/project/multilingual_revisions_helper

Ultimately, fixing the issue in Paragraphs (and other impacted modules like Pathauto) is the best solution, but I needed something fast for my clients, and I suspect that's true for others as well.

I'll look into creating a test that can help move this issue along, but hopefully this temp solution is helpful to those experiencing this problem

Berdir’s picture

Just starting to experiment, not much to see yet here. Managed to get the widget to show again, but now I need to figure out how to properly merge the translation in the draft. This will require a core issue, will open that asap to start discussing how to approach that. Edit: #2960253: [meta] Allow Paragraphs widget/field and similar use cases to to be considered translatable

I'll review and integrate the test coverage from earlier when this is starting to actually work.

Again. Do not bother using this yet, it won't work.

mtodor’s picture

I'll try to help a bit with tests.

But first I have one question (for now), are we going to have content moderation states for paragraph types?
I'm asking this because it's possible now to enable it for every paragraph type and then you have content moderation state selector for every paragraph subform. That looks super strange. Also, some exceptions are popping up when you try to change state for one paragraph.

Berdir’s picture

No, I don't think that makes sense. Paragraphs are a part of the workflow of the host entity, they should not be moderated on their own. I don't see how that would be possible as they are always references by a specific id.

Library is different, there is makes sense.

miro_dietiker’s picture

Crosspost, still posting... :-)
I'm not aware of Content Moderation providing any concept and UI for nested workflow states. With the idea of a composition relationship i think it's safe assumption that the host moderation state is defining a document workflow that thus applies to all containing Paragraphs - like we implement access control by delegating to the owning parent (looking at you, #2949412: Nested paragraphs automatically publish, even if the parent is a draft).

We have some discussion that is a bit related #2958775: [META] Decide about semantics of unpublished status

The workflow state of a host entity influences the host entity published status.
The published status of a Paragraph is a completely independent concept that is not influenced by the host entity workflow.

It's totally OK to publish a node with 10 paragraphs from which 2 paragraphs are and remain unpublished. Because that's the situation you reviewed. Publishing a specific Paragraph can then again go through the host entity workflow.

I think that makes perfect sense and is the best MVP with minimum complexity.

Berdir’s picture

Assigned: Unassigned » Berdir
mtodor’s picture

Status: Needs work » Needs review
FileSize
20.97 KB
16.86 KB

Here are some tests. I'm changing status to "Needs review" just to trigger them.

It's difficult to write meaningful tests since I'm not sure how it's supposed to work. :(
Anyway, there is interdiff, but it's not of big use because I have changed almost everything and patch from #6 was really good starting point.
I have split tests into two methods (one with only translation on a content entity, other with translation on a content entity and a paragraph type).

Some things that I have noticed are popping up few questions:

  1. On translation edit page (fe. /de/node/1/edit) actions to add/remove paragraphs are not available. I'm not sure, should they be there or not?
  2. Paragraph field is not translatable, but it's possible to set it over UI. Should we somehow block a user from doing such thing over UI?

I have noticed one problem with saving paragraphs in the "draft" state on a translation page. Here are steps to reproduce:

  1. create article with one text paragraph and publish it
  2. create translation for it and change content of that text paragraph => but save as Draft (on preview page it will be correct text for paragraph)
  3. go again to translation edit page for that article => old value for text paragraph will be there

Same problem happens when you change state to published on article preview page -> old paragraph is used.

Status: Needs review » Needs work

The last submitted patch, 24: 2951436_24.patch, failed testing. View results

Berdir’s picture

Thanks for working on that.

I unfortunately started just shortly before you uploaded your patch and made some bigger changes like converting to a browser test. But I already picked some parts from your test and will review it more closely to make sure we have those use cases covered.

This is still very much work in progress and it depends on the core patch that I will post soon in the referenced issue.

I did run into a major problem that I didn't quite expect, I was not able to publish two translation forward revisions that I created in parallel (Creating and viewing them worked). Maybe I'm missing something, but this seems to be a general core issue that is not related to paragraphs at all. What happens is that the second forward revision is then suddenly no longer actually a forward revision, so it is no longer accessible on /latest and while it still can be found through revisions.

I tried to revert it, but the revert flow does not seem to use the new createRevision() API yet, didn't investigate too much yet.

Status: Needs review » Needs work

The last submitted patch, 26: paragraphs-paragraphs-translatable-revision-2951436-26.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Bunty Badgujar’s picture

Hello,
I also having one issue related to this one. When node updated and saved as draft then views having paragraphs relation and paragraphs translation filter is not showing any content on live revision and latest revision.

Berdir’s picture

Ok, created a new issue in ERR to deal with the storage part, I also removed the hook that I added here in favor of the one from ERR, so to be able to test this, you need to combine all 3 patches.

The ERR issue has a kernel test that successfully merges two different drafts/forward revisions in different translations, so technically, things should be starting to work, if it weren't for the problems with the UI not showing those forward revisions anymore.

Status: Needs review » Needs work

The last submitted patch, 29: paragraphs-paragraphs-translatable-revision-2951436-28.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Ok, starting to make real progress. The latest patches in the core/ERR issues now allow to "properly" handle affected languages, which magically fixes the /latest page, so updated the test for that.

Still mostly just a proof of concept so far, needs more tests and the core patch will need a lot more work, tests and discussion, but things should start to work, so if someone wants to do some manual testing, combine the 3 patches and things should start to work.

Status: Needs review » Needs work

The last submitted patch, 31: paragraphs-paragraphs-translatable-revision-2951436-30.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

The last submitted patch, 19: paragraphs-translatable-revision-2951436-19.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Bunty Badgujar’s picture

Hi @berdir

I have tested these patches with stable paragraph (8.x-1.2) release and its working fine.

mtodor’s picture

I have tested this patch with patches for core and err (as it's suggested in #33). And I wasn't able to reproduce problems I had before.

Nested paragraphs still require attention, because it's possible to add/remove paragraphs for a nested paragraph in a translated content entity, that should be blocked too - I'm not sure is that in the scope of this issue or it's a separate issue.

Never the less, I have checked also code - the patch is quite complicated. ;)

  1. +++ b/paragraphs.module
    @@ -5,6 +5,8 @@
    +use Drupal\Core\Entity\ContentEntityInterface;
    +use Drupal\Core\Entity\RevisionableInterface;
    
    @@ -13,6 +15,7 @@ use Drupal\field\Plugin\migrate\source\d7\FieldInstance;
    +use Drupal\paragraphs\ParagraphInterface;
    

    I guess these uses are not needed anymore.

  2. +++ b/src/Plugin/Field/FieldWidget/ParagraphsWidget.php
    @@ -1088,6 +1088,10 @@ class ParagraphsWidget extends WidgetBase {
    +    // Signal to content_translation that this field should be treated as
    +    // multilingual and not be hidden, see
    

    The comment is not finished.

  3. +++ b/tests/src/Functional/ParagraphsContentModerationTranslationsTest.php
    @@ -0,0 +1,234 @@
    +    $assert_session->pageTextContains('paragraphed_test Page 1 DE has been updated.');
    +    $assert_session->pageTextContains('Page 1 DE');
    

    The 2nd assertion here is not relevant since it will take the result from message anyway.

  4. +++ b/tests/src/Functional/ParagraphsContentModerationTranslationsTest.php
    @@ -0,0 +1,234 @@
    +    $this->drupalGet("/node/{$host_node_id}");
    +    $assert_session->pageTextContains('Initial paragraph text 2 EN');
    +
    +    // Assert that the EN revisions were not affected.
    +    $this->drupalGet("/node/{$host_node_id}");
    +    $assert_session->pageTextContains('Initial paragraph text 2 EN');
    

    Duplicate code.

  5. +++ b/tests/src/Functional/ParagraphsContentModerationTranslationsTest.php
    @@ -0,0 +1,234 @@
    +    $this->drupalGet("/node/{$host_node_id}/latest");
    +    $assert_session->pageTextContains('Draft paragraph text 2 EN');
    +    $page->pressButton('Apply');
    

    What about having one test case where the article state is changed from draft to published inside edit form? I think it would be beneficial since sometimes strange things can occur when edit form is submitted.

miro_dietiker’s picture

@mtodor We have removed add / remove buttons and also the drag & drop while translating. It would be a regression if these actions suddenly show up again due to Content Moderation.

mtodor’s picture

@miro_dietiker I think you didn't understand me. I know that add/remove is not available for translation. But it's available for a nested paragraph in translation and I think that these actions should be removed there too. (unless I missed something during testing)

Berdir’s picture

This should work fine with nested paragraphs as well, will check. But yes, nested paragraphs is something that needs to be covered in the test as well. Another thing that is not yet done is also respecting the "hide untranslatable" fields setting of the parent for the paragraphs, as we also shouldn't show an untranslatable link/image/reference field for example when translating.

Berdir’s picture

Ok, bunch of improvements:

1. Verified that untranslatable nested paragraph fields work and do not show add/remove. It does work but currently need to make *both* the paragraph type translatable and the field untranslatable. Maybe we can find a way to avoid the first, but I think that's a separate issue.
( By adding a separate paragraph type, the default text fell away and I refactored it to no longer create two, we have a second now in the nested container, that's enough. interdiff is huge mostly because of that)
2. Extended test coverage on untranslatable fields on paragraphs. Making sure they can be changed on the default translation and are not visible on translations. For that, I had to enable the hide checkbox on the paragraph types, without, you get a weird validation error when trying to change them in a forward revision. And i implemented checking it in the paragraphs widget for hiding them.
3. A few other test improvements, including addressing the review from #36 (thanks), did change one draft publishing to go through the edit form.

TODO:
* Classic widget
* Show in the UI that the hide-untranslatable fields checkbox must be checked if content translation is used
* Review translatability clue logic, with the new #multilingual thing, I think it is actually not added anymore, so we don't need to remove it, but we might need to add it to visible untranslatable fields.
* Adapt to the final version of the core patch (which still needs quite some work), but I expect that will be mostly in ERR.
* Translatable paragraph fields: #2953343: Experimental asymmetrical content translation with Paragraphs breaks concurrent drafts
* Figure out what exactly is going on with the reports about getting validation errors even when not using content_moderation. I suspect it is related to \Drupal\Core\Entity\Plugin\Validation\Constraint\EntityUntranslatableFieldsConstraint and how that is forced on any entity, combined with the fact that working with paragraphs might end up saving non-default revisions.

Status: Needs review » Needs work

The last submitted patch, 40: paragraphs-paragraphs-translatable-revision-2951436-40.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

miro_dietiker’s picture

Sounds like great progress!

I think 1) is still a regression. Our container Paragraph types are marked as translatable. Before people even reported that if it is not marked as translatable, then displaying translation of children are broken. But OK, step by step.

Berdir’s picture

1) That was a typo/missing word. Obviously the field needs to be untranslatable but the paragraph type *must* be translatable. And that's not a regression :)

Berdir’s picture

* Our existing translation tests were indeed failing about the incorrect all translations suffixes, nice. I updated that by re-implementing the logic instead of trying to undo the suffixes that no longer happens. Strangely, I had to remove text_format from the types that should get a suffix because that didn't work for me. Not sure what is different in content_translation.
* Added a description to the hide untranslatable fields checkboxes in paragraphs
* Applied the same fixes to the classic widget, no tests yet, but we can just copy the test once finished and use thedifferent widget.

TODO (updated):
* Adapt to the final version of the core patch (which still needs quite some work), but I expect that will be mostly in ERR.
* Translatable paragraph fields: #2953343: Experimental asymmetrical content translation with Paragraphs breaks concurrent drafts
* Figure out what exactly is going on with the reports about getting validation errors even when not using content_moderation. I suspect it is related to \Drupal\Core\Entity\Plugin\Validation\Constraint\EntityUntranslatableFieldsConstraint and how that is forced on any entity, combined with the fact that working with paragraphs might end up saving non-default revisions.
* (new) Deal with the whole behaviors topic. Behaviors are untranslatable, so we have no choice but to hide those too when translating, but some use cases *could* theoretically result in translable fields in the behavior tabs. Not sure how to handle that yet.

Status: Needs review » Needs work

The last submitted patch, 44: paragraphs-paragraphs-translatable-revision-2951436-44.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Also starting to add a drupalci.yml, lets see if patching another contrib module will work.

The last submitted patch, 46: paragraphs-paragraphs-translatable-revision-2951436-46.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Total failure, wrong interdiff filename and bogus path to the core patch.

Lets try this again.

Status: Needs review » Needs work

The last submitted patch, 48: paragraphs-paragraphs-translatable-revision-2951436-48.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

So, with the fixes in the latest ERR patch and one small change, this is now passing as well. I'm not sure about that yet, right now, we only add the all translation marker if we actually are translating, unlike content_transation, which seems to do it always *if there are translations*.

Berdir’s picture

Updated to test against the latest ERR and core patches Found a problem in the previous ones that is now fixed and tests are passing again.

Berdir’s picture

This now also respects the hide untranslatable fields setting for behavior plugins and hides them then on translations. I also added a test for the classic widget which is almost identical except it doesn't get the new behavior field test coverage.

Berdir’s picture

Berdir’s picture

More test coverage, making sure that revisions can be reverted and only the current language is revert, just like with non-paragraph fields as well.

Berdir’s picture

And another update, moving #multilingual into form() according to the latest patch in the core issue.

eigentor’s picture

Some feedback from applying the current patch to the dev version of paragraphs:

It does not seem to fix this
https://www.drupal.org/project/paragraphs/issues/2920177
or this
https://www.drupal.org/project/paragraphs/issues/2956311
issue.

Which apparently means for the moment Paragraphs cannot be used in combination with content moderation on a multilingual site.

Berdir’s picture

Did you combine it with the corresponding core and ERR patch? This is mostly just test coverage and only works with those other patches. See the drupalci.yml file for the patches you need to apply.

eigentor’s picture

Indeed. I applied all three patches and now it appears to work.
@berdir: Thank you for all the work you put in.

For anyone trying to solve the same problem:

1. Update paragraphs to the dev version
2. Update Entity referernce revisions to the dev version
3. Apply the latest patch from this core issue #2960253: [meta] Allow Paragraphs widget/field and similar use cases to to be considered translatable
4. Apply the latest patch from the issue you are currently reading
5. Apply the latest patch from thie entity_reference_revisions issue #2961399: Support parallel translation forward revisions on untranslatable fields

Now the paragraphs should not be hidden anymore in a translated version of a node.

lobodakyrylo’s picture

If paragraph field contains required image and I click save node, drupal returns the next:

Warning: array_key_exists() expects parameter 2 to be array, null given in Drupal\image\Plugin\Field\FieldWidget\ImageWidget::validateRequiredFields() (line 306 of core/modules/image/src/Plugin/Field/FieldWidget/ImageWidget.php).
Drupal\image\Plugin\Field\FieldWidget\ImageWidget::validateRequiredFields(Array, Object, Array)
call_user_func_array(Array, Array) (Line: 283)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object, 'node_page_edit_form') (Line: 119)
Drupal\Core\Form\FormValidator->validateForm('node_page_edit_form', Array, Object) (Line: 571)
Drupal\Core\Form\FormBuilder->processForm('node_page_edit_form', Array, Object) (Line: 314)
Drupal\Core\Form\FormBuilder->buildForm('node_page_edit_form', Object) (Line: 74)
Drupal\Core\Controller\FormController->getContentResult(Object, Object)
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 582)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 151)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 664)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

Not sure why it happens but it can be solved with the next temporary patch. It changes a core module but I think the reason is in the paragraph module.

lobodakyrylo’s picture

Patch is

*** core/modules/image/src/Plugin/Field/FieldWidget/ImageWidget.old.php	Thu May 17 12:13:39 2018
--- core/modules/image/src/Plugin/Field/FieldWidget/ImageWidget.php	Thu May 17 12:09:50 2018
***************
*** 303,309 ****
        $image_field = NestedArray::getValue($form_state->getUserInput(), $parents);
        // We check for the array key, so that it can be NULL (like if the user
        // submits the form without using the "upload" button).
!       if (!array_key_exists($field, $image_field)) {
          return;
        }
      }
--- 303,309 ----
        $image_field = NestedArray::getValue($form_state->getUserInput(), $parents);
        // We check for the array key, so that it can be NULL (like if the user
        // submits the form without using the "upload" button).
!       if (!$image_field || !array_key_exists($field, $image_field)) {
          return;
        }
      }
rp7’s picture

About #60/#61
Unsure if related at all, but I guess there's no harm in sharing: I experienced this exact same error message in another use case & it appears to be a core bug, https://www.drupal.org/project/drupal/issues/2715859.

dubcanada’s picture

Nested paragraphs don't seem to work, seem to be getting an error.

Error message
Non-translatable fields can only be changed when updating the current revision.

Berdir’s picture

Nested fields are tested here and they work as long as all paragraph bundles are marked as translatable, even those that have just a non-translatable reference field. That can possibly be improved, but that's a pre-existing problem and not directly related to this. I suggest you open an issue about this.

kjankowski’s picture

@berdir I think it's worth mentioning that for paragraphs used as containers for other paragraphs, the option "Hide non translatable fields on translation forms" should be checked. Otherwise validation won't pass.

Berdir’s picture

Yes, exactly, that should be enabled for all paragraph types used in a workflow actually. I still need to update the documentation on how things need to be configured for everything to work correctly.

The existing documentation page is https://www.drupal.org/node/2735121. My plan was to use a more generic title and mention the status/suggested configuration on using it with content moderation (below a h2 Content moderation title or so):

* Which patches are necessary (until they are committed then that can be removed)
* That hide non-translatable fields must be enabled
* That all paragraph types, even containers with just a paragraph field must be marked as translatable.

Want to give that a try? Someone else can always improve it.

dubcanada’s picture

Okay after enabling "Hide non translatable fields on translation forms" and everything single field, it works.

Berdir’s picture

I'm not sure what you mean. You must enable all *paragraph types* and check the hide non-translatable fields on them all but *not* the paragraph fields themself on either the node or the paragraph types. Unless you are using the asynchronous translation patch but that isn't officially supported yet (and won't be for the classic widget).

Status: Needs review » Needs work

The last submitted patch, 55: paragraphs-paragraphs-translatable-revision-2951436-55.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Updated the used core patches, one was already committed but only to 8.6.x but trying to re-apply it shouldn't fail and should make it work on 8.5 as well.

I also migrated the existing paragraphs documentation to the new documentation system, slightly updated the existing multilingual page to mention that all paragraph types must be marked as translatable for now and created a new one specifically for content moderation that describes the necessary patches and configuration:

* https://www.drupal.org/docs/8/modules/paragraphs/multilingual-paragraphs...
* https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...

Status: Needs review » Needs work

The last submitted patch, 69: paragraphs-paragraphs-translatable-revision-2951436-69.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Same sudo user fail as in the ERR patch.

DuneBL’s picture

#71 solve many cases, but not the sub-paragraphs use case.

1-My setup:

-Drupal 8.5
-All paragraphs type are set as translatable (at least the "Authored by", "Published" and "Authored" fields)
-All paragraphs type have the option " Hide non translatable fields on translation forms" checked
-All paragraphs type doesn't have the field "Moderation state" set as translatable
-All the content type have the the field "Moderation state" set as translatable

Here are the installed patches:

PARAGRAPHS

Fix integration with content moderation in multi-lingual scenarios:
https://www.drupal.org/project/paragraphs/issues/2951436#comment-12642346 [comment #71]

Sub-paragraph fields cannot be translated:
https://www.drupal.org/project/paragraphs/issues/2864682#comment-12492141 [comment:11]

ENTITY REFERENCE REVISION

Support parallel translation forward revisions on untranslatable fields:
https://www.drupal.org/project/entity_reference_revisions/issues/2961399... [comment #37]

CORE

FieldItemList::equals is sufficient from the storage perspective but not from the perspective of ContentEntityBase::hasTranslationChanges
https://www.drupal.org/project/drupal/issues/2826021#comment-12632567 [comment #31]

Add hooks to act on a new revision being created
https://www.drupal.org/project/drupal/issues/2975754#comment-12631130 [comment #2]

Respect existing #multilingual property in content_translation_form_alter()
https://www.drupal.org/project/drupal/issues/2975762#comment-12631136 [comment #2]

2-Steps to reproduce:

1-Create a new node (in the case it is important, it is a "book page" type node)
2-the "content" field (of this node) is set as translatable and is a paragraph type
3-Add an "Accordion" to the content field (coming form "Bootstrap Paragraphs"
4-Add a sub-paragraph to this accordion: an accordion section
5-In the accordion section, set the "Accordion Section Title" and add a simple paragraph for the "Accordion Section Body"

The result is a nice English accordion

6-Now try to add a translation for this node: unfortunately, the "Accordion Section Body" is not there (No Paragraph added yet.)
It is impossible to translate!

DuneBL’s picture

regarding #72: the problem could be outside the translation process because I just discover that the "Accordion Section Body" also disappear from the English source node!!!

This is very strange:

-In the rendered english node (view), the "Accordion Section Body" is there/visible
-In the English edit form, the "Accordion Section Body" is empty (No Paragraph added yet.)
-If I use the link provided by Paragraph Edit to edit the accordion, the section body is there...

Just to make things clear, here is the deepness of this paragraphs

Content field
....-Accordion
........-Accordion Section
............-Accordion Section Body (simple paragraph)

in other word: 3 levels under the main field

cosmicdreams’s picture

Thank you for the excellent writeup DuneBL. By applying those patches to paragraphs 1.3, Drupal 8.5.4 and entity_reference_revisions my team and I were also able to repair the content editing screens that previously hide the paragraph fields.

But now we get a helpful message informing us that if we want to make edits we'll have to do that on the default language editing screen. What it doesn't say is that it is impossible to edit non default language versions of the paragraph fields.

This is a critical issue for me and my team so I'll be diving in the code today.

cosmicdreams’s picture

Digging a bit deeper into my situation:

Using:
Drupal 8.5.4
Paragraphs 1.x-dev
entity_reference_revisions 1.x-dev

The patches recommended above. (here's a snippet from the patches section of my composer.json:)

            "drupal/core": {
                "Revision create hooks": "https://www.drupal.org/files/issues/2018-05-28/revision-create-hooks-2975754-2.patch",
                "28256021 patch": "https://www.drupal.org/files/issues/2018-05-29/2826021-31.patch",
                "Make paragraphs respect multilingual": "https://www.drupal.org/files/issues/2018-05-28/respect-multilingual-2975762-2.patch"
            }
            "drupal/paragraphs": {
                "Fix integration with content moderation in multi-lingual scenarios": "https://www.drupal.org/files/issues/2018-06-06/paragraphs-paragraphs-translatable-revision-2951436-71.patch"
            },
            "drupal/entity_reference_revisions": {
                "Fix integration with content moderation in multi-lingual scenarios": "https://www.drupal.org/files/issues/2018-06-06/err-forward-translation-revision-2961399-37.patch"
            }

I also have need for multi-level paragraphs. In my case I have a "Layer" component that can have many different kinds of components added to it. In order to figure out what's going on I have configured the paragraphs and other content entities like this:

1. Every content type and paragraph type has been configured to hide untranslateable fields.
2. Every field that is attached to a paragraphs that can be translated (basically everything but the entity reference revision fields) has been configured to be translated by the user.

When editing a content type in an alternative language (not the default language) I am able to see the values of paragraph fields. But I'm not allowed to make any changes to those fields. In reading the issue thread above it sounds like that's intentional, so that a situation where the translation get's overwritten by the default language or lost.

That's what I see so far.

Berdir’s picture

I'll try to catch up here soon.

Looks like both of you are using translatable fields.

> 2-the "content" field (of this node) is set as translatable and is a paragraph type
> 2. Every field that is attached to a paragraphs that can be translated (basically everything but the entity reference revision fields) has been configured to be translated by the user.

Note that translatable paragraphs are not yet officially supported by this module and therefore are not and can not be tested here. It's quite possible that they are broken by the changes here, and the relevant patches/separate modules will need to be updated. I'm fairly sure that there's not much that can be done about that, not here anyway.

That said, if you really are using translatable paragraph fields then all those patches shouldn't be necessary. The reports we had from one client who needed this was that it worked for them with nothing but the patch from #2953343: Experimental asymmetrical content translation with Paragraphs breaks concurrent drafts, which is in the latest release.

cosmicdreams’s picture

Yea, this seems like something that recently broke because it was working before content_moderation and now it's not.

I'll see if that patch is included in our development environments.

What more information from me would help you on this issue?

DuneBL’s picture

Regarding #72/#73: removing the content moderation solve the issue...

cosmicdreams’s picture

Looks like an important detail was missed in setting up our tests. As found in: https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...

Additionally, make sure that the "Hide non translatable fields on translation forms" checkbox is enabled is enabled for all paragraph types that are used in moderated host entities.

After going to the content translation configuration pages and marking each paragraph field (really the entity_reference_revisions field) as untranslateable, the content editing experience began to change.

We'll report back tomorrow with more results.

mpp’s picture

@Berdir #71 thanks for the patch.

+ $paragraph_untranslatable_hide_description = t('Paragraph types that are used in moderated content requires non-translatable fields to be edited in the original language form and this must be checked.');

small typo: Paragraph types that are used in moderated content require non-translatable fields to be edited in the original language form and this must be checked.

jasonawant’s picture

Issue summary: View changes

Hi,

Does this page need to be updated to reflect this issue and patches?

https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...

Jason

mpp’s picture

Hi @jasonawant, yes. I updates the documentation with a link to this issue.

jasonawant’s picture

Thanks mpp!

I was going to ask DuneBL if content_moderation was enabled when reporting #72 and #73, which I see was when you said you disabled it in #78.

@Berdir, in #76, you state,

Note that translatable paragraphs are not yet officially supported by this module and therefore are not and can not be tested here.

If we aren't fixing translatable paragraphs in this issue, then where? Does the term translatable paragraphs really refer to translatable entity reference revisions field type?

I opened #2985448: Unable to translate nested paragraphs using the paragraphs_demo module today as it relates to translating content with nested paragraphs. Is that translatable paragraphs?

Using the following versions, patches, paragraphs_demo module and config. I am able to use content moderation to draft translated text paragraph, but not nested text paragraphs from the paragraphs_demo module.

Drupal 8.5.5
Paragraphs 1.x-dev#bedd55bbacd1456b77b9a814d0878b051ec6fc1c
ERR 1.x-dev#65067ad99268e91b1d76bafce244acfacdb77b7d

            "drupal/core": {
                "2975762 - Respect existing #multilingual property in content_translation_form_alter()":
                "https://www.drupal.org/files/issues/2018-05-28/respect-multilingual-2975762-2.patch",
                "2826021 - FieldItemList::equals is sufficient from the storage perspective but not from the perspective of ContentEntityBase::hasTranslationChanges":
                "https://www.drupal.org/files/issues/2018-05-29/2826021-31.patch",
                "2975754 - Add hooks to act on a new revision being created":
                "https://www.drupal.org/files/issues/2018-05-28/revision-create-hooks-2975754-2.patch",
                "2901459 - InvalidArgumentException when trying to save programatically created node translation with workflow ena":
                "https://www.drupal.org/files/issues/2018-05-15/2901459-19.patch"
            },
            "drupal/entity_reference_revisions": {
                "2961399 - Support parallel translation forward revisions on untranslatable fields":
                "https://www.drupal.org/files/issues/2018-06-06/err-forward-translation-revision-2961399-37.patch"
            },
            "drupal/paragraphs": {
                "2951436 - Fix integration with content moderation in multi-lingual scenarios":
                "https://www.drupal.org/files/issues/2018-06-06/paragraphs-paragraphs-translatable-revision-2951436-71.patch",
                "2864682 - Sub-paragraph fields cannot be translated":
                "https://www.drupal.org/files/issues/paragraphs-sub_paragraph_fields-2864682-11.patch"
            }

Regarding configuration steps

  • Install the standard install profile
  • Enable paragraphs_demo
  • Go to /admin/config/regional/content-language
    • Enable translatable for the Nested Paragraph paragraph type (w/out this changes to nested paragraph content will impact all translated content, include source translation)
    • Disable translatable for the Nested Content field
    • Save form
  • Add German translation to paragraphs_demo content
    • Go to /node/1/translations
    • Add German translation
    • Change values for title, text paragraph and nested text paragraph
    • Save translation
    • Compare source English and German translations, observe expected differences
  • Enable content_moderation module
    • Enable content_moderation as you do
    • Go to /admin/config/workflow/workflows/manage/editorial
    • Apply the workflow to the Paragraphed article content type
    • Save form
  • Attempt saving a draft revision
    • Draft a English source translation revision
    • Go to /node/1/edit
    • Add a text paragraph
    • Save as Draft moderation state
    • Observe validation error message, "Non-translatable fields can only be changed when updating the current revision."
  • Use default translatable configuration
    • Go to /admin/config/regional/content-language
    • Observe that Moderation state is set as translatable
    • Observe that "Hide non translatable fields on translation forms" is not selected
    • Go to /node/1/edit
    • Make an edit to the first text paragraph
    • Save as Draft revision
    • Observe expected validation error message, "Non-translatable fields can only be changed when updating the current revision."
  • Update translatable configuration and tested moderation
    • Go to /admin/config/regional/content-language
    • Per doc page, enable "Hide non translatable fields on translation forms" for all paragraph types that are used in moderated host entities, including nested paragraph type
    • Save config
    • Go to /node/1/edit
    • Make an edit to the first text paragraph
    • Observe nested paragraph summary information displayed
    • Save as Draft revision
    • Observe draft revision saved
    • Go to /de/node/1/edit
    • Make an edit to the first text paragraph
    • Save as Draft revision
    • Observe unexpected validation error message, "Non-translatable fields can only be changed when updating the current revision."
    • Go to /node/1/latest
    • Publish latest draft revision
    • Go to /de/node/1/edit
    • Make an edit to the first text paragraph
    • Observe nested paragraph summary information displayed
    • Save as Draft revision
    • Observe draft revision saved
  • Review nested paragraph is hidden
    • After the above steps, go to /node/1/edit
    • Observe nested paragraph summary information is not displayed; probably technically expected b/c hiding non translated fields; however
    • Go to /admin/config/regional/content-language
    • Disable nested paragraph option to "Hide non translatable fields on translation forms"
    • Click the nested paragraph edit button
    • Go to /node/1/edit
    • Observe nested paragraph summary information still is not displayed; not expected
  • Uninstall content moderation
    • Go to /admin/modules/uninstall
    • Delete all content moderation statues, go to /admin/modules/uninstall/entity/content_moderation_state
    • Uninstall and confirm uninstallation of content moderation at /admin/modules/uninstall/confirm
    • Uninstall and confirm uninstallation of content moderation at /admin/modules/uninstall/confirm
    • Go to /node/1/edit
    • Observe nested paragraph summary information still is not displayed; not expected

Possible issues

  • Unable to have two concurrent translated draft revisions
  • Unable to edit/translate nested paragraphs with content moderation enabled
  • Possible data loss when using content moderation with nested paragraphs and then uninstall content moderation
rp7’s picture

Great work so far here.

I'm currently working on a project for which this functionality is pretty critical (and its also using nested paragraphs). How can I help moving this issue further? (development itself will probably be a bit too advanced, but anything besides that...)

penyaskito’s picture

mpp’s picture

I'm seeing the same behaviour as reported in #83 and #84 namely data loss when using content moderation with nested paragraphs.

1. Not sure if it is related but we are using a collapsed paragraphs field widget (Paragraphs Classic widget with edit mode set to "Preview"), once the widget on a nested paragraph has been opened there is no more data loss when saving the node.

2. We're using version 1.3, the documentation didn't mention this but perhaps we should be using the dev version of paragraphs to make this patch work?

Update: setting edit mode to "Open" instead of "Preview" seems to be a workaround for the loss of data.

tguilpain’s picture

I am also seeing the same behaviour as reported above. So when editing nested paragraphs, if any of the paragraphs are not collapsed (aka open) then I will lose some of them when trying to save the node. Workaround is to make sure all paragraphs are open before doing any edit.

Drupal core is 8.5.6
drupal/paragraphs is 1.3.0 and we use the Experimental widget with default to collapse all paragraphs
drupal/entity_reference_revisions is 1.5.0

The site is multilingual (en + zh-hans), has content moderation and nested paragraphs.
I applied all the patches as described in the guide here https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...
Extract of my composer.json

"drupal/core": {
"2975762 - Respect existing #multilingual property in content_translation_form_alter()": "https://www.drupal.org/files/issues/2018-05-28/respect-multilingual-2975762-2.patch",
            "2826021 - FieldItemList::equals is sufficient from the storage perspective but not for code checking for changes": "https://www.drupal.org/files/issues/2018-05-29/2826021-31.patch",
            "2975754 - Add hooks to act on a new revision being created": "https://www.drupal.org/files/issues/2018-05-28/revision-create-hooks-2975754-2.patch"},
"drupal/entity_reference_revisions": {
            "Support parallel translation forward revisions on untranslatable fields": "https://www.drupal.org/files/issues/2018-07-26/err-forward-translation-revision-2961399-51.patch"},
"drupal/paragraphs": {
            "2951436 - Fix integration with content moderation in multi-lingual scenarios": "https://www.drupal.org/files/issues/2018-06-06/paragraphs-paragraphs-translatable-revision-2951436-71.patch"},

drupal/paragraphs is 1.3.0
drupal/entity_reference_revisions is 1.5.0

eloivaque’s picture

#81 works for me.

I uploaded to drupal 8.6 and applied last patches

Entity Reference Revisions
Support parallel translation forward revisions on untranslatable fields
Paragraphs
Fix integration with content moderation in multi-lingual scenarios

mediabounds’s picture

I'm also experiencing the issue with nested paragraphs disappearing on a host node where content moderation is enabled. I tracked down the issue to the recommended patch for Entity Reference Revisions (not Paragraphs) and posted an interim solution:
https://www.drupal.org/project/entity_reference_revisions/issues/2961399...

aerrasti’s picture

It worked for me.

Following the steps from: https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...

core 8.6.1
paragraphs : 1.3 + patch (#71)
entity_reference_revisions: 1.5 + patch(https://www.drupal.org/project/entity_reference_revisions/issues/2961399)

thanks!

ious’s picture

(Hi)
Drupal 8.6.1
"drupal/paragraphs": "1.x-dev#f9c964e"
"drupal/entity_reference_revisions": "1.x-dev#65067ad",

Patch "Support parallel translation forward revisions on untranslatable fields": https://www.drupal.org/files/issues/2018-09-12/err-forward-translation-r...
Patch "Fix integration with content moderation in multi-lingual scenarios": https://www.drupal.org/files/issues/2018-06-06/paragraphs-paragraphs-tra...

and following: https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...

I am experiencing the following behavior with “paragraph classic” as form display :

Problem 1: Paragraph deleted in original langage, re-appear after publishing outdated translated version.
1) EN version (original) : Publish page including paragraph A
2) Go to FR version : Create a new draft in french page including paragraph A (not modified)
3) Go back to english version : Delete paragraph A and publish the english version
4) Go back to FR version : The previously paragraph A is still existing, Publish FR version including paragraph A (not modified)
5) Go back to EN version : Previously deleted paragraph A is displayed on EN version

Problem 2: Error message “Non-translatable fields can only be changed when updating the original language. ” when existing outdated french revision
1) In translated (FR) version, I am creating a draft
2) Go to original version, publish a new revision including new paragraph B
3) Go to FR version: the paragraph B does not appear on the current revision, and when I am saving the translated version, the following message appear: “Non-translatable fields can only be changed when updating the original language.”

These 2 issues are connected, and make the system difficult to use.

Is it an issue from paragraphs or ERR ? any idea ?

Thanks!

pavlosdan’s picture

We are experiencing a very similar issue like problem 1 from #91 but with the experimental widget. (data loss)
using same patches as #91 on 8.6.1

1) EN version (original) : Publish page including paragraph A
2) Go to ZH_HANS version : Create a new draft in Chinese page including paragraph A (not modified)
3) Go back to EN version : Add paragraph B and publish the english version. Paragraph B shows up in published ZH_HANS version (intended?).
4) Go back to ZH_HANS version : Paragraph B exists in the published version but not the draft (latest revision),
5) Publish ZH_HANS version.
6) Previously added paragraph B is lost from both EN and ZH_HANS versions!

Two issues seem to be happening here:
1) (critical) would be the apparent data loss when 2 concurrent language drafts are being moderated at the same time.
2) We would expect the new paragraph to show when we create a new draft (as before with workbench_moderation) but not show on the already moderated and published translation.

The first issue could probably be solved by disallowing 2 drafts in separate languages to exist at the same time? So if the original node is moderated, a new draft cannot be created on the translation and vice versa.

Anybody’s picture

I can sadly also confirm #91... exactly the same issue which is really bad on multilanguage pages :( This also confirms to be a duplicate of #2956311: Error when save translation, I think.

maximpodorov’s picture

In the proposed patch I removed the changes for paragraphs.module which were introduced in #26 and #44 since they are not important functionally and conflict with patches from #2904705: Support asymmetric translation in experimental widget which are required for real multilingual sites (and they add unusable "use" clauses).

Status: Needs review » Needs work

The last submitted patch, 94: paragraphs-paragraphs-translatable-revision-2951436-94.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Even though I agree with #94 that those changes are not important for functionality I do think that they are important for UX. This issue is close to be committed, so I think we should keep it and when done, the other issue should remove the text.

This should fix the test fails. See: [#2958726]

Status: Needs review » Needs work

The last submitted patch, 96: paragraphs-paragraphs-translatable-revision-2951436-96.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Ha! So we had this already. :)

Status: Needs review » Needs work

The last submitted patch, 98: paragraphs-paragraphs-translatable-revision-2951436-98.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Huh? This was missing.

Working on more test coverage.

johnchque’s picture

There are some problems with the following workflow:

  • Create node, publish it.
  • Add a translation draft
  • Update the default translation, add/remove a paragraph, publish it.
  • Go back to the translation draft, it does not have those new paragraphs.

Adding test coverage. :)

Status: Needs review » Needs work

The last submitted patch, 101: paragraphs-paragraphs-translatable-revision-2951436-101.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Thanks for starting the test coverage for this, working on fixing it.

There's also a second scenario, creating a draft EN revision first and then adding a translation, that will complain about having translation changes and we can't save it. That's because core always compares it against the default revision, see #2833049: ContentEntityBase::hasTranslationChanges will compare a forward revision with the default one instead of the newest forward revision. We need to write a test for that in that core issue using an untranslatable field, then include the patch here in drupalci.yml. Because this existing case will run into the exactly same problem when trying to save.

We should also include a nested paragraph in the test you added, because that adds another level of complexity as well as removing one. I'd suggest a structure like this:

Original:
- Text 1
- Nested
- - Text 2

Then change to:
- Nested
- - Text 3
- - Text 2

And then not only check that the fields are there but also that the translation can be saved.

Berdir’s picture

This updates the used patches and enables content_moderation and correctly configures things in the paragraphs_demo module, to make manual testing easier.

Status: Needs review » Needs work

The last submitted patch, 104: paragraphs-paragraphs-translatable-revision-2951436-104.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

As suggested by @Berdir this should fix the tests. ;)

Status: Needs review » Needs work

The last submitted patch, 106: paragraphs-paragraphs-translatable-revision-2951436-106.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

We saw that there are many tests that depend on Paragraphs demo which is not good. Thus, since adjusting the tests to work with content moderation would take almost the same amount of time, we decided to remove the test dependency on paragraphs demo in this issue.

Fixing tests for the experimental widget. Uploading just to see if there are something else that is being broken with this. :) Still working on the other tests.

Status: Needs review » Needs work

The last submitted patch, 108: paragraphs-paragraphs-translatable-revision-2951436-108.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Fixing more tests. :)

Status: Needs review » Needs work

The last submitted patch, 110: paragraphs-paragraphs-translatable-revision-2951436-110.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Fixing more tests. :)

Status: Needs review » Needs work

The last submitted patch, 112: paragraphs-paragraphs-translatable-revision-2951436-112.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Insomnia and coding, killer combo. :)

Status: Needs review » Needs work

The last submitted patch, 114: paragraphs-paragraphs-translatable-revision-2951436-114.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

Adding more tests, I tried manually testing with the patch of #2833049: ContentEntityBase::hasTranslationChanges will compare a forward revision with the default one instead of the newest forward revision but I get the same error message when trying to save the translation of a node that has a draft "Non-translatable fields can only be changed when updating the original language.".

Status: Needs review » Needs work

The last submitted patch, 116: paragraphs-paragraphs-translatable-revision-2951436-116.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

So I found a failing scenario:
- We create a published node with one text paragraph, we create a draft revision adding a second paragraph, trying to save the translation will fail

It will not fail if we only update the content of the existing paragraph.

Berdir’s picture

Progress, changes are in the updated ERR patch. Now stuck on the validation error.

Status: Needs review » Needs work

The last submitted patch, 119: err-forward-translation-revision-2961399-69.patch, failed testing. View results

Berdir’s picture

Status: Needs review » Needs work

The last submitted patch, 121: paragraphs-paragraphs-translatable-revision-2951436-119.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

So I updated the tests to follow the expected behavior. The module will follow core patterns in the first implementation by showing the Paragraphs that are in the last Published original language node.

We expect to change the behavior of it on #3004099: Allow to translate paragraphs from pending revisions to display those Paragraphs added on pending revisions in the translation form.

While working on this, I did find a problem, when:
- Create a published node in EN with a Text Paragraph and a Nested one that contains another Text Paragraph.
- Create a draft, remove the first Text Paragraph, add a second text Paragraph inside the Nested one.
- Translate the node, with the current implementation, we should get the structure of the published EN node.
- The deleted paragraph in draft is still present in the translation form as expected, however, the Paragraph added inside the Nested one, is also displayed when it shouldn't.

It seems that we are not passing the right translation of the Nested Paragraph yet. Because in ERR when checking if !$new_revision->isDefaultTranslation() for node it returns TRUE but for the Nested Paragraph it returns FALSE, in entity_reference_revisions_entity_revision_create().

I also update the translation config of the Nested paragraph in demo. :)

johnchque’s picture

The last submitted patch, 123: paragraphs-paragraphs-translatable-revision-2951436-123.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 124: paragraphs-paragraphs-translatable-revision-2951436-124.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Berdir’s picture

Another ERR patch update, fixed those access test and also made one small fix so that ParagraphsExperimentalContentModerationTranslationsTest is passing. Didn't check Classic yet, I think that wasn't updated yet.

Status: Needs review » Needs work

The last submitted patch, 127: paragraphs-paragraphs-translatable-revision-2951436-127.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

johnchque’s picture

This passes locally. :)

Berdir’s picture

Ok, so we have a passing ERR + Paragraphs patch now again, within the limitations of Drupal core, see the updated documentation page (https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...) and the follow-up mentioned above and in that documentation page.

Our plan is to commit the ERR and Paragraphs patch by the end of this/beginning of next week and create new releases of both. There are 84 people following here, so if you are one of those, please test the latest patches. We're specifically interested in anything that could result in data loss/overwritten data. UI improvements/bugs can be fixed later, but we really want to avoid any bugs that could result in losing paragraphs or translations.

ious’s picture

Grateful for your work @yong9412 and @berdir,

Related to #91, I replayed the scenario and it solved my issue.

Drupal 8.6.1
"drupal/paragraphs": "1.x-dev#2cb0efc"
"drupal/entity_reference_revisions": "1.x-dev#e78d3d3",
"Support parallel translation forward revisions on untranslatable fields": "https://www.drupal.org/files/issues/2018-10-03/err-forward-translation-r..."
"Fix integration with content moderation in multi-lingual scenarios": "https://www.drupal.org/files/issues/2018-10-04/paragraphs-paragraphs-tra..."

Anybody’s picture

@Berdir and @yongt9412: Thank you very very much. I'm also testing and it looks good so far.
Can you tell if #2991757: Saving a translatable paragraph edits the paragraph revision instead of adding a new revision is affected by this patch and if it's still required? We have a case where we needed to use that patch, but I think they may be related...

Berdir’s picture

> Can you tell if #2991757: Saving a translatable paragraph edits the paragraph revision instead of adding a new revision is affected by this patch and if it's still required? We have a case where we needed to use that patch, but I think they may be related...

I don't know. According to the latest reports there, the test passed on 8.6 without any patches, if you do still have specific problem that is *not* fixed by this, then please update that issue with steps to reproduce. If it works now, then I'm more than happy to close that other issue as a duplicate.

acontia’s picture

I can confirm that the combination of patches + config described here (https://www.drupal.org/docs/8/modules/paragraphs/multilingual-and-conten...) works for me and I can't see any data loss or UI bug.

I have tested using:
- drupal/core:8.5.7
- drupal/paragraphs:dev-1.x#2cb0efcba81a0eb2707ec443f954c586e3112fd5
- drupal/entity_reference_revisions:dev-1.x#e78d3d3256b975cef7d48cdec68bc83759880dab
- Content Moderation enabled
- Multi-lingual site, 2 languages enabled for content translation

I also had to apply this patch (https://www.drupal.org/files/issues/2018-04-16/drupal_workflow-initial-s...) to avoid having the following error when accessing the edit form of some existing nodes with Content Moderation enabled:

Error editing translations: The state ‘’ does not exist in workflow.

I am updating from drupal/core:8.4.8 to 8.5.7 at the same time as I apply these changes, not sure if that's the reason I'm getting these errors.

The combination of patches that work for me are:

            "drupal/core": {
                "Respect existing #multilingual property in content_translation_form_alter()": "https://www.drupal.org/files/issues/2018-05-28/respect-multilingual-2975762-2.patch",
                "FieldItemList::equals is sufficient from the storage perspective but not for code checking for changes": "https://www.drupal.org/files/issues/2018-05-29/2826021-31.patch",
                "Add hooks to act on a new revision being created": "https://www.drupal.org/files/issues/2018-05-28/revision-create-hooks-2975754-2.patch",
                "Workflow: Get first state from config in case empty initial state": "https://www.drupal.org/files/issues/2018-04-16/drupal_workflow-initial-state-fix-2901459-16.patch"
            },
            "drupal/paragraphs": {
                "Fix Paragraphs integration with Content Moderation in multi-lingual scenarios": "https://www.drupal.org/files/issues/2018-10-04/paragraphs-paragraphs-translatable-revision-2951436-129.patch"
            },
            "drupal/entity_reference_revisions": {
                "Support parallel translation forward revisions on untranslatable fields": "https://www.drupal.org/files/issues/2018-10-03/err-forward-translation-revision-2961399-74_0.patch"
            }

Thanks very much for the hard work!

Anybody’s picture

Thanks @Berdir!
+1 for RTBC on the documented steps. I'll test #2991757 which doesn't work with latest dev and the documented steps... patch doesn't apply. So if someone else already used the same steps and has results, please tell us. I'll leave a note there too.

mpp’s picture

I applied the patches in #129 & 2961399#74 and they solve the loss of data I described in #86.

Our testers did some more in depth testing and no new issues came up.

RTBC+1

mpp’s picture

Status: Needs review » Reviewed & tested by the community

Setting to RTBC based on #131, #134, #135 & #136.

mfernea’s picture

Following the steps at #134 I got the issue fixed.

Don't forget: Make sure that the "Hide non translatable fields on translation forms" checkbox is enabled on admin/config/regional/content-language for all paragraph types that are used in moderated host entities.

miro_dietiker’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
135.58 KB

Updating patch without drupalci. ERR was committed.

Status: Needs review » Needs work

The last submitted patch, 139: paragraphs_2951436_moderation_139.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

miro_dietiker’s picture

Retrying after an ERR release.

miro_dietiker’s picture

Status: Needs work » Needs review

  • miro_dietiker committed e03d7e8 on 8.x-1.x authored by Berdir
    Issue #2951436 by Berdir, yongt9412, mtodor, maximpodorov, dubcanada,...
miro_dietiker’s picture

Status: Needs review » Needs work

So now that this is committed, we need to decide and define how to move forward with multilingual content moderation.
Will close this issue once the follow-ups are organized.

miro_dietiker’s picture

Status: Needs work » Fixed

Almost forgot in the rush of the commit: BIG THANK YOU to everyone involved here in helping us to make Paragraphs compatible with multilingual Content Moderation, by all kinds of activities, from reporting / reproducing problems, bugfixing, writing tests, and every other kind of support.

Closing this here as we have a parent meta. Will update the parent today.
There is remaining work to improve the experience and content moderation workflows, but we will handle this in follow-ups.

We will wait for some more positive feedback and some production "testing" to trigger the release button for this milestone.
Please report related bugs as follow-ups and link them to the parent.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

SoCalErich’s picture

I think I may need this patch as I am using Content Moderation and Paragraphs on a Multilingual site with Layout Builder and I cannot edit paragraphs attached to a custom inline block that contains in layout builder when the node goes from published to draft. As soon as I edit a paragraph and put the node in Draft status, if I go back to the paragraph in Layout Builder, I get "You are not allowed to edit or remove this Paragraph."

I am on Drupal 8.8.5, so this latest patch fails to apply for me. Can this be rerolled for 8.8.5?

vistree’s picture

I still have problems when using the combination of paragraphs, content translation and Content Moderation:
Maybe issue https://www.drupal.org/project/entity_reference_revisions/issues/3024588 is related?

To reproduce:
1. Create a new node and set language to "not defined"
2. Add a paragraph
3. Save node with status Published
4. Edit node and change language to EN
5. Save node
6. Try to edit again -> you get an error message:

InvalidArgumentException: Invalid translation language (und) specified. in Drupal\Core\Entity\ContentEntityBase->addTranslation() (line 955 of /web/core/lib/Drupal/Core/Entity/ContentEntityBase.php).

vistree’s picture

@miro: can we reopen the issue as of #148?

Berdir’s picture

No, this has been closed and committed 2 years ago. You can always create a new issue if something doesn't work for you.

jlatorre’s picture

I can confirm I still have the issue with following steps:

1/ Add source node EN
2/ Publish source EN
3/ Add translation FR
4/ Save translation FR as draft

So the content of the paragraph in translation FR is not the translated one but the source EN

bnjmnm’s picture

@jlatorre @vistree if this is at all a priority I recommend taking the 1-2 minutes to open a new issue reporting the problem you're experiencing. There's little to zero chance of anyone re-opening a 2 year old issue, but a high likelihood of someone attending to a new one.
When you report the issue, additional info such as the type of fields in the paragraph will really help too, as well as confirmation that the issue can be reproduced on a new site. The best way to get the issue addressed is include a failing test that reproduces the issue.

Disclaimer: I'm not a Paragraphs maintainer, just want to share what I've seen help get issues acknowledged in the queue .