Add translation integration

This is super important.

For new entities added with IEF:
Create nodes in parent node's language. If parent has no language set (because it has not been created yet), set language to undefined and iterate over all children to update their languages when parent is saved.

Agreed.

For entities which are referenced via IEF inside a parent entity:
Try to load referenced entity in parent's language. If there is no entity in the required language, load entity in default language and add a translate button next to it (right next to edit/remove buttons). Allow inline translation.
Once the entity is translated, hide the translate button.

I don't see why we need the translate button, if the parent entity is being translated, and the reference field is translatable, we can just start translating the referenced entities, no?

I don't see why we need the translate button, if the parent entity is being translated, and the reference field is translatable, we can just start translating the referenced entities, no?

I agree that, and in the single case add a button seem to be complex for ui... is better to start translating if the entity_reference is translatable.

For new entities added with IEF:
Create nodes in parent node's language. If parent has no language set (because it has not been created yet), set language to undefined and iterate over all children to update their languages when parent is saved.

This is the best way :D Good job.

But I think we'll have the same kind of problem that I face on https://www.drupal.org/node/2534408 outcome. We should perhaps consider this a method to simplify / debuger submit this part with a huge loop through the twigs, I analyze at length and I think it lacks a stage to say whether the entity is not therefore create author then puts the current user. This would be the same principle as for this issue, which aims to put the language by default if the parent entity has no language.

Any progress on this issue? Does anyone have some kind of unfinished patch file? At least the existing entities should be loaded in the parent's language... Of course I'm willing to help/contribute

First step towards translation integration: this patch loads the correct translation, if a translation for the given language already exists. So actually creating a translation for a given entity still has to be done, but at least one can now edit the correct translation.

I don't see why we need the translate button, if the parent entity is being translated, and the reference field is translatable, we can just start translating the referenced entities, no?

I agree that, and in the single case add a button seem to be complex for ui... is better to start translating if the entity_reference is translatable.

Both highly confuse me.
We have ported paragraphs module for D8 and covered the most common use cases in translation.

If you have entity translation enabled then the parent entity is multilingual and has certain translatable fields.
Similarly the child entity is multilingual and has translatable fields.
As a result, the parent points to the very same child entity which means the reference field is NOT translatable.
Chances are high that a user wrongly makes the entity reference field translatable because he does not fully understand how it works. With paragraphs we decided to make this use case perfect until we ever think about supporting translatable references. We even discussed how we can prevent the user from doing this or at least put a warning.

And yeah, i warned in the beginning that supporting translations is complex and there are many edge cases to cover and everyone disagreed with comments like with D8 things should be super easy. But we still discover special cases whenever we use it.

The paragraphs tests might give a hint about relevant cases / workflows.
Mind that with IEF things are much more troublesome than with Paragraphs, since the (component) entities could be edited separately and through this e.g. the source language might be out of sync!

If you have entity translation enabled then the parent entity is multilingual and has certain translatable fields.
Similarly the child entity is multilingual and has translatable fields.
As a result, the parent points to the very same child entity which means the reference field is NOT translatable.

That's right. But currently we have one problem with IEF:

Imagine, you have created a node in German, having a reference to another translatable entity, which also is created in German first automatically. Next, you wan to translate the node to English. Of course you still have the reference to the same entity. And of course the entity is translatable per se, but at this point it has no english translation. So, when create or edit the english translation of the node, and edit the referenced entity per IEF widget, than you still would modify the german version instead of creating an english translation without knowing it. That's the big problem.

Once you have the english translation of the referenced entity, my patch in #5 ensures that the correct language is loaded within the IEF widget.

Chances are high that a user wrongly makes the entity reference field translatable because he does not fully understand how it works.

I disagree with that. It really depends on the use case and your customer's needs. E.g. we are currently working on 2 multilingual D8 sites, both having lots of entity references. One site has translatable reference fields to offer true internationalization, allowing to have differently structured content per language and not to force to translate and keep every single field of the source language or add own content. The second site however is meant to have really a strict 1:1 translation of german and english. In this case we decided to have language neutral reference fields, but translate the entities itself.

Here's an updated patch, which...

• loads the correct translation, if exists
• creates a new translation, if the referenced entity is translatable

Thanks to miro_dietiker - I've looked into paragraphs module, how this is solved there.

This patch solved our problem for the common scenario: node is translatable, has untranslatable entity reference field, the referenced entities again are translatable.

I have not tested any other scenario: reference field translatable, no translation at all,... Also note, that I've introduced a $target_langcode variable besides the $parent_langcode variable, and most uses of the parent langcode have been replaced by target_langcode, which takes its value from the $form_state (like paragraphs does). I'm not sure, if this is always correct.

Again: it solved our problem with the described scenario, but it's far from RTBC: code reviews, both manual and automatic testing are necessary. But I think, it's a great start and hopefully a big help for everyone, that needs IEF in multilingual pages urgently :-)

small update of my patch: now trying to initialize the target language with the language information from the form state - this fixes a bug, that was raised, when you tried to add a new entity on editing the node translation, because the wrong language was set for the new entity

I'll change status to needs review now. We experienced no problems so far. Time for others to try ;)

Re-roll patch against current dev

It would be very beneficial to add a test to this patch. It would show how this is supposed to work and make sure we are all on the same page WRT that.

I think that we need to support translatable/non-translatable fields in the long run. There are use cases for both. Patch is a great start, but is missing huge part, which was already mentioned in #8.

+++ b/src/Plugin/Field/FieldWidget/InlineEntityFormBase.php
@@ -309,6 +310,17 @@ abstract class InlineEntityFormBase extends WidgetBase implements ContainerFacto
+   * @param \Drupal\Core\Form\FormStateInterface $form_state
+   * @param \Drupal\Core\Field\FieldItemListInterface $items
+   * @return string
+   */

Since I'm particularly interested in untranslatable entity with translatable entity reference I tried to implement a little change. This workflow is useful when the content is not the same between translations.

Before this change the entity reference would still point to the same entity between translations. So when translating a page if the user also modified the content of the entity reference it would also change the content of the original one since it was still the same entity.

Thank you for your update.
I would highly recommend to start ASAP with test coverage for what was discussed and improve it with every update when we cover more cases.

Thank you for this! I have tested it with the Paragraphs module, and this seems to work nicely. Will test some more!

regarding patch #14:

You'll need a specific if-condition in your added else-branch. It may work for your use-case, but for every other use-case this causes a big problem. No matter, if you have a single-language page or using a non-translatable reference fields pointing to translatable entities, you'll always get duplicates of your referenced entities on every save action.

We were wondering today, because an existing site behaved oddly, creating duplicate entities on every node save. We suspected that one of the last updates we made, must be guilty. After having the same problems on a fresh and clean installation, we tried different versions of core and IEF. And finally we saw, that using the patch from #14 instead of #12, was the root of our problem!

An additional problem:

When creating a new translation for content with an inline entity form, if you change the source language for the translation, you would expect the inline entity references to change to match those of the new source language, same behavior as other fields. However, they continue to match the first source language.

For example, suppose your site supports three languages: English, French, and Spanish. The default site language is English.
You have a node that has a multi-valued inline entity form, and this node was originally created in English, and then translated to French.
You would like to translate this node to Spanish, using the French version as the source.
You click the link to create a new Spanish translation. By default, the source is set to English, and all fields including the inline entities have the English versions pre-populated.
You change the source language to French.
You observe all the fields EXCEPT the inline entity reference fields change to match the French version.

Expected behavior: in this scenario, the inline entity reference fields should change to match the source version.

@paulmckibben: I'm not sure, but I guess that is not IEF behaviour but core's. Have you tried the same scenario using one of core's entity reference field widget, not IEF, e.g. a select list?

The patch needs a re-roll if anybody has time.

This has been marked as a must have for 8.x-1.0-beta1

Here's the re-roll... I did some quick tests with the complex widget only. Still works as expected for me.

Apropos tests.... we still need some automated tests, If I found time, I'd would contribute some. Unfortunately tonight and the next days it won't be possible for me - maybe on friday afternoon, but I can't promise now...

Setting to needs review to see if this breaks currents tests

@agoradesign or anyone else can someone explain to me in steps the expected behavior and how I can test manually and/or write tests.

I have time to work on IEF this week.

Sounds good.. ok, let's start with all possible and legitimate use cases, that may occur. From my point of view I'd say, that all that things have to be covered: to add and edit referenced entities...

• ...with both parent and child entities are translatable, the reference field itself being untranslatable (this is covered in my patch) --> referenced entity should be translated on save, if translation missing. If translation already exists, of coure the right one should be loaded into form and saved afterwards on submit
• ...with parent entity translatable, referenced entity not, but the reference field itself being translatable (see comment #14) --> I'm not sure, what the expected behaviour should be. Comment #14 proposes to clone the referenced entity, so that you can edit it independently. I think, this would actually make sense - currently this is not covered in the patch nor in the unpatched code

I'm not sure, if you should also address comment #18, as I already mentioned, that I'm not sure, if this is really a IEF problem/bug or a core bug or maybe normal/expected core behaviour.

I'm also not sure, if other constellations, like parent entity untranslatable, field (of course) untranslatable, but referenced entity translatable, make sense and should be tested too.

But anyway: these are just the use cases that come into my mind and make sense for me (and especially the first example is that, what I would expect). This maybe incomplete as well as incorrect - I'm no maintainer of IEF...

And don't forget: There are two widget types: InlineEntityFormComplex and InlineEntityFormSimple -> so every use case has to work in both variants.

Regarding manual/automatic testing: of course it would help and is important, if someone can confirm, that everything works expected after instensively testing everything manually, but the most important thing now would be to have a high as possible automated test coverage, in order to have everything well documented in these tests and to ensure that no future change breaks the functionality.

I don't know, if you have experience with writing automated tests (in Drupal or at least in PHP in general). The best thing is always to look at existing examples. It's always a good choice to look into core test classes. In this case, you should of course have a look into the existing tests in IEF - you should either add functions to the existing classes (probably the best) or add your own classes for multilingual test cases and still look at the existing tests and how they are written (http://cgit.drupalcode.org/inline_entity_form/tree/src/Tests?h=8.x-1.x)

Maybe you could also benefit from looking at the tests in Paragraphs module (http://cgit.drupalcode.org/paragraphs/tree/src/Tests/ParagraphsTranslati...)

In comment #24 it says:

"with parent entity translatable, referenced entity not, but the reference field itself being translatable (see comment #14) --> I'm not sure, what the expected behaviour should be. Comment #14 proposes to clone the referenced entity, so that you can edit it independently."

Here's some ideas I had. Coffee is still kicking in so my apologies if the ideas are stupid:

Option 1: The referenced entity should not be cloned automatically. The form should show the fields of the referenced entity. When editing the form, the referenced entity fieldset should be styled differently with messaging that conveys the inability to translate the fields.

Option 2: Have a setting in the admin UI for this and let the site administrator choose whether or not to clone in this situation.

Option 3: Disable the field on the edit form and have two buttons. One button would say "Clone" and the other button would say "Edit". If the editor chooses "Edit" then they are allowed to edit the source values. If they choose "Clone" then the entity is cloned.

Regards,
Loop

The patch in #21 is no longer applying against the latest dev version of IEF. Would it be possible to get this first version of translation support merged and then continue with further improvements?

@reekris
The patch isn't mergeable without tests.

We also need to replicate the functionality of this alter:

function language_form_alter(&$form, FormStateInterface $form_state) {
  // Content entity forms may have added a langcode field. But content language
  // configuration should decide if it should be exposed or not in the forms.
  $form_object = $form_state->getFormObject();
  if ($form_object instanceof ContentEntityFormInterface && $form_object->getEntity()->getEntityType()->hasKey('langcode')) {
    /** @var \Drupal\Core\Entity\ContentEntityInterface $entity */
    $entity = $form_object->getEntity();
    $entity_type = $entity->getEntityType();
    $langcode_key = $entity_type->getKey('langcode');
    if (isset($form[$langcode_key])) {
      $language_configuration = ContentLanguageSettings::loadByEntityTypeBundle($entity->getEntityTypeId(), $entity->bundle());
      $form[$langcode_key]['#access'] = $language_configuration->isLanguageAlterable();
    }
  }
}

Here is a reroll of patch #21 until the tests are added

+++ b/src/Plugin/Field/FieldWidget/InlineEntityFormSimple.php
@@ -36,6 +36,21 @@ class InlineEntityFormSimple extends InlineEntityFormBase {
+          $metadata->setSource($parent_langcode);

$parent_langcode var has not been defined at this point?

InlineEntityFormComplex and InlineEntityFormSimple have same 14 lines to get the new entity translation this should be refactored into a new method on InlineEntityFormBase

Here is an updated with a new function InlineEntityFormBase::getEntityItemTranslation which refactors the current logic that is that same between the 2 widget classes.

Maybe we could completely merge $parent_langcode and $target_langcode. Like I stated in #8, I was introducing the $target_langcode, as I needed this information, and I was unsure about possible side effects of completely changing the $parent_langcode value instead. So I took the more careful way to introduce a new variable instead. I fear, I'm not that well inwrought into IEF to decide this, but imho we should try to reduce this to one variable, if possible.

Before refactoring, the tests should be created first, as the current patch is at least tested by hand and already in production use on some sites.

This is looking better and better. However, I have one issue with translation: when I create a new translation of a parent entity, alter nothing, and save the translation, the child entities (using inline entity form) do not have corresponding translations--they still are only in the source language. All fields, including the entity reference, are translatable. Expected behavior: if I create a translation of the parent entity, the child entities should automatically have a translation to the same language if it does not already exist.

Note: I can work around this by manually editing each child entity.

I understand that an argument could be made that other entity references are not translated automatically, so why should IEF be different? I contend that since the typical use case of IEF is as a means for creating a composite field, that it should be treated the same as other translatable fields, and translations of the entities should automatically be created as long as the field itself is marked as translatable.

Thanks for the good work so far!

Reroll of patch.

We recently spent a lot of time in paragraphs to improve the multilingual handling, you might want to compare with #2698239: Multilingual workflow bugs. There are special cases when for example the language of the host entity is switched.

I think $parent_langcode is irrelevant, instead we should use $original_langcode (original language of entity).

This fixes the case where entity is added to translation of main node, and then entity is translated in the original node language.
In this case $parent_langcode and $target_langcode are both "en" which causes crash.

Could you provide an interdiff please?

The old patch from 32 does not apply anymore to current dev.
I don't think I can create an interdiff in that case, can I?

Patch code was not changed. It as just adapted to latest dev.
Edit: No, I replaced $parent_langcode with $original_langcode as I mentioned before.

Here is a use case I'm working on:

If reference field is not translatable (which means I want to add content to each language independently), but subentity is translatable. E.g. I can add different IEF entities to each translation of parent node.

Do you think it's a valid use case?

This is currently not working with this patch.

Sorry being unprecise. Yes, I was interested in the diff between your patches. If $parent_langcode -> $original_langcode is the only difference, I guess it's ok without interdiff (although they are always very handy)

Regarding the use cases: Maybe I misunderstand, but it sounds, you've mixed a little bit the scenarios.

I believe, that the default use case is to have the parent (node) entity and the subentity translatable, but the reference field not translatable, like you mentioned. And IEF has to take care, that always the right subentity gets edited/created/loaded.

Example:

1. Add a new node in EN, adding an sub entity with IEF
2. the sub entity also gets created in EN
3. Translate your node to DE and save
4. IEF has to take care, that the referenced entity also gets translated to DE

BUT of course IEF must also be able to handle other use cases, where the reference field is translatable. In this case, you can add different entities to each translation, independent from each other.

In my original patch I was concentrating on the first use case, however the second should have also worked correctly. It's a good idea, like I mentioned in a previous comment, to simplify and reduce the different langcode variables. Like berdir mentioned in #36, we should orientate on Paragraphs, how the translation handling was solved there.

At the end, we'd need a couple of unit tests to ensure that every possible use case is working as expected. However, every working patch would be a good start and great help for all users of IEF.

Of course there's still work to be done, but I'm setting the status to "needs review", in order to have the existing tests executed

You've described it very elegantly in #41.

The main use case where parent node and entity are translatable, but reference is not - I think it works very well now.

What I'm trying to do it is the other use case. Where reference is translatable. I think it also works for the most part.
The only problem is with entities translated from original node (when node translation is created), those become translations, not independent entities.
Meaning they will be deleted from all languages when deleting from one.

I agree about adding tests and taking experience from the Paragraphs fixes. But for now I just want to get it working.

Ok, I believe in the 2nd case, it would be best to always clone the referenced entity, so that it really becomes a separate entity, not only a translation.

I believe, that most of the time, when you make the field translatable, the referenced sub entity often won't be translatable at all, as it is a sub entity of an already translated field, and therefore its own language does not matter at all, until you display the same entity additionally somewhere else on the site independently from the parent node. But this is really an edge case. I've thought about this and still can't find a realistic example that makes sense at all.

@idflood in #14 tried to modify the patch to cover this use case. However it completely broke use-case 1, like I explained in #17. Maybe it helps you anyway - but the better role model is for sure Paragraphs.

I agree about adding tests and taking experience from the Paragraphs fixes. But for now I just want to get it working.

Absolutely agree, but I'm still hoping, that anyone except bojanz (please keep working on Commerce checkout ;-))) ) soon finds time for the tests too. The permanent changes and refactoring over the last couple of weeks leaded to so many necessary patch re-rolls. I hope, that we can achieve the final solution soon. Thanks to you I didn't have to do that this time... ;)

Ok, I think this does the trick. At lease I've tested both cases.

If reference field is translatable, we clone current entities when translating original node.

@mikgreen, did you mean for this to get tested?

Just a hint, we have been working on making Paragraphs ready for a stable release and learned that we were wrong again after multilingual support was "completed"...
We thought we have great test coverage and covered "all" multilingual cases when many issues went green.
Then we realised that all multilingual consistency, revision writing, validation, deep nesting, and many more were still wrong and needed to add tons of fixes and improve test coverage even more, eating even multiple weeks of developer time again.

So even after extensive review, expect that there will be many follow-ups and hidden areas in multilingual support and something stable needs multiple iterations.

@mikeryan thanks.
We still need to work on the patch.
Need to see what we can gather from Paragraphs translation experience.
Need to write tests.

It's unfortunately too late for the current versions, but imho it would be a great idea for future versions (D9 or different D8 branch) to refactor both codebases and remove redundancies. Both need to do the basically the same form handling stuff. Both run into the same hurdles and problems. A well working IEF could be the base for Paragraphs, which add their own forms, extending the basic IEF one

Working on this.

Initial version, written from scratch after reviewing the Paragraphs implementation.

Notable pieces:
1) TranslationHelper::prepareTranslation() ensures that the inline entity is being edited in the right language
2) When translating, the Complex widget hides the add and remove buttons, you can only edit what's already there.
3) When translating, the shared (non-translatable) fields are hidden from the inline entity form.
4) The complex widget enters the translation mode only if at least one of the allowed bundles has translations enabled
5) InlineEntityFormBase::removeTranslatabilityCue()

Automated tests in progress.

Known issue: Inline entities are created with the wrong langcode if you create the parent entity with a non-default language (by choosing a different language in the dropdown).

Committed a piece in http://cgit.drupalcode.org/inline_entity_form/commit/?id=89584bc.

Fixed the issue mentioned in #54.
Here's the latest version of the patch.

Added tests and a few tweaks, committed. See you in followups :)

I'm still researching this, but it looks like this makes IEF widgets fail if the 'parent' entity is newly created in a language other than the site's default.

Patch uploaded to #2747781: Entity reference translation breaks adding new parent entities in non-default languages.

I have created a follow-up issue for the known issue mentioned by bojanz in #54: #3218441: Inline entities are created with the wrong langcode if you create the parent entity with a non-default language.