Multilingual support in field_collection 8.x

I believe #2 is the more common use case, and it is the only one that I personally have encountered. Consider the example mentioned in the other ticket about this... "If I have a field collection with an image and a text, I may want to only translate the text field." It wouldn't be possible to implement that while creating an independent group of field collection items for each of its host's translations. #2618156: Translatability settings on the content translation page

There may be some value in adding the option for #1 as well, but it would be more difficult to implement and use the module if they were both supported, especially when considering nested field collections that may not use the same option as their parent.

#1 is (or will be) supported by using content translation, so it may not be necessary to support it in entity translation too.

Quick qeustion: were there any work done on field collection translatability problem to pick up from or is it in the process of consideration and planning?
Reason to ask, have a project with field collections (in some cases 2 level deep) and tmgmt. It's important for the client to have all content translations to be exportable/importable.
Maybe someone has any solutions for this?

There has been no specific work done to add translation support to 8.x-1.x.

8.x-2.x is essentially a rewrite, so a separate issue can be made for it.

Unless there are any objections we can pursue option #2 here.

Come to think of it, the two options shouldn't be too hard to combine from a UX perspective, even if that doesn't make the implementation any easier. The thing to enforce would be if a field collection field is translatable then none of the fields inside it (including nested field collections and their subfields, no matter how deep) could be translatable.

The way I see it, the path to implementing translation for field collections looks like:

1. Remove the option to make a field collection field in a host entity translatable.
2. Make field collection entities translatable. (ie. so they appear on /admin/config/regional/content-language)
3. Get the field collection entities and fields saving in the same language as their host. (#2618018)
4. Allow translatable fields in the field collections to be translated. (Probably the hard part. May help to split up this task into embedded form and separate field collection forms.)
5. Get the correct language displaying for the fields.
6. Automate the translation settings on /admin/config/regional/content-language for field collection entities if possible. Ideally some of these settings should be taken from host entity types and not be modifiable by users.
7. Add information for any automated settings that explains to the user why they can't make such changes.
8. Add checks to prevent a situation where a field collection field could be made translatable while something nested in it is also translatable.
9. Add support for translatable field collection fields if necessary.
10. Re-add the option to make a field collection field translatable.
11. Add help text / documentation about the differences between making field collection fields translatable and making their subfields translatable.

The image module provides translatability options for its built in subfields such as title and alt. These options appear on the general translatability settings page and the settings for the field itself. As a UI it seems to provide a reasonable way to set the options, though I don't think we'd want to copy the behaviour all the way through because even when the files themselves are set as not translatable the image fields are still allowed to have completely separate sets of images for each translation of a piece of content. If we are to make the inner fields in a field collection translatable we'd want to enforce a 1 to 1 pairing between field collection items in each translation.

I've done some work towards fc multi-language.
Run /update.php for existing content (please make sure you have DB backup).
As far as I've tested it's working on fresh install (deletion needs to be tested better).
Please review and feel free to make any kind of remarks.

Thanks @dramblys. It looks like a lot of functionality is here.

I'm interested in looking more at what this will do. A few general comments:

• Update functions should be in .install files.
• Most or all of these updates should not be run as queries but functions of \Drupal::entityDefinitionUpdateManager() should be used instead.
• Field collections can be applied to any fieldable entities, not just nodes. To handle this references to node should be changed to host and all entity types using field collections should be included.

For more details see:

• Overview of .install files
• Documentation for hook_update_N

Can you explain a bit about how your solution handles translating the field collection fields in the host entity vs translating the inner fields as per #2, or where there this gets us with respect to the plan in #8?

Please note #10 patch is good for fresh site installs only, working on the implementation for sites with existing fc content. Also I've noticed a floss: e.g. when deleting node translation it deletes the field collection, not the translation of it.

@jmuzz Thank you for remarks, working on it ;)

What this patch does:
Regarding #2
1. Translate the field collection items as its own entity
Regarding #8
1. No
2. Yes
3. Yes
4. Yes
5. Yes
6. Manual as it is for all other translatable entities
7. No
8. No check, though nested field collection item are translatable as well
9. Not sure what you meant by that
10. Again, not sure
11. Nothing have been documented till final tested release

And I need to do more testing - working progress ;)

Some more work done. Everyone feel free to review, test, make remarks.

Notes
field_collection-multilanguage-2618016-2.1-8.1.x.patch - this one is for changes in the module itself for translatability on new fresh installations.
field_collection-multilanguage-2618016-2.2-8.1.x.patch - this is basically .install file for updating projects with content in field collections (FC).

Instructions
For new installations: apply field_collection-multilanguage-2618016-2.1-8.1.x.patch.
For projects with content in FC:
1. Backup your DB.
2. Make sure you can rebuild cache and run update without being logged in or using admin panel.
3. Apply both patches.
4. Rebuild cache
5. Run /update.php

After some testing, working version:

Hi dramblys,

Unfortunately your patches (3.2) doesn't work for the site of my client. I'm getting the following error: "Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1052 Column 'langcode' in field list is ambiguous:" etc etc. See everything: http://pastebin.com/XftHEumh

Can you please help me? Thanks in advance.

Diederick

EDIT: ok, it's fixed. Don't used your patches. Instead I'm deleting the field collection after I'm making a translation of a page.

Hi Dramblys,

I'm seeing issues with your patch as well, once I enable translation for field collection content it seems to permanently break the field collection content and the node cant be saved again due to the following error

Drupal\Core\Entity\EntityStorageException : SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'field_name' cannot be null: UPDATE {field_collection_item} SET item_id=:db_update_placeholder_0, revision_id=:db_update_placeholder_1, field_name=:db_update_placeholder_2, uuid=:db_update_placeholder_3, langcode=:db_update_placeholder_4 WHERE (item_id = :db_condition_placeholder_0); Array ( [:db_update_placeholder_0] => 250 [:db_update_placeholder_1] => 250 [:db_update_placeholder_2] => [:db_update_placeholder_3] => 8e80c44f-d264-4dee-9596-fd00a4b3c527 [:db_update_placeholder_4] => en [:db_condition_placeholder_0] => 250 ) dans Drupal\Core\Entity\Sql\SqlContentEntityStorage->save() (ligne 770 de /var/www/vagrant/docroot/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

I've tried this with a fresh install of fc with your patch, creating new field collections etc... and i always end up with the same issue. Have you experienced this at all?

Hi Dramblys,
Ignore the above comment, it was due to interaction with another module. One real issue i did note however is that the field collection translation does not 'clear up' its old values when you delete it. They do get deleted successfully when the entire field collection is removed, but the current behavior means that if you delete and create a new translation you will get all the old field collection values added, which appears a little odd from a content management point of view.

@dvanbrenk will try to have a look today/tomo
@Tom.W I'm glad that someone is using mu implementation ;) Regarding the issue of translation delete functionality - you are absolutly right, I've spent some time to fix it, but had no more time for it so far.
If there would be more interest in the further development of this implementation I could spend more time to fix it.

There indeed is.

The update doesnt work for existing field collections.

I know this might seem like a nit. But the code style in the patch for upgrading is pretty far from normal Drupal projects. Any chance we can get something that more closely aligns to ? It make reviewing extremely complicated.

Some comments below and here's a link to the coding standards: https://www.drupal.org/coding-standards

1. +++ b/field_collection.install
   @@ -0,0 +1,295 @@
   +  __fc_tables_create($db);
   +  __fc_columns_update($db);
   

   Why double underscores?

2. +++ b/field_collection.install
   @@ -0,0 +1,295 @@
   +function __fc_assign_children(&$fciItem, &$fciArray) {
   

   Where is the function docs?

3. +++ b/field_collection.install
   @@ -0,0 +1,295 @@
   +  /**
   +   * Recursive check for field collection in field collection
   +   */
   

   This type of doc block normally goes on the function. Inline comments are formatted differently.

4. +++ b/field_collection.install
   @@ -0,0 +1,295 @@
   +  $q = "RENAME TABLE field_collection_item TO {$prefix}field_collection_item";
   +  $db->query($q);
   +  $q = "RENAME TABLE field_collection_item_revision TO {$prefix}field_collection_item_revision";
   +  $db->query($q);
   ...
   +    $q = "UPDATE field_collection_item AS fci SET fci.langcode = '{$langCode}' WHERE fci.item_id = '{$fcId}'";
   +    $db->query($q);
   

   This will only work on SQL backed installs of Drupal. Why can't we use the query api provided in D8?

Also, the docs in https://www.drupal.org/node/2535476 are probably worth a read.

Going to take a stab at some of this feedback today.

Development is moving to 8.x-3.x.

About the deletion issue: If the field collection fields themselves are translatable instead of the inner fields they will need to be completely separate datasets for each translation. This means cloning the field collection objects when a translation of their host gets made. If you want to have each translation pointing to the same field collection items and with each item having multiple translations so the inner fields can be translated that does mean that deleting a field collection item in one translation will delete it in all of them.

OK, very little testing, but here's some changes to the hook update. My testing to this point shows that nothing blows up on update. Interdiff is against only the base functionality in #14 as there is nothing in common on the hook update.

In a more than two language setup, with a node already translated in two languages(e.g. english and french):

If you change the source language during node translation (to e.g. german) from default source language (e.g. english) to another language (e.g. french), the field content of the field_collection fields stay in the default language(english). The field content should be loaded from the selected language( french).

Non field_collection fields show the right field content in the selected language(french).

1. @heddn I added source language support(#29) to the field_collection-multilingual_support_in-2618016-27.patch patch

-              $field_collection_item->addTranslation($langcode, $field_collection_item->getFields(TRUE));
+              if (\Drupal::request()->attributes->get('source') !== NULL){
+                $sourcelangcode = \Drupal::request()->attributes->get('source')->getId();
+                $field_collection_item->addTranslation($langcode, $field_collection_item->getTranslation($sourcelangcode)->getFields(TRUE));
+              }else{
+                $field_collection_item->addTranslation($langcode, $field_collection_item->getFields(TRUE));
+              }

2. in case you delete a node translation(#12) all field_collections of all translations of this node are deleted. Only delete the current translation:

-      $field_collection_item->field_collection_deleting = TRUE;
-      $field_collection_item->delete();
+      if ($field_collection_item->isTranslatable()){
+        $langcode = explode('/', \Drupal::request()->getPathInfo())[1];
+        $field_collection_item->removeTranslation($langcode);
+      }else{
+        $field_collection_item->field_collection_deleting = TRUE;
+        $field_collection_item->delete();
+      }

I'm trying to apply those patches but can't get it works.
First of all which version are they made for ?
It seems that's for 8.x-1x-dev but update script crash with this exception :
LogicException: The Field Collection Item entity type cannot be translatable as it does not define a translatable "langcode" field.

As comment #24 says development moved to 8.x-3.x I also tried on this version. Update goes well but I couln'd create or even see nodes with FCs in it.

Does someone manage to make it works ?

@paulguy I wasn't able to apply them either, nor find which point in git history were those patches written against, so I ended up merging them manually with the latest 8.x-3.x-dev but it still needs some extra work to actually work. I'll post a patch as soon as I have it working.

Okay, a followup on my previous comment (just for political correctness ;) ).... it looks like those patches were written against 8.x-1.0-alpha1 which is no longer in use however.

Thanks for your answer. In a way I'm glad to know that I didn't miss something when I tried...
We ended to create a new node for each language when collection were used. But that's a quick & dirty solution so let me know if you create a new patch and i'll be happy to test it.

Have a look at Inline Entity Form, and in particular #2494959: Add translation integration. Translation is (partially) working there, It looks like good example code.

Attached is the re-rolled patch against latest 3.x-dev. it's not exactly working, but that's what is initially ported. I'm still trying to get working version soon.

Now translations work, but I'm not a big fan of using path for how deleting translations is originally written, I'm looking into other possible ways, because what if we are deleting a translation programmatically, or as part of a batch job ...

Okay, after a little bit more testing, I found the following piece of code in ContentEntityStorageBase::invokeFieldMethods (which basically is responsible for invoking \Drupal\field_collection\Plugin\Field\FieldType\FieldCollection::delete

  // For non translatable fields, there is only one field object instance
      // across all translations and it has as parent entity the entity in the
      // default entity translation. Therefore field methods on non translatable
      // fields should be invoked only on the default entity translation.
      $fields = $translation->isDefaultTranslation() ? $translation->getFields() : $translation->getTranslatableFields();
      foreach ($fields as $name => $items) {
        // call_user_func_array() is way slower than a direct call so we avoid
        // using it if have no parameters.
        $result[$langcode][$name] = $args ? call_user_func_array([$items, $method], $args) : $items->{$method}();
      }

what that practically means is: fieldType delete() methods are never called for fields that are not marked as translatable ... that makes a lot of sense, but not for fields that reference entities that are editable on the parent entity form (e.g. FieldCollection, Paragraphs, IEF ... ) actually I could reproduce the same behavior with paragraphs as well, and I'm filing a separate bug there, but it's certainly worth thinking whether we should implement another method to hook into the deletion of parent items translation and delete corresponding FC translations or is that just kind of an overkill? thoughts are really welcome here.

FooZee this patch works with existing nodes?

Hi all,

Is the patch field_collection-multilingual_support_in-2618016-37.patch work and is ti work with production site with content in fc ?

Is it the only file to appply ?

Reagards

Hi FooZee,

I would love to have Translation working with Field Collections in Drupal 8. So we think about the propper "delete()" method, and then we can maybe commit that into the actual "8.x-3.x-dev" branch of the module? That would be great. Are you still working on the issue and want some opinions about your question? If yes, I would really like to help out!

Best regards,
CreativeClaire

Updated the issue summary with more details of the required situation which we call "translatable".

Delete function is updated without getPathInfo(). Patch against 8.x-3.x-dev(2016-Nov-21)

Sorry, just jumping in after going through the comments/patches
Somehow, the schema definition for field_collection_item_field_data got lost around #37, which will cause the following error when trying to translate items.

The website encountered an unexpected error. Please try again later.

Drupal\Core\Entity\EntityStorageException: Exception thrown while performing a schema update. Cannot add field field_collection_item_field_data.content_translation_source: table doesn't exist. in Drupal\Core\Entity\Sql\SqlContentEntityStorage->wrapSchemaException() (line 1456 of core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

Hopefully I can provide a patch, but I'm currently rushing towards a deadline

Well, that was short lived, I added the schema back in from the 30 re-roll patch, was able to add translatability to my field collections, but as soon as I went to the front end, I got the following message

The website encountered an unexpected error. Please try again later.

Drupal\Component\Plugin\Exception\PluginNotFoundException: The "" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 125 of core/lib/Drupal/Core/Entity/EntityTypeManager.php).

Unfortunately I don't have any time to keep at this currently, but I hope it helps figure out some of the issues, sorry!

@sleitner: The 'delete()' method doesn't seem to work properly. I'll explain (1) What I did and (2) What the error is.

(1) What I did:

• Create a content type with a "Field Collection" field in it
• Made the fields in this field collection translatable. The translation functionality actually works pretty well. I created two translations. The first one (the original) was German, the second one English.
• I created a node of that content type and translated it. Also worked pretty well.
• Then I wanted to delete this node again. And that's where the errors happen...

(2) What's the error:
When I delete the secondary translation (English in my case), that works well. But when I try to delete the primary translation (German in my case), that happens:

The website encountered an unexpected error. Please try again later.

Drupal\Core\Entity\EntityStorageException: The specified translation (de) cannot be removed. in Drupal\Core\Entity\Sql\SqlContentEntityStorage->delete() (line 718 of core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

Drupal\field_collection\Plugin\Field\FieldType\FieldCollection->delete() (Line: 244)
Drupal\Core\Field\FieldItemList->delegateMethod('delete') (Line: 217)
...

That's not good, right? I used the "field_collection" module, version "8.x-3.x-dev" (2016-Nov-21) and the patch "field_collection-multilingual_support_in-2618016-45.patch". I'll try out patch "field_collection-multilingual_support_in-2618016-37.patch" soon.

Or did I do something wrong? Otherwise I hope my testing helps.

I had (the same) problem deleting a translated node with field collections. The attached patch fixed it for me.

First of all thanks for all the work done !

I can't get the update.php script working with the last patch.
field_collection_update_8003 is detected but anytime I try I get this error :

An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: /update.php/start?id=164&op=do_nojs&op=do
StatusText: Internal Server Error
ResponseText: 

With no other logs or error :( Am I the only one ? Maybe its not a FC issue but I can't figure out why...

@myself Ok with php error logs I saw that I forget to add the 2 use lines on the top of field_collection.install file ...
Patching a module manually is definetively not the good way to go !

More testing now

Editing a content with FC lead to an error "entity (X) doesn't exist", I had to recreate every FC.
Thats not a big deal but adding or editing a content with FC always lead to this error now :

The website encountered an unexpected error. Please try again later.

Drupal\Core\Entity\EntityStorageException: SQLSTATE[HY000]: General error: 1364 Field 'uuid' doesn't have a default value: INSERT INTO {field_collection_item_field_data} (item_id, revision_id, field_name, langcode, host_type, default_langcode) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array ( [:db_insert_placeholder_0] => 366 [:db_insert_placeholder_1] => 366 [:db_insert_placeholder_2] => field_bc_accordion [:db_insert_placeholder_3] => en [:db_insert_placeholder_4] => node [:db_insert_placeholder_5] => 1 ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->save() (line 770 of core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

Also I had to apply an update hook from patch #25 to get field_collection_item_field_data table.
My production site use FC 8.x-1.0 and i'm trying to update to 8.x-3.x with translations. Do I have to apply other patches thatn those of this page ?

Not to be a damper here, but for those coming to this issue, they might want to consider there is an alternative to field collections in D8. And paragraphs already supports multi-lingual.

#2784931: Proposal: Deprecate Field Collections for Drupal 8, focus on Entity Reference Revisions & Paragraphs
#2831131: [META] Outperform field collections

@paulguy, the patch does _not_ work with existing field collections. An upgrade path is not (yet) available.

@heddn: Thanks for the hint. Just tried it with the "paragraphs" module, translation works pretty well.

Here is the update script for existing 1.x field collections. It is working well for default language content, but I'm not completely sure about other languages. For now I have only default language content and few translated items. The translated items aren't looking good (missing or extra items), so I'll delete and re-translate them.
Apply it after #49.

The previous patch doesn't fix some items with empty host_type value, so here is the additional patch. Apply after #57. Run updates afterwards.

last amendments by @artreaktor fail for me.

drush8 updb -y
The following updates are pending:

field_collection module : 
  8001 -   Update database tables to support extended entity references. 
  8002 -   Update database to support entity references for pre-existing content types. 
  8003 -   Add content translation capability. 
  8004 -   Content migration 1.x => 3.x.    Update existing content in accordance to 3.x scheme. 
  8005 -   Content migration 1.x => 3.x.   Update items without host_type set. 

Do you wish to run all pending updates? (y/n): y
Performing field_collection_update_8001                                                                                                                                                                                                                                                                                [ok]
Performing field_collection_update_8002                                                                                                                                                                                                                                                                                [ok]
Performing field_collection_update_8003                                                                                                                                                                                                                                                                                [ok]
SQLSTATE[42S22]: Column not found: 1054 Unknown column 'langcode' in 'field list': UPDATE {field_collection_item} SET langcode=:db_update_placeholder_0                                                                                                                                                                [error]
WHERE  (item_id = :db_condition_placeholder_0); Array
(
    [:db_update_placeholder_0] => es
    [:db_condition_placeholder_0] => 1
)

Performing field_collection_update_8004                                                                                                                                                                                                                                                                                [ok]
Failed: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'langcode' in 'field list': UPDATE {field_collection_item} SET langcode=:db_update_placeholder_0                                                                                                                                                        [error]
WHERE  (item_id = :db_condition_placeholder_0); Array
(
    [:db_update_placeholder_0] => es
    [:db_condition_placeholder_0] => 1
)

Cache rebuild complete.    

@lathan, there should be "langcode" field in "field_collection_item" table according to 8.x-3.x version of FieldCollectionItem annotation

@artreaktor I can confirm that I get the same error as @lathan.

I tried out the patch + the patches posted by @artreaktor in #57, #58. I ran into a couple of issues, like #59, which, even if its fixed, causes more issues, see my review below.

1. +++ b/field_collection.install
   @@ -132,3 +134,25 @@ function field_collection_update_8002() {
   + * Add content translation capability.
   + */
   +function field_collection_update_8003() {
   +  $storage_definition['langcode'] = BaseFieldDefinition::create('language')
   +    ->setLabel(t('Language code'))
   +    ->setDescription(t('The field collection item language.'))
   

   This is missing the data / revision data table.

2. +++ b/field_collection.install
   @@ -132,3 +134,25 @@ function field_collection_update_8002() {
   +  \Drupal::entityDefinitionUpdateManager()
   +    ->installFieldStorageDefinition('langcode', 'field_collection', 'field_collection', $storage_definition['langcode']);
   

   This is the wrong entity type :)

Hello Team,

I am using drupal8 I am trying to translate filed collection item in views. but its is not translating. I use the same option. which i used for other content types field. please help me. i am stuck

Hello All,

I am working on a website with version Drupal 8.5.3 and using core multilingual module for translation when I try to translate that there was the problem in field collection areas, It translates both nodes same times. Normally new translated node/page is only linked and does not share same content if I change that, But in this case, when I change in English same content I am seeing in German page. If I change in German same content is seeing in the English page.

E.g I have a content type testimonials whose field is Title, Body and Field collection Meta Content whose fields are

Normal Content, Left Content, and Right Content. Now in this Title and Body is translating properly but Meta content field collection when I update content in one node the same content is showing in other language nodes.

Any help appreciated? How should we resolve the Issue?

#49 works for a fresh installation of FC.

@Sutharsan thanks.

I too miss multilangual support. If this missing issue wont be fixed, is there an alternative modul instead of field group? Just only need it to group on the page tags and further information links/downloads etc.

If the maintainer doesnt want to fix the missing feature translation, why should the community not to give it a further chance?

In D7 there was also work on a patch:

https://drupal.org/node/1344672
https://drupal.org/node/1316162

Strreamlined multilingual is one of the main progress of D8 so I wonder if there would be no solution for field collection.