Hi there,

It would be nice to have multilingual support for the migrate module.
The attached patch adds a destination handler for migrate which takes care about translations.
It's not a really elegant nor a fast solution - but it works. (At least in my test cases ;) )

The patch adds support for node and content based translation.

Files: 
CommentFileSizeAuthor
#36 entity_translation-migrate_support-929402-36.patch4.35 KBbrockfanning
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]
#33 translation-migrate-support-929402-33.patch4.99 KBgreenjuls
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]
#32 translation-migrate-support-929402-32.patch5 KBsteinmb
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]
#30 translation-migrate-support-929402-30.patch5.67 KBmake77
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-30.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#12 translation-migrate-support-929402-9.patch5.15 KBbastnic
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_1.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]
#9 translation-migrate-support-929402-9.patch5.71 KBbastnic
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_0.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]
#7 translation-migrate-support-929402-7.patch4.46 KBdas-peter
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-7.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]
translation-migrate-support.patch4.45 KBdas-peter
PASSED: [[SimpleTest]]: [MySQL] 0 pass(es).
[ View ]

Comments

plach’s picture

Hi Peter, thanks for working on this.

I'm afraid I'll have to won't fix this issue (and any other similar one): the reason is Content Translation aims to replace a core module, hence adding dependencies to contrib modules might render core inclusion more difficult. An exception is the Title project which will have to make its path into core too, somehow. Any other module with strong possibility to get into core might be taken in consideration for integration, though.

I'd like sun's feedback about this before actually closing this issue, but IMO the right way to go is either moving this feature request to the Migrate queue or creating a separate project.

das-peter’s picture

I fully agree with you. This is nothing that has to go into core, I'll ask mikeryan (migrate maintainer) how we can handle this - because an own project would be just overhead.
I'd like to see better multilingual support in migrate itself anyway :)

sun’s picture

+++ includes/translation.migrate.inc 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,106 @@
+        // Load old node to make sure it's the right id - mapping of migrate is not language sensitive
+        $current_node = node_load($entity->nid);

But why is this node-centric? (Sorry, if this is natural for Migrate module; didn't work with it yet)

Powered by Dreditor.

das-peter’s picture

Thanks sun, you're right!
Looks like I mixed some stuff - beside the fact that it's ugly like hell ;)
I hope all this can be cleaned by changing some stuff in migrate itself.

sun’s picture

Status:Needs review» Postponed

Alright, I hope you don't mind if we mark this issue postponed for now then. I think we have loads of more important, "basic" todos for this project currently, and contrary to those, this issue sounds fairly advanced to me. I'd be happy to revisit this at some point, because I've heard that Migrate becomes a very important solution, so we logically should make a best effort to support it. But yeah, "later". :)

das-peter’s picture

No problem - I need it anyway and it doesn't really matter where it resides. Thus I'll continue this in my project specific code. As soon as it's "later", I'll bring it back ;)
Btw: Let me know if you have some workpackages of the basic todos. I use the module already and I'm waiting for more feedback on #924968: Initial work - what's somehow related to this module. Means I'm interested in working on multilingual stuff ;)

das-peter’s picture

StatusFileSize
new4.46 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-7.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

I'll try to keep my patch here up to date.

plach’s picture

Project:Content translation» Entity Translation
Version:7.x-2.x-dev» 7.x-1.x-dev
bastnic’s picture

StatusFileSize
new5.71 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_0.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

A quick update on this patch, seems to works with my use case.

bastnic’s picture

Status:Postponed» Needs review

Status:Needs review» Needs work

The last submitted patch, translation-migrate-support-929402-9.patch, failed testing.

bastnic’s picture

StatusFileSize
new5.15 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_1.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

reroll without the path of my project

bastnic’s picture

Status:Needs work» Needs review

Status:Needs review» Needs work

The last submitted patch, translation-migrate-support-929402-9.patch, failed testing.

donquixote’s picture

Yeah, I want this!

@das-peter,
how is this designed to work? Is one row = one language, or is one row = all languages at once?
If one row = one language, what is the primary key? As far as I remember, primary keys in migrate destinations have to be single-value integers, or not?
I guess, as you did not add any destination class (just a destination handler), it is one row = all languages at once. So the field handler needs to deal with arrays as field values.
Do rollback and --update work?

@plach, sun,
what about node_save() with entity_translation enabled, does this save all languages or just one translation?

plach’s picture

what about node_save() with entity_translation enabled, does this save all languages or just one translation?

All field translations available in the $entity data structure are saved to the storage, It's the standard core behavior.

donquixote’s picture

@das-peter / bastnic:
One thing I still don't get...

<?php
   
if (entity_translation_enabled($entity_type) && property_exists($entity, 'nid') && $entity->nid) {
?>

This will only work, if the $entity->nid is already set.
So, either during an --update run, or if the migration (class) that imports the translations is a different one than the migration (class) that created the node itself.

How do your migration classes look like, that use this handler?
How do you get the translations from the source db into $entity in the first place?

I suppose, at the time that MigrateTranslationEntityHandler::prepare is called, this has already happened, and now the translations sit in $entity->translations->data[$lang] ?

donquixote’s picture

I'm getting a little confused, what the EntityTranslationDefaultHandler::setTranslation() method actually does.
In fact, it is only writing on the $entity object, which it keeps as a protected variable.
Is there any documentation about this stuff?

KarenS’s picture

Just a note that this could be a really useful feature to create a system to bulk update translations. As noted in another issue, lots of translations are done offsite and the 'translation' work is mostly doing lots of copy/paste into the forms. How much nicer would it be to ask for translations to be provided in something Migrate can use (XML or a spreadsheet or a database), map the source to the site, then use Migrate to bulk update everything at once? So getting Migrate working properly could be very useful.

bastnic’s picture

@donquixote indeed, i began to save the node only in english in a first import, and then I do what @KarenS suggest to import all translations in another migrate import class.

das-peter’s picture

I'll need some time to get into this again.
As far as I remember I changed again some stuff to keep it working.

The approach I've used was to have a row per language. Where only the translatable content differs from row to row.

plach’s picture

I'm getting a little confused, what the EntityTranslationDefaultHandler::setTranslation() method actually does.
In fact, it is only writing on the $entity object, which it keeps as a protected variable.
Is there any documentation about this stuff?

That method just sets the updated translation records in the translation handler. These are subsequently stored through EntityTranslationHandlerInterface::saveTranslations().

donquixote’s picture

Ok.
But saveTranslations() is not called in the patch in #12. Is this triggered by something else?

plach’s picture

In entity_translation_field_attach_update() through node_save().

bastnic’s picture

I'm coming back on this ticket with a beautiful WTF effect. I really really did see my patch working but not anymore.

First :

      // Content based translation
      if (in_array($entity->type, variable_get('entity_translation_entity_types', array()))) {

can't work, we should have

      // Content based translation
      if (entity_translation_node_supported_type($entity->type)) {

Second,

// Preserve original language setting
$entity->language = $entity->translations->original;

... cause entity_translation not to work with (I think) migrate 2.3. So I did that:

--- a/entity_translation/includes/translation.migrate.inc
+++ b/entity_translation/includes/translation.migrate.inc
@@ -61,6 +61,7 @@ class MigrateTranslationEntityHandler extends MigrateDestinationHandler {
           );
         }
         // Preserve original language setting
+        $entity->field_language = $entity->language;
         $entity->language = $entity->translations->original;
       }
       // Node based translation

--- a/migrate/plugins/destinations/fields.inc
+++ b/migrate/plugins/destinations/fields.inc
@@ -91,6 +91,8 @@ abstract class MigrateFieldHandler extends MigrateHandler {
         return LANGUAGE_NONE;
       case isset($arguments['language']):
         return $arguments['language'];
+      case !empty($entity->field_language) && $entity->field_language != LANGUAGE_NONE:
+        return $entity->field_language;
       case !empty($entity->language) && $entity->language != LANGUAGE_NONE:
         return $entity->language;
         break;

I sincerely have no idea why my previous patch had ever worked!

sinasalek’s picture

Component:Code» Base system

I replaced name and description of a vocabulary and i can longer migrate to it.
This is the error i'm getting

SQLSTATE[21S01]: Insert value list does not match column list: 1136 Column count doesn't match value count at row 1: INSERT INTO {field_data_description_field} (entity_type, entity_id, revision_id, bundle, delta, language, description_field_value, description_field_summary, description_field_format) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6_0, :db_insert_placeholder_6_arguments, :db_insert_placeholder_7, :db_insert_placeholder_8); Array ( [:db_insert_placeholder_0] => taxonomy_term [:db_insert_placeholder_1] => 9783 [:db_insert_placeholder_2] => 9783 [:db_insert_placeholder_3] => food [:db_insert_placeholder_4] => 0 [:db_insert_placeholder_5] => en [:db_insert_placeholder_7] => [:db_insert_placeholder_8] => plain_text [:db_insert_placeholder_6_0] => [:db_insert_placeholder_6_arguments] => Array ( [format] => filtered_html ) ) (modules/field/modules/field_sql_storage/field_sql_storage.module:448)
radiobuzzer’s picture

Hi, until this is resolved, I have to mention how I did it, in case this helps anybody. I used the "prepare" method of the Migrate api, which is added in the end of any Migration class.

In this example I am migrating two fields, name_en and name_el into one translatable field. I also know that the default language is 'el' for all of the nodes, but this can be easily changed

<?php
class RegionMigration extends Migration{
  public function
__construct() {
    ...
  }
  function
prepare($entity, stdClass $row){
   
$entity->language = 'el';
   
$entity->title_field['el'][0]['value'] = $row->name;
   
$entity->title_field['en'][0]['value'] = $row->name_en;
   
$entity->translations = (object) array(
           
'original' => 'el',
           
'data' => array(
                   
'el' => array(
                           
'entity_type' => 'node',
                           
'entity_id' => $entity->nid,
                           
'language' => 'el',
                           
'source' => '',
                           
'uid' => '0',
                           
'status' => '1',
                           
'translate' => '0',

                    ),
                   
'en' => array(
                           
'entity_type' => 'node',
                           
'entity_id' => $entity->nid,
                           
'language' => 'en',
                           
'source' => 'el',
                           
'uid' => '1',
                           
'status' => '1',
                           
'translate' => '0',

                    ),
            )
    );
    return
$entity;
  }
}
?>
primozsusa’s picture

Hi, did anybody solved or has an example of using migrate with entity translation field. Maybe field handler example... #27 works kind of. if you have one record with multiple languages for import. but for taxonomy term with #27 the problem is that the data is not saved the same as if it would be done manually.
any suggestions welcome.
thx Primoz

mvdve’s picture

You probably want to look at Issue 1069774.
There is complete description of how to create the translation. This can be done in the complete function.

make77’s picture

Issue summary:View changes
Status:Needs work» Needs review
StatusFileSize
new5.67 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-30.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Hi,

I updated the patch for the latest version of the module (7.x-1.0-beta3).

Status:Needs review» Needs work

The last submitted patch, 30: translation-migrate-support-929402-30.patch, failed testing.

steinmb’s picture

Status:Needs work» Needs review
StatusFileSize
new5 KB
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]

Broken patch. This patch apply cleanly on dev. Let's see if bot also is happy.

greenjuls’s picture

StatusFileSize
new4.99 KB
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]

Hi, I just updated to version 7.x-1.0-beta3 of entity_translation module but it broke my migrations.
Problem is that the translations were not loaded in MigrateTranslationEntityHandler::prepare().

I replaced the following piece of code

<?php
// Load translations if necessary
if (!property_exists($entity, 'translations')) {
 
$entity->translations = $translation_handler->getTranslations();
}
?>

with:

<?php
// Load translations
$translation_handler->loadTranslations();
?>
plopesc’s picture

Hello
I'm trying to use migrate and enityt_translation with no luck, and I think this patch can be helpful.
Could you include an example of how are you implementing it?

Thank you in advance

primozsusa’s picture

What finally worked for me:
- make sure your filed is set translatable in UI
- mapping

   
    $this->addFieldMapping('language')->defaultValue('en');
    $this->addFieldMapping('translate')->defaultValue(true);
    $this->addFieldMapping('field_tk_description', 'desc_en');
    $this->addFieldMapping('field_tk_description:format')->defaultValue('nc_filtered_html_editor');
    $this->addFieldMapping('field_tk_description:language')->defaultValue(array('en', 'sl'));

- prepareRow

  public function prepareRow($row) {
    if (parent::prepareRow($row) === FALSE) {
      return FALSE;
    }
    $row->language = array('en', 'sl');
    $row->opis_en = array($row->desc_en, $row->desc_sl);
    return TRUE;
    }

- prepare

  public function prepare($entity, $row) {
    //$entity->language = 'en';
    // this is not needed because $row->desc_en = array($row->desc_en, $row->desc_sl);
    // is already working from prepareRow
    //$entity->field_tk_description['sl'] = $entity->field_tk_description['en'];
    //$entity->field_tk_description['sl'][0]['value'] = $row->desc_sl;
  }

- complete: to update entity_translation table

  public function complete($entity, stdClass $row){
    $entity->language = 'en';

    $handler = entity_translation_get_handler('node', $entity);
    $translation = array(
      'translate' => 0,
      'status' => 1,
      'language' => 'sl',
      'source' => $entity->language,
    );
    $handler->setTranslation($translation);
    $handler->saveTranslations();

    return $entity;
  }

brockfanning’s picture

StatusFileSize
new4.35 KB
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]

I may be missing something, but the latest patch in #33 didn't seem sufficient for my needs. It's limited to nodes, and also limited to existing nodes, which I don't think was intended. It also has a section about supporting "node-based translations" which I'm not clear on, as I thought that entity_translation was the alternative to node-based translations. Also I don't believe it is registering the destination handler correctly.

Given all that I wrote a new patch that is working for me.

The way I'm using it is to map an array of language codes to each translatable destination field's ":language" subfield, and a corresponding array of values to the field itself. For example, something like this in the migration constructor:

<?php
$this
->addFieldMapping('field_subtitle', 'legacy_subtitle');
$this->addFieldMapping('field_subtitle:language', 'legacy_subtitle_languages');
?>

Where $row->legacy_subtitle might be...

<?php
array(
 
'My legacy subtitle',
 
'Mon héritage de sous-titres',
);
?>

And $row->legacy_subtitle_languages would be...

<?php
array(
 
'en',
 
'fr',
);
?>

Also in combination with migrate_d2d and this patch: https://www.drupal.org/node/2389783 the mapping can be as simple as (assuming a D7 multilingual to D7 multilingual migration):

<?php
$this
->addFieldMapping('field_subtitle', 'field_legacy_subtitle');
$this->addFieldMapping('field_subtitle:language', 'field_legacy_subtitle:language');
?>
heldercor’s picture

@brockfanning, doesn't this break fields with multiple values?

brockfanning’s picture

@heldercor, not that I know of, though in my testing all translatable fields have been single-value. In theory it would be the same, but the $row value would be an array of arrays, like:

<?php
$row
->legacy_numbers = array(
  array(
   
'one',
   
'four',
   
'seven',
  ),
  array(
   
'uno',
   
'cuatro',
   
'siete',
  ),
);

$row->legacy_numbers_language = array(
 
'en',
 
'es',
);
?>

But as said, I haven't tested that.