TF #4: Translatable fields UI

Re-rolled patch, so that the testing bot will notice it.

Hmm, that didn't work. Re-re-rolled.

subscribing.

Excellent work. This patch illustrates the power of field-based translation and the relative ease of implementing an admin UI for the backend infrastructure achieved in #367595: Translatable fields.

While I briefly read over the code, this patch review is based on a demo site and focuses on the translation design and interface.

Rather than imagining a translation UI from scratch, the patch takes the approach of emulating what we already have in the translation module. I think this is the right approach. No doubt through a lengthy process we could evaluate and refine our translation interface, possibly completely redesigning it, but here isn't the place to do so. This patch wisely sticks closely to what we have.

Patch summary

The patch implements two new modules and a new locale include file.

The first module, entity_translation, provides a set of API functions for handling records related to the existence of translations of entities. E.g., when a node is translated, a record is created noting the entity type (node), the language of the translation, etc.

The second module is field_translation, parallel to translation module. Like translation, field_translation provides a tab on the node page alongside "Edit" for managing translations of the node. Here the tab is "Field translation" and, as with translation module, is shown when a node has a designated language. The field translation page for a node shows enabled content languages and links for creating new translations or editing existing ones.

The language of the original node is designated the "source" translation. Clicking on a link provides a form for editing a translation.

Comments, questions, and issues

1. Relation to translation.module. Implementing this in a new module separates the two models of translation (field-based and node-based). However, the interaction of the two models is something we need to think through. Unless we explicitly prevent it - which is not done in the current patch, and is in any case probably not a good idea - both models may coexist on a given site.

   This coexistence will be potentially confusing both to site admins and to end users. Particularly confusing would be the coexistence of both models for a given content type. Is there any scenario where we would want both models to coexist for the same content type? I.e., a translation set, each member of which can have fields that are translated - back? - into multiple languages? This would be a mess. It seems like we should prevent the coexistence of both models for a given content type.

   How? The cleanest way I can think of is to test the language_content_type setting. If it's TRANSLATION_ENABLED, we don't offer the Field translation tab, leaving translation to translation.module.

2. What fields to show on field translation edit form? Currently, the field translation edit form shows a mix of translatable and non-translatable fields. I found this confusing, particularly as there is no visual clue as to which fields are translatable. My inclination was to translate all fields on the form, but doing so produced an odd outcome: some changes were made to the original node (e.g., body), others to the translation.

   The simplest and I think most intuitive solution would be to show only translatable fields on the translation form.

3. Theming. Currently we are in the default theme rather than the admin one when viewing lists of existing translations and when creating/editing a translation. Presumably we want to be in the admin theme.

4. Meaning of "source". The term "source" is used in two different ways for field translation. First, the language of the node is termed the "source" language. To change this source language, we can edit the node and change its language. (I noticed that the available languages are filtered to ensure that we can't change the node to the language of one of the existing translations. Good call.)

   However, "source" is also used in a second sense, to describe the language that a translation is being created from (e.g., from English to French). This "source" can be changed on the translation overview page. At least, we probably need to use different terms for these two different concepts. But I don't know if I have a good suggestion. Maybe instead of "Source translation language" we could put "Create new translations from:".

5. Default text when creating a translation. I expected to find the existing source text given when I went to create a new translation (as is the case when creating a translation with translation module), but instead the field is empty. This looks like a bug. In translation module, hook_nodeapi_prepare_translation() is invoked to allow modules to provide such default values, which in some cases will differ from the source language's values. It looks like we need to invoke this hook.

6. Losing tabs. When creating or editing a translation, we lose our tab (local task) context.

7. Reusability for non-nodes. It's great to see the abstract entity_translation methods, ready to be applied to other entities. But does it merit a distinct module? Do we have any other pure API modules in core? Looking to field_translation, it's currently limited to nodes, but is that a necessary limit? My initial feeling is, no, field_translation should eventually enable translation of any fieldable entity. And so it seems to me that entity_translation could be rolled into field_translation rather than being a separate module.

8. Flag translations as outdated. This is a bit complicated. The current patch implements a flag in the node edit form, but not on the translation edit forms. This works if we assume the node language is the original source. But IIRC translation module offers this flag in all languages. That is, when editing the French translation of a node created in English, one can choose to flag existing translations - including the English original - as outdated. Do we need the same here? In practice this would be a switch on the translation form.

   Another minor point is that the vertical tab should show the current status of the field and update when the field is updated. Suggested text: "Don't flag translations as outdated" and "Flag translations as outdated".

Overall, this patch is a great draft in the right direction and shows the strength of the great foundational work in #367595: Translatable fields.

I'm *out* of this discussion (even though it's great to see it happen) ;-)

Just a quick note: "Having untranslatable field values on the field translation form might be useful, to avoid undesired modifications we might display them as readonly "
Tricky, because readonly means different things for each widget. That means we need to require every widget for every field type to support a readonly state. If we can avoid this, it's better IMO.

7. Reusability for non-nodes. Being able to extend this functionality to any fieldable entity would be great, but currently there are some hook implementations which are strictly node-related. Probably we would need to leverage #488542: Tie Field UI pages to any fieldable entity. Suggestions from yched would be welcome here.

Hm, a bit lost here :-) Can you summarize the problem ?

Got that part, but what's the kind of stuff that is specific to nodes and would need to be abstracted out "leveraging #488542: Tie Field UI pages to any fieldable entity" ?

re #14: yes, hook_field_attach_x() should be fine for hook_node_x() equivalents. We don't have any actual use cases for them currently, so using them would provide a good safe-check ;-)
Also, note that a form_alter() should be able to recognize any form with fields by checking for the presence of $form['#fields'].

If there are other, specific needs, I guess it's possible to enrich the 'translatable' section in hook_fieldable_info().

Side note: the link in #15 is wrong :-)

Node title I can't see being converted to a field this release (although it should be eventually). However when displaying the node title in various places, it ought to be possible to replace it with the contents of a translated field via contrib - so even if we can't deal with that in core, it ought to be possible to fix.

I can see use cases for a language column on comments regardless of field translation - sites are often listing recent comments regardless of node associations, and they could use that to filter on current language. It's also possible we'll eventually make comments into a field (so removing comment->nid and allowing a comment thread to be attached to terms or users) - at which point translatable comment fields looks like a possibility.

Taxonomy - currently we have a 'term reference' field, but haven't converted all the taxonomy_node_* to use fields yet. When that's done, it's likely that similar to node.title, it'd be easier to swap out term->name with the contents of a translated field - since we'd be overriding a formatter instead of hard-coded term->name everywhere.

Well, more generally, in an approach where translated nodes are not "different" nodes, any nodeapi addition loses translatability.

@yched, "any nodeapi addition loses translatability"

Yes. I think it's okay though. This is after all "field translation". As with many other features offered by Field API but not by ad-hoc nodeapi additions, the choice for module developers becomes, if you want it translatable via field translation, make it a field.

subscribe

nodeapi additions not being translatable is a strength of this approach - with module like nodequeue, voting API/fivestar, flag and others, you have to add a whole extra layer of support for 'translation sets' as a distinct unique entity apart from nodes to get them to work on a single piece of content as a single piece of content. This was a major part of the CivicActions/Sony internationalisation project, and dealing with tnids was the most difficult part of that project, since it requires every single nodeapi module to refactor itself if site builders are going to have any kind of option. With translatable fields, assuming more and more modules move towards field API for attaching themselves to things (quite possible to do even if you want to maintain your own specialized storage), we'd need only a conversion to a field, not a lot of special-cased code for translation sets.

For fields themselves, the i18nsync module exists in contrib, for the sole purpose of copying field values between translation sets for when they need to be duplicated, this patch also partially grew out of #132075: Attach fields to translation sets. which tried to solve the same issue with Field API in core (although a bit differently).

Since we don't remove tnid handling (not yet anyway), this gives site builders some options on how to do things for nodes, not to mention the benefits for translation of non-nodes, when we can finally stop abusing the {locale} table with all the user interface and performance implications that has.

subscribing

.

One detail I'm concerned about is that users setting up their site will be lost trying to choose between "Translation" module and "Field translation". We can't offer any help before the modules are enabled, so we've got only a short module description to try to convey the difference. I don't think we have any other similar case in core where we have alternate versions of comparable functionality that users can choose between.

I wonder if we could/should roll this into translation module rather than introducing a new module. It would be a bit more complicated. I guess we'd need a way to enable multi-node translation, field translation, or both. At least we could offer some reasonable help to guide admins in making this choice.

#367595: Translatable fields was committed, so setting this to needs review so test bot can have a crack at it.

Tagging.

subscribing

add tag

I think we could turn Translation into a simple stub module

I was thinking something similar, except that, instead of further modules, the two implementations would be in include files that are loaded as needed. But, on second thought, that could quickly get messy, e.g., when each needs separate hook implementations. We might be best keeping the current patch structure, introducing a new module, and just focus on making the module descriptions and help as clear as we can.

how about "page translation" (node based) and "content translation" (field based) or something similar.

EntityTranslationHandler should be named EntityTranslationHandlerInterface. All interfaces should have an Interface suffix.

You should also not force-include files that contain only classes. Classes and interfaces will be autoloaded by PHP automatically.

No time right now to review other stuff, but catch pointed me here and that's what jumped out at me.

OK I read through this. Can't really comment on OOP conventions but this looks somewhat similar to how entity loading does it, which got in already.

I have two main concerns here from just reading through:

1. The patch relies on etid (an internal ID for an entity type) - this is a concept currently only known to field_sql_storage.module. If we can think of this use-case for using it, it's likely contrib modules will too, so this requires possibly factoring the etid code from field_sql_storage.module upwards into field.module somewhere so that other modules can access it without making assumptions. This would be neither a schema nor API change (possibly an API additional though). Although frankly the only time I've seen etid used, was when a join on that table caused temp tables and filesorts in field_attach_query(), so I'm not at all convinced there's a benefit to having it at all.

2. The patch defines a new hook, hook_translation_info() - which merges it's results via hook_entity_info_alter(). Part of the plan for hook_entity_info() was to have a single routing hook for all info about an entity. I'm not sure at the moment whether declaring separate hooks and merging the data is a great and amazing way to avoid 2000 line hook implementations and keep things readable, or if it's unnecessarily adding hooks when technically one is enough. So just bringing it up as a point to consider.

I found a few code style/docs issues. This isn't exhaustive since I was a bit distracted by irc while looking through.

+++ modules/entity_translation/entity_translation.handler.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,427 @@
+   * Fix the language of the original field values.
+   *
+   * Ensure that the original translation language matches the language assigned 
+   * to the original field values.

I'd rather this was "resolve" than fix - it's post-hoc cleanup rather than actually fixing a bug isn't it?

+++ modules/entity_translation/entity_translation.handler.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,427 @@
+  /**
+   * Return the entity type identifier.
+   */
+  protected function getEtid() {
+    // @todo: Consider avoiding the use of a field SQL storage function to 
+    // identify the entity: we might have different storage engines.
+    return _field_sql_storage_etid($this->entityType);
+  }

Why do we need the etid here? As far as I know that should be completely internal to field_sql_storage.module

+++ modules/entity_translation/entity_translation.module	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,360 @@
+/* Helper functions */

No need for these.

+++ modules/entity_translation/entity_translation.node.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,181 @@
+ * Implement hook_form_alter().

hook_form_FORM_ID_alter()

+++ modules/entity_translation/entity_translation.node.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,181 @@
+/**
+ * Implement hook_node_load().
+ */
+function entity_translation_node_load($nodes) {
+  // @todo: Multiple load.
+  foreach ($nodes as $node) {
+    entity_translation_get_handler('node', $node)->load();
+  }
+}

Why hook_node_load() here, but no other hook_$entity_load() anywhere else? It looks like the supporting code has been added for nodes, but not yet for everything else. In which case that's probably OK since none of that is an API change to tidy up later.

I'm on crack. Are you, too?

Summary of an IRC discussion tonight:

- This still needs some work, and is not going to be RTBC by the deadline for API/feature freeze.
- While this could live in contrib now that the remaining TF patches were committed, we've seen with Field API that people don't really test out the features without a UI to bang on.
- It also seems fairly tragic for us to have this much more robust field-level translation system and no way for anyone to actually use it without turning to contrib, particularly since Drupal 6 was "the internationalization release".
- While the issue doesn't reflect it (grrr! :P), Bojhan did give this a UX review earlier on IRC. He pointed out some issues that have since been resolved and said he could live with the rest. (apparently)
- This patch is basically going to render the old one-node-per-language way of doing translations moot, in favour of field-level translation. This basically creates a double-UI which is incredibly confusing. One way suggested to circumvent this was to move the old node-based translation to contrib. IMO though, core should provide an upgrade path from the old way to the new way. This is dependent on CCK having an upgrade path in core as well (which as of now is vaporware)

I think Dries and I need to have a discussion as to the fate of this patch. Whether we extend the exception for it as a "release blocker', whether to include it under the banner of "UI clean up", or whether to hold the line and chuck it to contrib to deal with.

Please continue to work on it in the meantime; it certainly won't hurt anything.

This is one of the UI's I just haven't been able to follow/review. Bojhan has been working with plach on it, but didn't comment here.

Two remarks:

1.FWIW, quickly scanned the screenshots in #40 and they don't seem to scary. Interface text very important here, needs work
2. trategically I might prefer to add a UI for another use case with field-level complexity interactions. Just so that core has this problem to solve, not contrib. If smallcore + ui layer is the way for D8, then translatable fields UI is one for the core ui layer to provide patterns for, not contrib.

As such, for UX future this is worthwhile to try and get committable.

Many of this is above my head, here are a couple notes from a cursory look at the patch.

+++ modules/entity_translation/entity_translation.handler.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,507 @@
+   * Return TRUE if the entity translation is actually being translated.

"translation being translated" ?

+++ modules/entity_translation/entity_translation.handler.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,507 @@
+   * @param $transation

typo

+++ modules/entity_translation/entity_translation.handler.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,507 @@
+    foreach (field_info_instances($this->entityType, $bundle) as $instance) {
+      $field_name = $instance['field_name'];
+      $field = field_info_field($field_name);
+
+     // do something on $this->entity->{$field_name}
+
+    }

The pattern in field API (more specifically field.attach.inc, which controls field operations) is to use field_invoke() for those "do something on all the fields of an object". Not sure this can easily be applied here, but it's a bit sad to see all those foreach duplicate the work of the field_invoke() iterator.

+++ modules/entity_translation/entity_translation.handler.inc	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,507 @@
+        foreach ($entity->{$field_name} as $lang => $items) {
+          if ($lang != $langcode) {
+            $this->entity->{$field_name}[$lang] = $items;
+          }
+        }

This looks like a no-op to me, but I'm probably missing something.
A comment might be good.

+++ modules/entity_translation/entity_translation.module	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,422 @@
+            'entity key' => '#node',

Just a note that I think #367006: [meta] Field attach API integration for entity forms is ungrokable makes $form['#{entity_type}] standard for entity forms.

+++ modules/entity_translation/entity_translation.module	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,422 @@
+      'object keys' => array(
+        'humanReadableId' => 'title',
+      ),

should this be camel cased ? the rest of the translation_info() array uses simple space separated strings as keys.

+++ modules/entity_translation/entity_translation.module	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,422 @@
+  variable_set('entity_translation_edit_form_info', $edit_form_info);

Why is this part stored separately ?

+++ modules/entity_translation/entity_translation.module	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,422 @@
+    'entity translation' => array(
+      'title' => t('Translate entities'),

perm machine name seems off - why not just 'translate entities' ?
Side note : do we need separate, per-entity-type permissions ? (as is : common users should be able to translate their own profile info but not actual nodes)

+++ modules/entity_translation/entity_translation.module	1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,422 @@
+/**
+ * Implement hook_field_attach_pre_insert().
+ */
+function entity_translation_field_attach_pre_insert($obj_type, $object) {
...
+function entity_translation_field_attach_pre_update($obj_type, $object) {

hook_field_attach_pre_[insert|update]() : those hooks, although misnamed, were originally targeted at 'advanced' storage tricks (like per-bundle storage). Can this use the regular hook_field_attach_[insert|update]() (non-"pre") hooks instead ?
Edit: er, those hooks are currently missing from field_attach_[insert|update]() - well, it's a perfectly valid use case for adding them, then.

I'm on crack. Are you, too?

FYI, created #622534: Cleanup in hook_field_attach_*().

I have some questions mainly related to usability issues:

1. in entity_translation_overview() the original translation has the title as link while the other translations have 'view'. Why this discrepancy? Is it only an implementation issue?
2. In the translation page we have "translation settings" and "url alias" mixed in the form. Can we put this elements in a vertical tab like in the node form?
3. If I click the edit tab from a translated version of the node I go to the original node edit form. IMO for a user this behavior can be weird and misleading; normaly the edit in drupal is contextual to the displayed content so I expect to edit the translation and not the original node.
4. In admin/content we could add the 'translate' operation.
5. In the translate form the untranslatable fields is showed like in the node view; would not it be better to display this fields as not editable or at least with a description of the field?
   

       if (!$field['translatable']) {
         $form[$field_name] = array(
           '#markup' => drupal_render($field_view[$field_name]),
           // Place the element where it would appear if displayed.
           '#weight' => $instance['display']['full']['weight'],
         );
       }
   

6. should not the translation be published by default as it is for a node?

So, I have actually given this a big review a few weeks ago - going trough a lot of interfaces with plach. I think this is a good state to go from, there are some minor text issues - but I don't see much value in working on those here.

@peximo

2. Seems somewhat over-done, to put these in a vertical tab. These are clearly, and will only be two elements - vertical tabs is not supposed to be used for that.

3. Should see a fix.

4. Could bloat the table?

5. In the translate form the untranslatable fields is showed like in the node view; would not it be better to display this fields as not editable or at least with a description of the field?

That would mean we need to require each widget to support a 'disabled' state. A little bit of work, outside the scope of this issue.
I agree, however, that the "test / test" text on translate entity form currently looks confusing. We should probably at least display the field styled with the regular field.tpl.php output and CSS. That would mean moving forward on #612894: field_format() is just a pile of code that doesn't work.

I was unsure whether we want to add this to core. However.

After working on some other patches (f.e. #301902: Allow more users to see the node admin page), I realized that I have absolutely no clue how the field structure of a translatable field looks like and how to code for it when it is populated with translations. Even after being involved in all these TF patches. That's a major problem. And I have no clue what strange/wrong code we will see in contrib, if contrib developers don't have something to play + understand as well.

Hence, my point is that there is no way around committing this. So we need to get the implementation into RTBC state as soon as possible. UX tweaks can follow afterwards.

Honestly, I didn't expect so much OOP in here. I guess it makes sense, and I really like how the module alters itself into the otherwise untouched entity handling. But it also means that we need Crell or some other OOP masters to review this patch. I will try to review and test this patch more in-depth this weekend.

Reading #61 once again, I'd say that we want to move Translation module into contrib (as node_translation module), and at the same time, make this patch replace translation.module in core. "Entity translation" does not sound like something I'd want to enable as a regular user. But a "Translation" module that allows me to translate fields everywhere - that, I absolutely want to enable. :)

Agreed with sun. The core translation module has very few open issues, and hasn't had many over the whole of the D7 release cycle. It shouldn't require a huge maintenance burden during the release cycle, and if Jose didn't mind, we could move it back to the i18n project maybe?

While we could write an upgrade path for body (and title if translatable titles gets in), doing so for CCK fields relies on the overall CCK to Field API upgrade path, with additional tweaks, so there's a node translation -> translatable entity conversion, CCK -> Field API conversion, then CCK/nodetransation -> Field API/Entity translation patht to wrangle. But keeping node translation in contrib means there's an upgrade path regardless.

yeah, i18n_node sounds even more appropriate. I totally forgot that it originally came out of it.

Created #627734: Take back Translation.module?

@webchick: Would you be ok with this?

Two things bug me in the patch:

1. Just viewing the translation_overview page can make some writes to field values ?
   

   function entity_translation_overview() {
     if ($handler->initOriginalTranslation()) {
       field_attach_update()
     }
   }
   

   I sort of gather this is in order to do some verification / cleanup / preparation, but updating field data on a simple page view sounds scary.

2. {entity}_save() cascades down to a field_attach_load():
   {entity}_save() - field_attach_update() - hook_field_attach_pre_update() - $handler->updateTranslations() - $handler->prepareRevision() - field_attach_load()
   Load within a save, and that deeply nested, sounds like a recipe for strange effects and hair-pulling debug debug...

We MUST commit this.

Can we get a new patch that replaces Translation module?

This is cool but not really a replacement for translation module.

This is not really about having a few fields that are tranlatable, but about having a *full site workflow* for multilingual content. Like who creates/can create each translation?, should all translations have the same owner/status/date/comments/taxonomy?, what happens with my node listings?, will all content need to be translated / is meant to be translated? will my translators need to be able to edit the original content / content in any language?...

Really, this patch is *far* from figuring out all that questions and from providing flexible answers for them. While on the other side we have an *already working* translation module

I think for Drupal 7 we should go without this module, multilingual fields are a big enough step forward and this 'Entity translation' could live in contrib until the solution proves to be mature enough to work for a few hundred live multilingual sites.

catch #68> The core translation module has very few open issues, and hasn't had many over the whole of the D7 release cycle

Wouldn't that mean that it actually works for a lot of people and its a good reason to keep it in core? The i18n module has more than 15000 (reported) users (who use translation module too), so at least I can tell the lack of issues is not because no one uses it.

So if you guys want to get this in core, ok, go ahead. But please don't say it is a replacement for translation module because it is far from that yet.

subscribing

subscribing

@Jose "Wouldn't that mean that it actually works for a lot of people and its a good reason to keep it in core? "

It works pretty well by itself, however as soon as you need to integrate translated nodes with other modules, you quickly run into problems:

1. Menu links can only link to one system path path - translated nodes each have their own translation - this requires duplication of menu links or entire menus, when other multilingual paths are handled fine with a single sytem path.

2. Any module acting on hook_nodeapi() needs to also account for translation sets - CCK, nodequeue, voting API, flag - these all need additional code branches or special workarounds on sites running translation module.

#307810: Multilingual support for flagging
#307207: Multilingual voting: option to tally votes by translation set
#251092: Multilingual Support

Those patches are just a handful of examples - the reality is that many modules simply don't work with translation sets as a consistent thing at all.

So the reason D6 translation module has few issues itself, is because it pushes the responsibility for managing data which needs to be shared or tracked across translation sets down to every single module interacting with nodes in contrib. This was the initial impetus behind this patch, and remains the compelling reason to try to make it the primary mechanism for translating nodes (whether that's in core in Drupal 7 or 8 is a different thing of course).

@plach #79,
I only want to add that we never said this is ready and that covers all the possible uses cases that node translation can face. And we never said that committing this presents no risks, but we have a new and possibily fragile API in core that without this one can't be tested at all.

Ok, then we just need a contrib module to test the core API, right?

@catch #82,
So the reason D6 translation module has few issues itself, is because it pushes the responsibility for managing data which needs to be shared or tracked across translation sets...

While if replacing translation by this new one, it won't allow to manage some data that needs to be kept separate for each translation. Then we'd just have the opposite feature requests. "Hey , I want poll votes to be kept separated by language, then be able to display them aggregated". What about content status, author, title, taxonomy????

Really, we've had this dicussion for years about which model is better, you can read this from the beginning, http://groups.drupal.org/i18n

Just to save you the reading, the answer is "no one knows". But this is the fact: there are thousands of multilingual sites running with the current translation module.

So my main issue with all this is: OK, it looks cool, now please: Show me your multilingual site (And until we can see that, please let us have the current module, which at least works... sometimes... for some people)

And please, don't get me wrong. This development here is great. But still we don't have a working site workflow.

For me, this is one clear case where the concept needs to live and grow up in contrib for a while. Because if it gets into core, cool, then you'll have it frozen for one year until the next Drupal version, no improvements allowed.

So IMHO the best thing that could happen to this module is to be moved into contrib. It doesn't really matter that much what happens with the translation module....

Yes, as mentioned before - the current state of translatable fields in D7 is mostly unknown, because no one is able to use and test it. No developer is able to see how a $node data structure looks when there are translatable fields in it. And I'm 100% sure that we have a lot of broken code in HEAD currently that was committed, because no one is really aware of how to deal with translatable fields in their patches and tests.

Brainstorming... what about not dropping Translation module (yet) and merging TF UI into Translation module? Would that be doable? Although I don't see any need for the old Translation module functionality with translatable fields being in place, this perhaps would be a compromise we could agree on...

I was e-mailed and asked to respond to the question about how to handle entity translation in Drupal 7 in relation to this patch, Content Translation, etc. and a path forward for core. Here's a summary:

Pros:

1. Translation works for any entity, not just nodes.
2. Supports lots of scenarios that previously needed field synchronization (poll votes, ubercart products, etc.)
3. Actually puts our translatable fields API into practice, so that we can properly test it (and probably find lots of bugs in the process ;))
4. Has UI thumbs up from UX team
5. Upgrade path seems to be working from an API side
6. Helps move the Drupal community into new paradigm thought rather than legacy thought about translation

Cons:

1. All of the functionality of Content Translation is not yet replicated. For example, fine grained access control between different entity translations.
2. Having both Content translation and Entity translation together creates a big UX mess.
3. No test coverage
4. Work on this is still in the experimental stage; While Content Translation has several thousand sites to show for it, this new paradigm is untested
5. Committing it means freezing its development

I've given this a lot of thought, and I hope that comes through (perhaps too much :P) in my response below. I want to emphasize again my profound gratitude and respect for you all and the tremendous work you've done here.

First, some timeline stuff. The original labelling of the Translatable Fields issues (TF #1, #2, #3, and #4) was explicitly laid out in order of "impossible/difficult/easy to achieve from contrib." This was exemplary project planning, and hats off to plach and/or sun, and/or whoever else came up with this great scheme. By feature freeze (10/15), #1, #2, and #3 were thoroughly reviewed, RTBCed, and committed. #4 was the only one that remained. The original patch had one hunk that required a core change but that was committed a couple of weeks ago.

plach, sun and I had a similar discussion to what's being asked here on IRC the day after feature freeze. Basically discussing whether this patch could make it in if it were ready by UI freeze. I had expressed doubts then that the patch would in fact be RTBC before UI freeze, and had further doubts about locking this into core when it was undergoing such rapid development, particularly given that TF API was brand-spankin'-new.

Today we are at 12/6, which is an additional 3 weeks past the original UI freeze date, and 3 months past the original code freeze date. And, unfortunately, this patch is still not RTBC. This most certainly is NOT a negative reflection of the people doing tremendous work here (this issue, in fact, exemplifies how core developement should be done, in my view). But rather it's a reflection of the scope of the paradigm shift we're dealing with in this new module. It's also lacking tests, so we still don't know what bugs lie under the surface, nor how long it will take to clear those up.

(And just to head this off at the pass, before someone inevitably drags Overlay module into this dust-up, remember that that patch had been specced in April, initial prototype posted in July, and has been basically functionally identical since then. It was only under-the-hood tweaks that had been made over the subsequent months, and that module was literally doing nothing but sitting there and getting buggier by it not being hammered on by 200 people.)

I have chosen to bump this issue to 8.x, and attack Entity Translation from contrib, for the following reasons:

1. Timing. The comments in this thread rightly describe this patch as a fundamental paradigm shift. The time to commit paradigm-shifting patches is early in the release cycle, not when you're past the final marker just before the last critical bug sprint before a release.
2. Foundation. When Gábor went to tackle the problem of bringing i18n features into core, what he did was carefully analyze and cherry-pick the best architecture, functionality, and features that were available in contrib at the time. Not all of this made it in in time for code freeze, but the parts that did were very solid and contrib modules such as i18n were able to build off if them, and in a much more integrated way.

   I believe you have all done exactly that for Drupal 7's release cycle: put the necessary API bits into core, so that contrib may innovate solutions around it, and during the Drupal 8 release cycle the next maintainer can carefully cherry-pick from the available modules that best utilize the TF functionality, rather than locking people into the first serious stab we took at it.

3. Lack of strong, compelling pros. While the pros list above has TONS of great stuff on it, there's not one of them that are no longer in the list if we move this module to contrib (with the possible exception of forcing a paradigm shift on people, but see #2).

   I know sun stressed above that this module provides a way to test core's TF APIs, which currently have no means in core of doing so. This is true. But this is also not the only API in core that has no core implementation, and yet we still manage to find bugs in the node access system, private file handling system, XML-RPC handling, and so on. Sorry, but this is not a compelling enough reason to commit, given the other things to consider here.

4. Retaining momentum. On the other hand, one of the enormous benefits we gain by NOT committing this patch now and locking its behaviour into core for 3+ years is allowing the wonderful work that's gone on in this issue to continue in contrib, over multiple iterations and multiple branches. There's a strong team here that I'm confident can give this paradigm shift the focused attention it deserves if they don't have deadlines hanging over their heads like guillotines all the time.
5. Risk. There's no test coverage here. There is strong opposition from the maintainer of the i18n project, whose cooperation is needed in order to carry out the plan of pulling the current Content Translation module out of core and into i18n. Yet, as José points out, CT is working well enough for thousands of sites atm, where this new module is totally untested "in the field".

Again, I want to sincerely thank everyone here for their work. Just... I have to put on my release manager hat on this, and this is the only sensible thing to do, from my point of view. I hope you understand.

Does it mean that the translatable fields API is available in the core, but its user interface will be in a contributed module?

Correct.

As the decision is given, can you put it to a contributed module? I am eager to try it and and start working on a multilingual site using translatable fields :)

Let's discuss this contrib part first. I'm preparing in the meantime.

I think 'recycling' the 'translatable' one would sound good (long names are awful for developing).

Let's save the 'translation' one for when you guys finally kick it out of core :-)

I considered to use Translatable module for DDT only, but we may re-consider this.

However, I see no problem with overriding core's Translation module from contrib. This would make the upgrade path for existing users easier, decrease confusion, but also make the upgrade path into D8 core easier. I therefore opened #652930: Project ownership of Translation module and also contacted the current project owner.

Perhaps, that's not evident: sites/all/modules/translation/translation.module completely replaces modules/translation/translation.module, if existent. That's a fairly hidden core feature. ;)

That would surely depend on whether we would want to maintain Content translation module's functionality at all or just provide an upgrade path. If you look at the commit history, then it's obvious that this module didn't see any love and no major changes in the past years. Hence, the second option would be to just copy and mix in the old functionality, so we'd have entity (more than nodes?) AND field translation within one module.

So it wouldn't really matter. But, honestly, I think we discussed translatable fields to death and the result was always same: the fragile synchronization approach of Content translation module must die and there is no use-case that we cannot handle with translatable fields. Hence, instead of providing the option for the user at all, we probably want to nuke it.

I posted a related isssue, #653656: Link to new fields translation module in translation module description.

This is where I start to worry. Why everybody wants to drop the old one before the new one is really working?

There are many use cases 'entity translation' cannot handle yet. What about different taxonomy for each language? What about having independent revisions for each translation? Published / promoted / etc different for each translation? And in general what about not having translated content but different content for each language? And what about benchmarks for entity translation?

So I would allow site builders to decide. If you provide a different (compatible) module maybe we could choose a different approach for each content type. Though I agree only one approach is good for core, there's nothing wrong with having options in contrib.

I would be really happy if we can use fields translation for content like polls or images, really I cannot wait to drop that hacks from i18n. But if you tell me I need to use entity translation too for my forum which is 100% independent for each language, and has different taxonomy too for each language, that's where things start feeling wrong.

So please: first provide a solution, then drop the old one if you don't need it, but not before.

subscribing - looking forward to testing Translation module in D7

As we already are at > 100 follow-ups here, and we'll surely improve the TF UI in http://drupal.org/project/translation during the D7 release cycle (making this issue and patch obsolete), I'm going to mark this won't fix. We should re-start a new issue from scratch during D8, when we have a solid + working contrib module.

Opened #1188388: Entity translation UI in core for Drupal 8.