Problem/Motivation

Drupal core exposes translation language fields for the entities it supports. Most entities just use the LanguageField provided by views but there are also some entity fields that override that in user, node and taxonomy. User has a link to user option, node has a link to node option and taxonomy always links to the taxonomy term. These specialized field handlers are not universally applied, some language fields of users and taxonomy do not use their handlers but the default views handler.

The views default language field handler and the node language field handler both have a "Native name" option which is a no-op. So if you configure a node or a view field handler to display in native name, that will never work. There is a @todo at both places which points to #1616594: META: Implement multilingual CMI which has long been closed.

Proposed resolution

1. Unify the language field handling by having one field handler instead of entity type specific handlers.
2. Extend the language formatter from StringFormatter, so we get linking to entity for free.
3. Remove the special cased entity specific language field handlers.
4. Restore the native name display functionality in language formatter.

Note that original language handling in views is broken and will not be possible to solve the same way, see #2450195: Original language of entities not accessible in views anymore for an issue solving that.

Remaining tasks

Review. Commit.

User interface changes

Consistent translation language field options across entity types.

API changes

Specialized views handlers are removed from node, user and taxonomy. Views using those would need to be updated.

CommentFileSizeAuthor
#168 interdiff.txt1.62 KBGábor Hojtsy
#168 2384863-168.patch26.58 KBGábor Hojtsy
#166 interdiff.txt5.89 KBGábor Hojtsy
#166 2384863-166.patch26.57 KBGábor Hojtsy
#164 interdiff.txt867 bytesGábor Hojtsy
#164 2384863-164.patch24.13 KBGábor Hojtsy
#162 interdiff.txt559 bytesGábor Hojtsy
#162 2384863-162.patch24.22 KBGábor Hojtsy
#159 interdiff.txt1.59 KBGábor Hojtsy
#159 2384863-159.patch23.68 KBGábor Hojtsy
#157 2384863-157.patch25.26 KBGábor Hojtsy
#133 2384863-133.patch24.96 KBGábor Hojtsy
#125 2384863-diff-122-125.txt996 bytesvijaycs85
#125 2384863-native-language-125.patch25.21 KBvijaycs85
#122 2384863-native-language-122.patch24.95 KBGábor Hojtsy
#120 interdiff.txt1.59 KBGábor Hojtsy
#120 2384863-native-language-120.patch24.89 KBGábor Hojtsy
#117 2384863-native-language-117.patch23.3 KBGábor Hojtsy
#116 interdiff.txt710 bytesGábor Hojtsy
#116 2384863-native-language-116.patch20.33 KBGábor Hojtsy
#114 interdiff.txt2.52 KBGábor Hojtsy
#114 2384863-native-language-114.patch23.29 KBGábor Hojtsy
#111 2384863-diff-97-111.txt3.21 KBvijaycs85
#111 2384863-native-languge-111.patch23.4 KBvijaycs85
#108 2384863-diff-106-108.txt5.23 KBvijaycs85
#108 2384863-native-languge-108.patch23.9 KBvijaycs85
#106 2384863-views-native-language-render-106-rerolled-from-93.patch21.28 KBGábor Hojtsy
#97 interdiff.txt3.05 KBGábor Hojtsy
#97 2384863-views-native-language-render-97.patch23.08 KBGábor Hojtsy
#93 interdiff.txt2.02 KBGábor Hojtsy
#93 2384863-views-native-language-render-93.patch22.36 KBGábor Hojtsy
#91 interdiff.txt1.77 KBdawehner
#87 2384863-views-native-language-render-87.patch21.26 KBGábor Hojtsy
#87 interdiff.txt1.34 KBGábor Hojtsy
#84 2384863-views-native-language-render-84.patch21.26 KBGábor Hojtsy
#77 interdiff.txt1.08 KBGábor Hojtsy
#77 2384863-views-native-language-render-77.patch22.68 KBGábor Hojtsy
#74 interdiff.txt7.24 KBGábor Hojtsy
#74 2384863-views-native-language-render-74.patch21.6 KBGábor Hojtsy
#70 interdiff.txt1.24 KBGábor Hojtsy
#70 2384863-views-native-language-render-70.patch21.65 KBGábor Hojtsy
#69 interdiff.txt2.36 KBGábor Hojtsy
#69 2384863-views-native-language-render-69.patch20.9 KBGábor Hojtsy
#67 interdiff.txt3.29 KBGábor Hojtsy
#62 2384863-views-native-language-render-62.patch20.43 KBGábor Hojtsy
#61 interdiff.txt3.32 KBGábor Hojtsy
#61 2384863-views-native-language-render-60.patch19.76 KBGábor Hojtsy
#60 2384863-views-native-language-render-59.patch23.08 KBGábor Hojtsy
#58 interdiff.txt11.84 KBGábor Hojtsy
#58 2384863-views-native-language-render-58.patch22.87 KBGábor Hojtsy
#56 2384863-views-native-language-render-56.patch13.36 KBGábor Hojtsy
#50 interdiff.txt1.15 KBrodrigoaguilera
#48 2384863-native-language-48.patch13.33 KBrodrigoaguilera
#47 2384863-native-languge-47.patch13.03 KBGábor Hojtsy
#36 2384863-diff-32-36.txt2.79 KBvijaycs85
#36 2384863-native-languge-36.patch13.2 KBvijaycs85
#32 2384863-views-native-language-render-32.patch10.41 KBGábor Hojtsy
#32 interdiff.txt7.72 KBGábor Hojtsy
#24 TAX (Taxonomy term) | s61976f889d5ebb7.s3.simplytest.me 2014-12-17 19-00-52.png29.51 KBGábor Hojtsy
#20 link_to_node.txt5.13 KBvijaycs85
#19 interdiff.txt660 bytesGábor Hojtsy
#19 2384863-views-native-language-render-19.patch6.72 KBGábor Hojtsy
#17 interdiff.txt2.94 KBGábor Hojtsy
#17 2384863-views-native-language-render-17.patch6.07 KBGábor Hojtsy
#16 interdiff.txt929 bytesGábor Hojtsy
#16 2384863-views-native-language-render-16.patch4.66 KBGábor Hojtsy
#14 interdiff.txt966 bytesGábor Hojtsy
#14 2384863-views-native-language-render-14.patch4.37 KBGábor Hojtsy
#12 2384863-views-native-language-render-12.patch3.43 KBGábor Hojtsy
#5 interdiff.txt3.4 KBGábor Hojtsy
#5 2384863-views-native-language-render-5.patch4.1 KBGábor Hojtsy
#1 2384863-views-native-language-render.patch986 bytesGábor Hojtsy
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Gábor Hojtsy’s picture

Gábor Hojtsy’s picture

Issue tags: +D8MI, +language-content, +sprint
vijaycs85’s picture

Title: Views node language field native language option is a no-op » Remove native language specific workarounds
Issue summary: View changes
Gábor Hojtsy’s picture

Title: Remove native language specific workarounds » Native language settings in views fields are no-ops
Gábor Hojtsy’s picture

Covering the other case and the config schema as well. No test yet.

Gábor Hojtsy’s picture

Issue summary: View changes
Status: Needs review » Postponed
vijaycs85’s picture

  1. +++ b/core/modules/node/config/schema/node.views.schema.yml
    @@ -94,12 +94,12 @@ views.argument_validator.node:
    -  type: views_field
    +  type: views.field.node
    

    extend/reuse++

  2. +++ b/core/modules/node/src/Plugin/views/field/Language.php
    @@ -41,11 +40,9 @@ public function buildOptionsForm(&$form, FormStateInterface $form_state) {
    -    // @todo: Drupal Core dropped native language until config translation is
    
    +++ b/core/modules/views/src/Plugin/views/field/LanguageField.php
    @@ -40,11 +39,9 @@ public function buildOptionsForm(&$form, FormStateInterface $form_state) {
    -    // @todo: Drupal Core dropped native language until config translation is
    

    @todo--

This is good to go.

Gábor Hojtsy’s picture

Well, this does need its own tests to ensure the option actually works :)

Gábor Hojtsy’s picture

Issue summary: View changes
Status: Postponed » Needs review

Status: Needs review » Needs work

The last submitted patch, 5: 2384863-views-native-language-render-5.patch, failed testing.

Gábor Hojtsy’s picture

Gábor Hojtsy’s picture

Status: Needs work » Needs review
Gábor Hojtsy’s picture

A quick attempt at test coverage.

Status: Needs review » Needs work

The last submitted patch, 14: 2384863-views-native-language-render-14.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
4.66 KB
929 bytes

The language names will not be displayed translated if the translation is not there, hahaha.

Gábor Hojtsy’s picture

Even more tests to cover both behaviors of the other language field and to test the original behavior of the view, so all two field handlers in both variants.

Status: Needs review » Needs work

The last submitted patch, 17: 2384863-views-native-language-render-17.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
6.72 KB
660 bytes

If we remove the link_to_node setting then this becomes more applicable to the more general language field handler too.

vijaycs85’s picture

FileSize
5.13 KB
+++ b/core/modules/node/tests/modules/node_test_views/test_views/views.view.test_language.yml
@@ -159,7 +159,6 @@ display:
-          link_to_node: false

hopefully, it won't come again as I can see it is there in core views configuration.

Gábor Hojtsy’s picture

@vijaycs85: that setting makes sense for the field as in the view. It does have a FALSE default if not provided in the view, so the effective version is the same. However it would not be valid for a language type field, so instead of removing it in the test where I convert the field type, I removed it altogether because it works the same way.

jhodgdon’s picture

I tried this out in a Node view and it seems to be working fine.

I tried a Taxonomy view and it doesn't have this setting though on its language fields. Should it? Probably taxonomy has its own plugin, ugh. Is that out of scope for this issue? It seems as though if we have the option on generic language plugins and node plugins, we should have it for all of the language plugins.

And... Can we get rid of the specific entity language plugins and unify them possibly, rather than having 5 different ones or whatever we have?

Gábor Hojtsy’s picture

@jhodgdon: I consider that a different problem from this issue which is about the plugins not providing the advertised features.

Gábor Hojtsy’s picture

@jhodgdon: for the views data / field types question, NodeViewsData overrides the language fields to use the node_language plugin but EntityViewsData defaults language fields to the language plugin. So entities with a language field other than node should offer this field handler up for their language field. For example for taxonomy terms:

jhodgdon’s picture

I didn't see that taxonomy setting. If you use the field "Translation language" instead of "Original language", the Native Language box is NOT in the settings. ?!?

Gábor Hojtsy’s picture

So node, taxonomy and user seems to have their own language field handlers that extend from their respective main module handlers. And then the general language handler. Only node and the general one has native language support. While this issue was opened to fix the falsely advertised native feature that did not work before this issue, we can also unify the fields here, but that would require some examination that those special field features are actually not needed.

jhodgdon’s picture

OK. So I got interested in the question of what the individual entity-specific Language plugins are doing that the base one in \Drupal\views\Plugin\views\field\LanguageField is not doing. (This plugin is the default field plugin used for all language fields on entities unless the specific EntityViewsData class overrides the plugin choice).

Here is a survey:

a) Taxonomy
- Uses \Drupal\taxonomy\Plugin\views\Field\Language (ID "taxonomy_term_language") as the plugin for the Translation Language field, but not for the Original Language field (which in itself is odd; the Original Language field uses the base Views plugin default).
- This plugin extends the Taxonomy field plugin. This gives it options to link to the taxonomy term page and to convert spaces in the name (in this case, the language name?!?) to hyphens.
- The taxonomy language plugin does not currently have the native language display option on it.
- To me, neither of the options from the Taxonomy plugin makes any sense for a Language field and I think we should get rid of this override and the \Drupal\taxonomy\Plugin\views\Field\Language plugin class entirely, and use the default base views plugin instead, which would give us the native language option for free, and make the two language fields for Taxonomy behave the same.

b) Node
- uses \Drupal\taxonomy\Plugin\views\Field\Language (ID "node_language") for Translation language, Revision Translation language, and Revision Original language fields.
- This plugin extends the Node field plugin, with the addition of a native language option.
- The Node plugin gives you the option to link to the node. Like Taxonomy, I don't really think it makes a lot of sense to link the language field to the node.
- So again I think we should just get rid of this language plugin and use the Views default language plugin instead.

c) User
- uses \Drupal\user\Plugin\views\Field\Language (ID "user_language") for the "Original language of the user information" field, but not for the other language fields.
- As usual this gives you a "link to user" option, see (a) and (b).
- Again I think we should drop this, and that would also make all language fields on User behave the same, since the others use the default plugin already.

d) Other entities: there are no [field][id] plugin overrides for user language fields that I can see in the other classes that extend EntityViewsData: AggregatorItemViewsData, AggregatorFeedViewsData, CommentViewsData, or FileViewsData,.

So... If we think that it's important for Language fields to have the "link to entity" option, we could add that to the base Language plugin. That was actually done for regular short text fields recently on #2387019: String field formatters cannot link to their parent entity. But that is all we would lose if we got rid of the 3 specific language plugins and switched to using the one default Views plugin. To me, this option doesn't seem all that important to have, but if we need it, it could certainly be provided. The way to do that, actually, would probably be to make a new field plugin called "entity language" with this option, and use that as the default for all language fields on entities in the base EntityViewsData class. (The base language plugin for Views as a whole doesn't know about entities.)

Anyway, thoughts? I don't think losing the "link to entity" option is a major problem, and unifying everything in one central plugin, and getting rid of 3 classes and several lines in those EntityViewsData customizations would be good.

Gábor Hojtsy’s picture

I can see use cases for example to display a translation of a node with a link using the original language name back to the original language for example. Is there a a use case of a language field that is NOT on an entity? Should we have a basic_language and a language formatter like with strings now?

Gábor Hojtsy’s picture

jhodgdon’s picture

Ah, good point. If the links actually are produced in the correct language, that would be a good use case. (I am not sure they do. At some point we should test this, once we're closer to having views actually working for multilingual... they may be working right but I'm just not sure. Actually I'll add this to the giant #6 on the Views meta.)

So, I think yes we should have a basic_language and language formatter. While we have no use cases in Core for the non-entity language, that is not conclusive because all our Views integration in Core is entities. Oh wait, we do have several things in Core that are not entities or fields that have views integration, like dblog and tracker... I don't see any language fields in any of them, but in theory someone could have a non-entity db table with views integration and a langcode column and want to have a language field, so yes I think we should keep a non-entity version around. Making its name basic_language and keeping language for the entity version would mean not having to change Core, so that does sound like the best idea with the least impact, since I think most views integration in d8 contrib is likely to be entities, right?

Onward!

jhodgdon’s picture

Adding the ability to link back to the entity would also be consistent with the string decision: just put the checkbox on all "short strings". If it isn't used, it doesn't cost much.

And I think... wouldn't it be as simple to implement as having the entity language plugin extend the new entity-string-with-link plugin, rather than extending plugin base or the non-entity language plugin?

Gábor Hojtsy’s picture

Rolled the patch with removal of the "special" language fields than, just using one common language field. (No linking yet, we should discuss if this is an 80% use case given that the views content rewrite features can ALWAYS be used to link it).

jhodgdon’s picture

+1000 on this patch! I love that almost all of it is "remove stuff", and that most of the added lines are in a test, with a few lines added to make the "native language" functionality on the default Views language field actually work.

One thing I think this patch is missing is to remove also the 'user_language' custom field from UserViewsData? I see Node and taxonomy but not User.

And agreed we still need to decide whether the Language field needs "link to entity" on it. You can definitely use rewrites to do this... well probably anyway. I am not sure that rewrites would be easy to use to get a link to the entity in the correct language?

vijaycs85’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +LONDON_2014_DECEMBER

Looks good to me.

jhodgdon’s picture

Status: Reviewed & tested by the community » Needs work

We need to do user as well as node and taxonomy, see #33. I guess I forgot to change the status.

I'm also not sure we decided it was OK to remove the ability to link language fields to their entity, which was present in the special fields for node, user, and language.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
13.2 KB
2.79 KB

Ok, here is a patch that removes user_language field.

I'm also not sure we decided it was OK to remove the ability to link language fields to their entity, which was present in the special fields for node, user, and language.

Still need to address this one.

Gábor Hojtsy’s picture

  1. +++ b/core/modules/user/config/schema/user.views.schema.yml
    @@ -67,18 +67,6 @@ views_field_user:
    -views.field.user_language:
    -  type: views_field
    -  label: 'User language'
    -  mapping:
    -    link_to_node:
    -      type: boolean
    -      label: 'Link this field to its user'
    -
    -views.field.user_language:
    -  type: views_field_user
    -  label: 'User language'
    -
    

    LOL on the duplicate definition, haha....

  2. +++ /dev/null
    @@ -1,51 +0,0 @@
    -      $uid = $this->getValue($values, 'uid');
    -      if ($this->view->getUser()->hasPermission('access user profiles') && $uid) {
    -        $this->options['alter']['make_link'] = TRUE;
    -        $this->options['alter']['path'] = 'user/' . $uid;
    -      }
    

    All right, if we are to do the link feature, is this the kind of stuff we want done there? The string formatter does not check if you have access to the parent entity. (This may actually be a problem of the StringFormatter?)

jhodgdon’s picture

Right now there is no access checking at all for base entity fields, see #2385443: Test that base entity fields on views respect field level access. Once that is taken care of, which actually would be on #2342045: Standard views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency, which will make all base entity fields rendered using entity formatting, I think we'll be OK.

The reason I think so is that if you've got a row that is entity X and you have no permission to view entity X, then you shouldn't be seeing *anything* in that row at all, including the language field, whether or not it is linked to entity X. So I don't see how that is a problem? It may not even be a problem now; hopefully views is not letting you see any results for entity X if you don't have entity view permissions.

vijaycs85’s picture

@Gábor Hojtsy, reg: #37.2 - Looks like it is a copy/paste from parent class core/modules/user/src/Plugin/views/field/User.php. It doesn't look like related to language.

Gábor Hojtsy’s picture

@jhodgdon/@vijaycs85: indeed, it makes sense to not need this on the field formatter level. So the only remaining question is indeed the linking feature loss with this patch.

jhodgdon’s picture

If we assume (not saying we should!) that we need to not lose this functionality of linking the field to the entity, how much would it take to make it happen? It seems like the answer is "not much":
- We'd need a specific entity language field handler class, which would either extend the existing "short text with link to entity" handler that was created recently and make it into a language renderer, or extend the existing "language that isn't entity" handler and make it link to the entity.
- We'd need to make this the default handler for language fields in the EntityViewsData base class, either by (a) giving this new one the "language" ID and giving the old non-entity one a different ID, or (b) by changing EntityViewsData. Probably (a) is easier.

I think that rather than debate whether we need this functionality or can afford to lose it, we should just do this.

Gábor Hojtsy’s picture

Well, we'll also need test coverage for the non-entity language field then. It is true that we can still test it on entities. So may not be a biggie to add that.

dawehner’s picture

The reason I think so is that if you've got a row that is entity X and you have no permission to view entity X, then you shouldn't be seeing *anything* in that row at all, including the language field, whether or not it is linked to entity X. So I don't see how that is a problem? It may not even be a problem now; hopefully views is not letting you see any results for entity X if you don't have entity view permissions.

... for those kind of access checking we always rely on query rewrite, so for example the hook_node_grants/records() system. This is taken care of.

Great work and great cleanup here!

+++ b/core/modules/views/src/Plugin/views/field/LanguageField.php
@@ -40,11 +39,9 @@ public function buildOptionsForm(&$form, FormStateInterface $form_state) {
+    $languages = $this->options['native_language'] ? \Drupal::languageManager()->getNativeLanguages() : \Drupal::languageManager()->getLanguages();
+    return isset($languages[$value]) ? $languages[$value]->getName() : '';

Do we have field formatters for that? ... we would at one point need that, in case we really want to support it.

Gábor Hojtsy’s picture

@dawehner: we already extended the scope based on what @jhodgdon suggested, I would avoid unifying it with field formatters as well here. The original scope was some views exposed settings that lied because they did not have any effect.

dawehner’s picture

Issue summary: View changes
Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

@dawehner: we already extended the scope based on what @jhodgdon suggested, I would avoid unifying it with field formatters as well here. The original scope was some views exposed settings that lied because they did not have any effect.

Well sure, it was more of a general question ... especially in the context of the base field formatter issue.

If I read the patch we already add a test.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 36: 2384863-native-languge-36.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
13.03 KB

Rerolled. It did not apply on the user.views.schema anymore because the user_language duplication was removed earlier. So we only need to remove one now. Still needs work for the restoration of the linking to entity feature. I looked into how this can be restored by a user on the UI and that looks very confusing. You need to add the entity id to the field view, then hide it and then know the URL pattern used for the entity at hand. If we are fine with that workaround, then this can go in as-is.

rodrigoaguilera’s picture

I did a manual testing
-Enable all 4 multilingual modules
-Add a new language
-Translate the native name of the new language using config translation
-Add a new language field to the content listing view checking "Display in native language"
-Create nodes in different languages
-Check the language translation

without the patch the languages are shown in English
With the patch the languages are translated.

I also reviewed the code and fixed a couple of comments and an indent problem.

Gábor Hojtsy’s picture

@rodrigoaguilera: thanks! Can you please post an interdiff (https://www.drupal.org/documentation/git/interdiff) so we can review the changes made? Thanks!

rodrigoaguilera’s picture

FileSize
1.15 KB

here it is

jhodgdon’s picture

Great! #45 marked it RTBC. #46 was a reroll. Interdiff to the latest patch looks good, and there was a manual test even. And I agree that the patch looks good, so I'm also +1 on RTBC.

However:
- The issue summary doesn't seem to match the patch.
- We need a beta evaluation added to the summary.

Gábor Hojtsy’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Updated the summary. Looking at how 3 removed field handlers had linking features (user and node configurably and taxonomy automatically), not sure we should get this in without that. What is so wrong with having an EntityLanguage field handler and no Language field handler for non-entities? That is to only support the entity case for language fields?

Status: Needs review » Needs work

The last submitted patch, 48: 2384863-native-language-48.patch, failed testing.

jhodgdon’s picture

Looks like the patch doesn't apply now...

Regarding not having a non-entity Language field available in Views... I am not sure whether or not that is OK. For other fields, Views has maintained non-entity versions for base database data. But maybe Language is so entrenched in our entity system that having a langcode field in a non-entity table doesn't even make sense?

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
13.36 KB

Looking into implementing the entity and the entityless one... I think we spent enough time debating if we need it, we should just go ahead and do it or we'll agonize on it for eternity. There are core tables with langcode that are not entities, eg. locale and path alias tables.

Gábor Hojtsy’s picture

Assigned: Unassigned » Gábor Hojtsy
Gábor Hojtsy’s picture

So I went to add entity linking support and by researching how best to do that realized, that it would be better to just extend the existing string FIELD formatter which requires a minimal subset of #2342045: Standard views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency, only the parts absolutely needed here. It makes sense to have the same features in entity field displays as in views field displays and not to need that implemented twice. So we can get rid of the views language field entirely hopefully in favor of the field formatter which can then get the linking feature as inherited from the string formatter easily. Looking forward to what testbot thinks of this one :)

Status: Needs review » Needs work

The last submitted patch, 58: 2384863-views-native-language-render-58.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
23.08 KB

Restoring the language views field, no specific tests yet (only restored pre-existing changes from #56 in views.field.schema.yml and LanguageField.php in views). @dawehner points out we should support locale like use cases. See eg. #1853534: Reintroduce Views integration for locale.module.

Gábor Hojtsy’s picture

Resolving fails by undoing some misguided plugin_id changes in shipped views. Filters, sorts, etc. for languages should not be switched to field, just fields :)

Gábor Hojtsy’s picture

Use the same native_name naming for the views field and add tests for that too.

The last submitted patch, 60: 2384863-views-native-language-render-59.patch, failed testing.

dawehner’s picture

+++ b/core/modules/views/src/EntityViewsData.php
@@ -300,7 +300,7 @@ protected function mapSingleFieldViewsData($table, $field_name, $field_type, $co
       case 'language':
-        $views_field['field']['id'] = 'language';
+        $views_field['field']['id'] = 'field';

So I guess we still need to drop this change for now, right?

Gábor Hojtsy’s picture

@dawehner: no, why? Then we loose the linking to the entity feature which the field formatter implements.

The last submitted patch, 61: 2384863-views-native-language-render-60.patch, failed testing.

Gábor Hojtsy’s picture

FileSize
3.29 KB

Forgot interdiff for the last change.

Status: Needs review » Needs work

The last submitted patch, 62: 2384863-views-native-language-render-62.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
20.9 KB
2.36 KB

Resolve the schema fails. The field needs a type key to specify the field formatter. I messed up a view and the schema fix discovered a bug in another view (minor).

Gábor Hojtsy’s picture

Fixing more fails. The summary need to be returned so it can be altered and the entity views data test should expect the new thing.

jhodgdon’s picture

Status: Needs review » Needs work

Looking good!

Nitpicks:

a)

+  public function settingsSummary() {
+    $summary = parent::settingsSummary();
+    if ($this->getSetting('native_name')) {
+      $additional_summary = $this->t('Displayed in native name');

This last line should be "Displayed in native language". oops.

b) I don't think settingsSummary() is returning a value? oops. [Oh, you fixed that in the latest patch while I was reviewing, good!]

c) I'm unclear on why core/modules/views/src/Plugin/views/field/LanguageField.php changed the setting machine name from native_language to native_name? I guess it was for consistency with the Field API plugin, but it seems like most of the previous specific Views plugins like node_language used "native_language" as the setting name too. Why make this change? I don't see any discussion above, and I don't see "native_name" anywhere in Core prior to this patch.

The last submitted patch, 69: 2384863-views-native-language-render-69.patch, failed testing.

The last submitted patch, 70: 2384863-views-native-language-render-70.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
21.6 KB
7.24 KB

So I renamed to native_name because the option toggles to display the native name of the language, not the native language per say. We can also think it displays the name in the native language of the language, which is slightly more complex but explains native_language :D I opted to do the rename because it seemed like we are getting rid of the views field and at that point there was no precedent for the setting. Then I added back the views field, so now it again has a precedent. There is no precedent for native_language btw outside of this issue either.

Gábor Hojtsy’s picture

Assigned: Gábor Hojtsy » Unassigned

So now this seems to be down to two fails:

1. NodeLanguageTest fails on no 'entity_type: X' value on the field. That should be a relatively easy fix I hope :)
2. HandlerAllTest fails with

Fatal error: Cannot use Drupal\Component\Utility\Xss as Xss because the name is already in use in /var/lib/drupaltestbot/sites/default/files/checkout/core/modules/views/src/Plugin/views/field/Field.php on line 11
FATAL Drupal\views\Tests\Handler\HandlerAllTest: test runner returned a non-zero error code (255).

which is to me a huge WTF. I found core/modules/views/src/Plugin/views/field/Boolean.php uses it as UtilityXss for some reason (also puzzling), but that sounds like may have been the workaround for this WTF. Why is that happening?

May be able to continue work on this tomorrow, not tonight anymore.

Status: Needs review » Needs work

The last submitted patch, 74: 2384863-views-native-language-render-74.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
22.68 KB
1.08 KB

So the HandlerAllTest fail is due to how there is a core/modules/views/src/Plugin/views/field/Xss.php, so the Xss class used in core/modules/views/src/Plugin/views/field/Field.php leads to the conflict. We should rename the Xss utility class used in Field.php to UtilityXss like in Boolean.php. Thanks @dawehner for the guidance. Should now only have one test failing.

Status: Needs review » Needs work

The last submitted patch, 77: 2384863-views-native-language-render-77.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Postponed

The remaining fails need changes in how views handles field formatters. There are several details not provided for them to work. Postponing on #2407801: Views generic field handler does not work with base fields.

Berdir’s picture

@dawehner suggested that I add a comment here that it would be useful to be able to print out the language code and not the label (For example, in feeds and other kind of data exports). Maybe add that as an option here while all this stuff is being worked on. Or some sort of output rewriting, does not have to be easy, just possible :)

Gábor Hojtsy’s picture

@Berdir: maybe I made a mistake by doing this with the field formatter instead of copying the thing in views, but that does not mean I would not love to keep scope control somehow :) I suspect views output rewriting may support that already, or if not, it would be a great addition in another issue. This was supposed to resolve the falsely advertised option in views and is already adding new field formatter options because that was the fix to make less copy-paste code necessary.

Gábor Hojtsy’s picture

Status: Postponed » Needs work

#2407801: Views generic field handler does not work with base fields now landed. Should pass now, although it will not apply yet.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
21.26 KB

Rerolled. Should hopefully pass now given prior fails were all from #2407801: Views generic field handler does not work with base fields.

The last submitted patch, 77: 2384863-views-native-language-render-77.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 84: 2384863-views-native-language-render-84.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
1.34 KB
21.26 KB

First fix config immutability.

Status: Needs review » Needs work

The last submitted patch, 87: 2384863-views-native-language-render-87.patch, failed testing.

Gábor Hojtsy’s picture

Part of the reasons for the fails is that EntityViewDisplay.php receives the right rendered value BUT then it does not get the right value for access in buildMultiple():

          $build_list[$key][$field_name] = $formatter->view($items);
          $build_list[$key][$field_name]['#access'] = $items->access('view');

So although the built view value will be right, the access nulls that out. I don't get why would that happen, the entities where we have the language field on are published, otherwise they would not show up in my view. What else should decide field visibility on this item?

Gábor Hojtsy’s picture

It would be amazing to get one more pair of eyes on this, I feel like I spent way too much time on this :(

dawehner’s picture

FileSize
1.77 KB

I think the problem is language_entity_field_access(), see interdiff.

It at least let more of the tests pass, some still have a different behaviour, but I have seen similar problems in the other issue as well.

Gábor Hojtsy’s picture

Title: Native language settings in views fields are no-ops » Language base field handler should use views field handler, provide unified options
Priority: Normal » Major
Parent issue: » #2407801: Views generic field handler does not work with base fields

Ah, that makes sense. Retitling and elevating as a sub-issue of #2407801: Views generic field handler does not work with base fields because this implements that for the language base field basically. Not sure of sub-issues of that like this one should be critical or not.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
22.36 KB
2.02 KB

Updated patch includes the one line fix from @dawehner with some docs updates and another one line cleanup I found yesterday while debugging.

Status: Needs review » Needs work

The last submitted patch, 93: 2384863-views-native-language-render-93.patch, failed testing.

Gábor Hojtsy’s picture

Doh. So the remaining fails seem to be two things:

1. The field is not actually rendered in the language of the row found. That to me looks like the territory of #2394883: Language setup for entity and field based rendering in views is independent, confusing UI, lacking test coverage. We cannot really decide here which translation to take. Field::process_entity() has some logic, but it is in the realm of #2394883: Language setup for entity and field based rendering in views is independent, confusing UI, lacking test coverage to resolve that to support row language. We can of course avoid to test for a nonexistent feature that we are not going to resolve either by not having translated entities in the view. Going to try that next.

2. When switching the config to the views built-in field, it does not actually use that to render the language names, it keeps using the prior setup (I assume due to some kind of caching). Will look into how to resolve that.

Gábor Hojtsy’s picture

So this fixes #96/1 by not testing for the values of a translated node, because #2394883: Language setup for entity and field based rendering in views is independent, confusing UI, lacking test coverage is not yet resolved. #97/2 is still intact and that is not due to caching. The formatter based field handler is used all the time instead of my specified one. Don't know yet why.

yched’s picture

I don't really feel qualified to vet the approach or actual code, just one minor remark :

+++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/StringFormatter.php
@@ -127,7 +128,7 @@ public function viewElements(FieldItemListInterface $items) {
       // The text value has no text format assigned to it, so the user input
       // should equal the output, including newlines.
-      $string = nl2br(String::checkPlain($item->value));
+      $string = $this->viewValue($item);

The comment would now be more relevant in the class' implementation of viewValue() :-)

Gábor Hojtsy’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 97: 2384863-views-native-language-render-97.patch, failed testing.

Gábor Hojtsy’s picture

Gábor Hojtsy’s picture

Status: Postponed » Needs work

That one landed. Back here.

tstoeckler’s picture

+  public function settingsSummary() {
+    $summary = parent::settingsSummary();
+    if ($this->getSetting('native_language')) {
+      $additional_summary = $this->t('Displayed in native language');
+      if (!empty($summary['#markup'])) {
+        $summary['#markup'] .= '; ' . $additional_summary;
+      }
+      else {
+        $summary['#markup'] = $additional_summary;
+      }
     }
+    return $summary;
+  }

Settings summary should simply return an array of strings, not a render array.

See FormatterInterface::settingsSummary().

Gábor Hojtsy’s picture

Well, the existing settingsSummary() on StringFormatter (the base class) returns a render array *before this patch*. The API does not define if it would be a render array or a simple string array?!

Gábor Hojtsy’s picture

Well looking at https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Field%21F... string formatter is the only outlier. Duh. Will fix it here then.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
21.28 KB

First rerolling #93. Ignore #97 as the issue that those fails were due to is now in :) So we can avoid removing some of the test coverage as #97 did and instead move back to the fuller patch in #93.

Did not yet address the feedback from @tstoeckler and @yched but intend to do so.

Status: Needs review » Needs work
vijaycs85’s picture

In this patch:

1. Addressed both #98 and #103.
2. Re-exported views.view.language_test.yml

TODO: Fix test fails.
Looks like the new language field always gets the current language instead of the item's language at below $item:

+++ b/core/modules/views/src/Plugin/views/field/LanguageField.php
@@ -40,11 +45,9 @@ public function buildOptionsForm(&$form, FormStateInterface $form_state) {
+    $languages = $this->options['native_language'] ? \Drupal::languageManager()->getNativeLanguages() : \Drupal::languageManager()->getLanguages();
+    return isset($languages[$value]) ? $languages[$value]->getName() : '';

Status: Needs review » Needs work

The last submitted patch, 108: 2384863-native-languge-108.patch, failed testing.

Gábor Hojtsy’s picture

  1. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
    @@ -55,10 +55,10 @@ public function settingsSummary() {
           if (!empty($summary['#markup'])) {
    -        $summary['#markup'] .= '; ' . $additional_summary;
    +        $summary[] .= '; ' . $additional_summary;
    

    Given you fixed the base class, this will never happen :) We should just add another item to the summary array (only keep the else case of this one).

  2. +++ b/core/modules/node/tests/modules/node_test_views/test_views/views.view.test_language.yml
    @@ -159,12 +162,24 @@ display:
    +          click_sort_column: value
    +          type: language
    +          settings:
    +            link_to_entity: false
    +            native_language: false
    +          group_column: value
    +          group_columns: {  }
    +          group_rows: true
    +          delta_limit: 0
    +          delta_offset: 0
    +          delta_reversed: false
    +          delta_first_last: false
    +          multi_type: separator
    +          separator: ', '
    +          field_api_classes: false
    

    The reason I did not have these was that when you turn these between a plugin_id field and language, not all options will be valid. As you can see in the fails, you started to get config schema fails due to these. Instead of needing to remove a whole bunch of items, I removed the ones that were used with their default values, so the test is cleaner. If you add them back, you also need to add removal code to the test when you switch the plugin id of the field.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
23.4 KB
3.21 KB

Thanks @Gábor Hojtsy.
#110.1 - Just updated the if condition, so the summary gets same text.
#110.2 - Yep, rolled back changes I made to views in #108.
Fixed few test fails as per our discussion on IRC.

Status: Needs review » Needs work

The last submitted patch, 111: 2384863-native-languge-111.patch, failed testing.

Gábor Hojtsy’s picture

  1. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
    @@ -22,19 +23,53 @@
    +      if (!empty($summary)) {
    +        $summary[] .= '; ' . $additional_summary;
    +      }
    +      else {
    +        $summary[] = $additional_summary;
    +      }
    

    I don't think you implemented what I suggested here?

  2. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/StringFormatter.php
    @@ -144,4 +143,19 @@ public function viewElements(FieldItemListInterface $items) {
    +  protected function viewValue(FieldItemInterface $item) {
    +    // The text value has no text format assigned to it, so the user input
    +    // should equal the output, including newlines.
    +    return nl2br(String::checkPlain($item->value));
    +  }
    

    This should be done conditional on a flag in a Drupal state. If you do this unconditionally, then the first part of the test is futile, the field formatter is never tested.

    Also the comment is outdated here.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
23.29 KB
2.52 KB

Here is a short update for those.

Status: Needs review » Needs work

The last submitted patch, 114: 2384863-native-language-114.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
20.33 KB
710 bytes

Fix the state checking in the test module code, so it fails again "the right ways". Still problematic with row language :/

Gábor Hojtsy’s picture

Duh, patch was somehow missing all NodeLanguageTest changes. Huh.

The last submitted patch, 116: 2384863-native-language-116.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 117: 2384863-native-language-117.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
24.89 KB
1.59 KB

Debugging the fails.

1. So langode is the worst animal to do this on :D First it is not translatable, so the translation langcode was always und. Fixing that in Field::getFieldLangcode().
2. However the langcode field value on the entity is still the original one and not the translation. Not sure how best to make that one use the appropriate langcode for translation. Added a @todo for that. I'll try to summon @plach for that to help :(

Status: Needs review » Needs work

The last submitted patch, 120: 2384863-native-language-120.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
24.95 KB

Rerolled. The user language field was changed for URL API changes, but we remove it anyway :D

jhodgdon’s picture

I'm trying to organize the "fix this field" issues and make sure there are issues for all of them, so making sure they're all listed on #2393339: [META] Make sure Views base fields are using Field API for formatting, and do not lose functionality as a start.

Status: Needs review » Needs work

The last submitted patch, 122: 2384863-native-language-122.patch, failed testing.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
25.21 KB
996 bytes

clearing viewData cache seem to fix the 4 fails...

Status: Needs review » Needs work

The last submitted patch, 125: 2384863-native-language-125.patch, failed testing.

Gábor Hojtsy’s picture

@vijaycs85: yay, thanks! Now only the rendering language problem for the langcode needs to be fixed. @plach: do you have ideas?

jhodgdon’s picture

So I'm a bit confused here about the rendering problem.

There are actually 2 language fields that Views is allowing us to add to views: the "original language" of the node (from the base node table) and "translation language" (from the node field data table).

If we're using Field UI rendering... can we actually get at both of these database fields? Is the problem that Field rendering says "Oh, you want a field called 'langcode'. I see that is on the node table, let's use that" rather than checking the field data table? Really it needs to figure out that in the View you have chosen either the one on node base or the one on the field data table, and get that data, rather than trying to know which one to choose.

I'm just kind of guessing here, based on what you said in #120, trying to understand what's happening.

Gábor Hojtsy’s picture

As far as I understand, the entity field API will not render a translation of a non-translatable field, that does not make sense. The langcode field is not translatable. To display the proper "translation" of the langcode field, that would need special casing in the entity field API. In #120 I already fixed Field::getFieldLangcode() to special case the langcode field when looking up the translation language (second part of https://www.drupal.org/files/issues/interdiff_10167.txt). Still when we attempt to actually render the field in that translation, it is not different in that translation because its not translatable. That still needs fixing.

The same problem should obviously not appear for other fields, because if they are not translatable they will *actually* not be different in the translations. Unlike language which is a special flower.

jhodgdon’s picture

Ah. So maybe we shouldn't use the basic 'field' handler for language fields then? We may just need to subclass that handler and do something different?

Gábor Hojtsy’s picture

@jhodgdon: I think it would be more uniform if Field.php could handle it, but either way it will need to have that code for the language field somehow either there or in a subclass.

jhodgdon’s picture

Yeah, it does seem like a special case.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
24.96 KB

Patch did not apply anymore, rerolled.

Gábor Hojtsy’s picture

Assigned: Unassigned » plach

I tried my best to butcher my way into rendering the right language. Once again the obvious reason for the fails is that the field langcode is different from the entity langcode. ContentEntityBase keeps only one copy of non-translatable fields and even if we attempt to change that one copy of the langcode field it freaks out because such a translation already exists of course. So I went to remove the translation but that cannot be removed because its the default langcode... Duh. So this really needs some adult supervision. I cannot even butcher my way into a success on output let alone any meaningful code solution.

This would not work:

diff --git a/core/modules/views/src/Plugin/views/field/Field.php b/core/modules/views/src/Plugin/views/field/Field.php
index c9bc2f2..3a6536f 100644
--- a/core/modules/views/src/Plugin/views/field/Field.php
+++ b/core/modules/views/src/Plugin/views/field/Field.php
@@ -713,6 +713,7 @@ public function getItems(ResultRow $values) {
     if (!$original_entity) {
       return array();
     }
+    /** @var \Drupal\Core\Entity\ContentEntityInterface $entity */
     $entity = $this->process_entity($values, $original_entity);
     if (!$entity) {
       return array();
@@ -723,9 +724,18 @@ public function getItems(ResultRow $values) {
       'settings' => $this->options['settings'],
       'label' => 'hidden',
     );
-    // @todo (in this patch) make this able to render the langcode field itself
-    // for this translation.
-    $render_array = $entity->get($this->definition['field_name'])->view($display);
+    /** @var \Drupal\Core\Field\FieldItemListInterface $field */
+    $field = $entity->get($this->definition['field_name']);
+    $langcode_key = $entity->getEntityType()->getKey('langcode');
+    if ($this->getFieldDefinition()->getName() == $langcode_key) {
+      // The language field it not translatable, so set the value needed for
+      // the span of rendering. We need to remove the corresponding translation
+      // temporarily to be able to do so.
+      $langcode = $entity->language()->getId();
+      $entity->removeTranslation($entity->language()->getId());
+      $field->setValue($langcode);
+    }
+    $render_array = $field->view($display);
 
     $items = array();
     if ($this->options['field_api_classes']) {
jhodgdon’s picture

I'm confused about what is working and what is not working here.

Status: Needs review » Needs work

The last submitted patch, 133: 2384863-133.patch, failed testing.

Gábor Hojtsy’s picture

@jhodgdon: so the problem is the language field itself is not translatable; so it does not have values for translations; you cannot therefore render a language field in a translation, it will always use the base value (original language of entity) as it is not translatable.

I fixed Field::getFieldLangcode() with this code to special case the langcode key so it is capable of looking up the right translation:

    // Retrieve the translation language code for this field. Even though
    // the language field is not translatable (it is what is used to key
    // translations), apply the language code lookup for that too, because that
    // allows us to display the language name itself in the right language.
    $langcode_key = $entity->getEntityType()->getKey('langcode');
    if ($this->getFieldDefinition()->isTranslatable() || $this->getFieldDefinition()->getName() == $langcode_key) {

So that works around the case, that the langcode field is not translatable. So that makes Field::getFieldLangcode() find the right translation. Then Field::process_entity() uses that to:

    $langcode = $this->getFieldLangcode($processed_entity, $values);
    $processed_entity = $processed_entity->getTranslation($langcode);

So we have the right translation from then on. And then the actual rendering is in Field::getItems():

$entity->get($this->definition['field_name'])->view($display)

The Entity::get() here will skip looking for translations for this field, because .... its not translatable. Yeah. So although the entity language is cool here, the field will never have that language, because its never translatable. The butchering I tried above is to try and set the field value anyway temporarily, but you cannot do that because the entity default language would change, and you already have a translation in that language.

jhodgdon’s picture

It seems like the only way around this is to make a special subclass of the Field views-field-handler-plugin class to handle the Language field(s), rather than trying to make the generic Field handler work.

Gábor Hojtsy’s picture

Yeah my problem is I don't know what code would go in that class. Once we know the code, its easy to put it into Field or a subclass of Field, either way :) I am not sure how to get around the very tight and otherwise right protection around the language field :)

plach’s picture

I started looking into this, not done yet...

plach’s picture

@Gabor:

I tried a few experiments without luck, I'm afraid this is telling us there's something wrong with our current model. Basically here we want to display the translation language, but there is no such field defined. Given that any piece of data attached to a (content) entity needs to be a field, we are basically missing a (translatable) field definition for the translation language.

Now, if we look at our API, we already have the translation language in the form of the ContentEntityInterface::language() method, and we have the default language via ContentEntityInterface::getUntranslated(). The concepts are there, but they are not backed by corresponding field definitions.

So what I tried to do (I'm not really done yet, wanted to get some feedback before coding too much), is going a pretty radical way and make our langcode definition translatable. If we look at our standard schema the {entity_field_data}.langcode is actually translatable, as we have a different value for each translation. The missing part is a default_langcode untranslatable field definition allowing us to tell which is the default translation.

After overcoming the initial shock, I realized this change seems to fit pretty well our current architecture: for instance this would allow us to have a language selector on content translation forms letting us change the translation language. This is not a big use case probably, but I think it clarifies pretty well that the direction should be correct.
Another example is that this way Rules would be able to distinguish between the various translation objects natively, by simply relying on field definitions, without needing to special case the translation language.

If you agree this is an approach worth trying we should open a separate issue and postpone this on that.

Btw, I tweeted about this at https://twitter.com/plach__/status/568937859844956161

Berdir’s picture

Not sure I fully understand this yet.

This would essentially mean the field definitions would be the way the storage already is, and it wouldn't actually change the storage, correct? So less magic needed in the storage that makes the non-translatable langcode field actually stored per translation and the non-field default_language column.

Sounds like a change that makes sense, I've wondered before why it is not translatable, $node->getTranslation('de')->langcode->value isn't what you expect it is, as you said yourself.

Translatable vs. localizable, given we make the language field translatable, how does content_translation know that it is not actually a field that can be "translated" by the user? Or can it, in a way, based on your comment?

As TMGMT maintainer, I have yet another meaning of "translatable" to think about: The parts of an entity that I can extract and send to a translator, based on the tell that I sent you, my current code started picking up the content_translation_source field (with a value 'und') as a translatable text because it was translatable. But I think that can be solved by adding an options provider, which is also required for validation, @fago already opened an issue for that.

yched’s picture

+1 on #141, makes a lot of sense IMO.
(if, by any chance, the refactor happened to also clean up the fact that ContentEntityBase internally stores the field values of the default language under 'und' rather that the actual langcode, I'd be double yay :-) But chances are this is an orthogonal refactor ?)

Regarding content_translation :
I have lost track a bit on whether BFD::setTranslatable(TRUE) means
"the field *is* translatable on all bundles, no config will ever change that"
or
"the field *supports* translatability, and content_translation UI can configure it to actually be translatable or not on a given bundle".

Because the latter doesn't sound like the behavior we'd want for that "translatable language field". If the entity type & bundle are configured to be translatable, I shouldn't be able to keep the "language" field untranslatable, that would be very broken.

plach’s picture

Status: Needs work » Postponed

Sounds good, created #2431329: Make (content) translation language available as a field definition and replied there. Marking this postponed again.

fago’s picture

This would essentially mean the field definitions would be the way the storage already is, and it wouldn't actually change the storage, correct? So less magic needed in the storage that makes the non-translatable langcode field actually stored per translation and the non-field default_language column.

yeah, the change makes a lot of sense to me as well - so the field definitions really show you how things are stored. Clarity++. It's quite a bit of change for the meaning of $entity->langcode but given language() already works different, I think aligning them to work the same way makes it work as developers would expect it.

However, I do think I miss what it buys us here. A possibility to easily render the used display language?

Gábor Hojtsy’s picture

@fago: yes, the problem this is blocked on is the translations' langcode cannot be rendered as a field (because its not a translatable field). See #137.

Gábor Hojtsy’s picture

Issue summary: View changes
jhodgdon’s picture

Like fago, I'm still a confused about why we need to make langcode a translatable field.

It seems like what we need for this issue is

a) An entity-aware method to use as a Views formatter, which would print out the language of the translation being displayed, with the option to display the language name either in the defined Views display language for that row, or in the language's native language.

b) The same as (a) but to print out the original language of the entity.

The solution proposed currently in this issue is to use the generic Entity Field views formatter for these values, but this is not working because langcode is not currently a translatable field. So the proposal here is to make langcode be a translatable field.

It seems to me that an alternative solution would be to make two new formatters for "entity-aware translation language" and "entity-aware original language" that would use the existing language methods on ContentEntityInterface to discover what language to print, and then print it out. I don't understand why that wouldn't work?

We may have other reasons, as discussed above, for making langcode a translatable field, but it seems like the "make a non-generic formatter for Views" solution would be much easier, and would solve the problems of this issue?

Gábor Hojtsy’s picture

@jhodgdon: so the fields are exposed as individual fields supported in views (original and translation language). If we are to make individual formatters for them too, then how do you tie them so they are only ever to be used with the original and translation language, so you cannot render your original language with the translation language formatter, etc? Field formatters are tied to field types not views exposed data blobs, no?

jhodgdon’s picture

Yes, technically formatters are independent. However, they are specified in the entity views data, not by users, so this isn't a problem.

So technically, you could put anything in for the plugin ID for a field handler in your entity views data, like assigning the custom 'node' formatter to be the formatter of the 'status' field on the User table. But you choose not to do that, because it wouldn't work if you did.

Similarly, we would choose to use the correct formatters in the entity views data for the language fields.

Gábor Hojtsy’s picture

So one of the points of this issue was to use the field formatter so that we don't need to redo the same code in views. If we are to have separate classes for the original language and translation language, and then need to reimplement the formatting code there, that sounds like a lot of duplication no?

jhodgdon’s picture

That is a point... As I see it, I think the reasons we need this issue are:

a) There are currently several custom language filters for specific entities, which have duplicated code, mostly so they can link to the entity and also hopefully display the language name in the native language (which is most likely not working).

b) Any entity without a custom language filter is using the base Views handler for non-entity-aware language fields. This is not correct, because:
- entity access is not checked
- it will just be displaying the language code from the database that is in the views query result, rather than the language of the translation that was selected for display for the entity via the Language settings of the view.

So the two proposals for solving this are:

1. Use the generic entity Field handler for all entity-related Language fields. But this will only work for the "language of this translation" field I think, and only if we also make the "language of this translation" field be a real, translatable field, which it isn't currently, and also if we make the "original language" be a real, non-translatable field, which it also isn't currently.

2. Make two specific Views handlers to handle the "language of this translation" and "original language" pseudo-non-fields. We might even be able to combine them into one handler with a setting to say whether to output the translation or original language, since the only thing that differs between the two cases is how to read this value from the $entity object.

I am not sure which of these is easier, but my guess is that (2) is easier, and also I think that (2)'s fix is more localized to solving the problem of dealing with language fields of entities in Views, while (1) is a more widespread fix. I think that both of them would solve the problem of this issue; we may have other reasons for wanting to do (1) but getting it actually done is, I'm pretty sure, going to be more work and have a lot of other implications on Drupal Core.

plach’s picture

Assigned: plach » Unassigned

If we want to keep the current approach and avoid special-casing the (translation) language field, I don't see other solutions than making the langcode field translatable. I wouldn't mind if a different solution were adopted here, but I think this issue uncovered a flaw in our current architecture so I will work on #2431329: Make (content) translation language available as a field definition anyway.

Gábor Hojtsy’s picture

All right, let's see if #2431329: Make (content) translation language available as a field definition is not possible "in time" for this one and then return to resolve it differently.

jibran’s picture

plach’s picture

Status: Active » Needs work

No need to start from scratch I'd say ;)

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
25.26 KB

First a reroll.

Gábor Hojtsy’s picture

Gábor Hojtsy’s picture

FileSize
23.68 KB
1.59 KB

We can roll back the Field.php changes now.

Gábor Hojtsy’s picture

Title: Language base field handler should use views field handler, provide unified options » Translation language base field handler should use views field handler, provide unified options
Issue summary: View changes

Updated title and issue summary to set scope of this to translation language only given that we need to resolve default language in #2450195: Original language of entities not accessible in views anymore.

Wim Leers’s picture

+++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
@@ -22,22 +23,49 @@
-    // The 'language' cache context is not necessary, because what is printed
-    // here is the language's name in English, not in the language of the
-    // response.
...
-      $elements[$delta] = array('#markup' => $item->language ? String::checkPlain($item->language->getName()) : '');

Note that this was added because the docs for ::getName() say this:

  /**
   * Gets the name of the language.
   *
   * @return string
   *   The human-readable English name of the language.
   */
  public function getName();

So if that's wrong, the interface should be updated as well.

Gábor Hojtsy’s picture

FileSize
24.22 KB
559 bytes

Fixed the interface. It really depends on the construction of the object. Most languages are config entities which may or may not be loaded in some translation.

Wim Leers’s picture

Status: Needs review » Needs work
Related issues: +#2449459: [PP-1] Make URLs cache context aware
+++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/StringFormatter.php
@@ -144,4 +143,19 @@ public function viewElements(FieldItemListInterface $items) {
+  /**
+   * Generate the output appropriate for one field item.
+   *
+   * @param \Drupal\Core\Field\FieldItemInterface $item
+   *   One field item.
+   *
+   * @return string
+   *   The textual output generated.
+   */
+  protected function viewValue(FieldItemInterface $item) {

This is incapable of returning associated cache contexts & tags… which AFAICT means LanguageFormatter cannot actually extend StringFormatter and implement ::viewValue().
Alternatively, make this return a render array with #markup containing the string, then #cache[context] can be set as necessary. StringFormatter must then merge the cacheability metadata.

That will then allow you to resolve the language cache context @todo.

Setting the necessary cache contexts for the link (in case the link setting is enabled) will be taken care of in #2449459: [PP-1] Make URLs cache context aware — no work needs to be done for that here.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
24.13 KB
867 bytes

Discussed this with @WimLeers and I in fact started to refactor viewValue() for render arrays. But turns out we don't need to do that. The language value is either displayed in the original stored configuration value or the native name of that language. It does not depend on a language context, regardless of displaying the languages on a Chinese page, the formatter would either display the language in its configured original name or in its native name. Not in Chinese.

When the field formatters change the caches are already invalidated, so since this only varies by the data and the formatter setting, we are fine as-is. Removing the @todo.

Patch should be complete.

Wim Leers’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
    @@ -22,22 +23,47 @@
    -    // The 'language' cache context is not necessary, because what is printed
    -    // here is the language's name in English, not in the language of the
    -    // response.
    

    I wonder if we want to keep a comment like this, that uses the German/Deutsch example, explaining that you'd always see EITHER of those two strings depending on the field settings, and not a translation of it (such as "Allemand" in French or "Duits" in Dutch).

  2. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
    @@ -22,22 +23,47 @@
    +    $languages = $this->getSetting('native_language') ? \Drupal::languageManager()->getNativeLanguages() : \Drupal::languageManager()->getLanguages();
    

    Should or shouldn't we inject these?

  3. +++ b/core/modules/language/language.module
    @@ -497,10 +497,10 @@ function language_field_info_alter(&$info) {
    +  if ($items && $operation == 'edit') {
    

    Wow, nice catch!

    I think this points at missing test coverage?

  4. +++ b/core/modules/node/src/Tests/Views/NodeLanguageTest.php
    @@ -226,4 +227,55 @@ public function testLanguages() {
    +      $this->assertText('Français', 'French language shown translated.');
    +      $this->assertText('Español', 'Spanish language shown translated.');
    ...
    +      $this->assertNoText('Français', 'French language not shown translated.');
    +      $this->assertNoText('Español', 'Spanish language not shown translated.');
    

    s/translated/in native form/

  5. +++ b/core/modules/node/tests/modules/node_test_views/node_test_views.views.inc
    @@ -0,0 +1,16 @@
    +  if (Drupal::state()->get('node_test_views.use_basic_handler')) {
    

    Missing leading backslash.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
26.57 KB
5.89 KB

1. Added.
2. Injected in the formatter. I have no idea how would a views handler allow for injecting services (minus adding a method for it specifically), so did not do it there.
3. Well it failed when we did not make this change, so it does have test coverage :) We did not use the language field access checking for view operations before, only for edit. We don't have a view specific limitation, it should always be visible.
4. Good point, fixed.
5. Fixed.

Wim Leers’s picture

Only nits and nitty nits. Patch looks good, but I can't really sign off on the Views stuff, so keeping at NR.

  1. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
    @@ -26,6 +29,58 @@
    +   * Constructs a StringFormatter instance.
    

    LanguageFormatter :)

  2. +++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/LanguageFormatter.php
    @@ -8,8 +8,12 @@
    +use Symfony\Component\DependencyInjection\ContainerInterface;
    +use Drupal\Core\Language\LanguageManagerInterface;
    

    Nit: we usually sort these alphabetically.

  3. +++ b/core/modules/node/src/Tests/Views/NodeLanguageTest.php
    @@ -226,4 +227,55 @@ public function testLanguages() {
    +   * Assert the presence of language names in their English or native forms.
    

    Nit: Asserts.

Gábor Hojtsy’s picture

FileSize
26.58 KB
1.62 KB

All those changes make sense.

dawehner’s picture

+++ b/core/modules/node/tests/modules/node_test_views/node_test_views.views.inc
@@ -0,0 +1,16 @@
+ * Implements hook_views_data_alter().
+ */
+function node_test_views_views_data_alter(array &$data) {
+  // Make node language use the basic field handler if requested.
+  if (\Drupal::state()->get('node_test_views.use_basic_handler')) {
+    $data['node_field_data']['langcode']['field']['id'] = 'language';
+  }

+++ b/core/modules/views/src/EntityViewsData.php
@@ -318,7 +318,7 @@ protected function mapSingleFieldViewsData($table, $field_name, $field_type, $co
-        $views_field['field']['id'] = 'language';
+        $views_field['field']['id'] = 'field';
         $views_field['argument']['id'] = 'language';

Do we still need the toggle?

Gábor Hojtsy’s picture

@dawehner: that is to test the views built-in handler (that does not use a field formatter, because it is not for entity based views) so we have a solution for a language field on a non-entity like locale or URL alias data. You requested that above with a reference to #1853534: Reintroduce Views integration for locale.module.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Ah, this makes sense.

This looks perfect for me.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Nice patch. This issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed 8d87a92 and pushed to 8.0.x. Thanks!

  • alexpott committed 8d87a92 on 8.0.x
    Issue #2384863 by Gábor Hojtsy, vijaycs85, rodrigoaguilera: Translation...
Gábor Hojtsy’s picture

Issue tags: -sprint

Thanks all!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.