Original language of entities not accessible in views anymore

Uck. Good catch.

Since Entities do not have a field or pseudo-field that will give you the language of the default translation, it doesn't seem like once we move to #2384863: Translation language base field handler should use views field handler, provide unified options that we would be able to use the same treatment for this views field, right? So it would just need to be a field using the bare Views language handler plugin, not the Entity Field plugin, and it would go against the langcode database field on the base entity table?

In other words, what we'd want to do I think is to special case the 'langcode' field in the EntityViewsData class, so that we are using the Field handler for the langcode database field on the entity field data table, and the bare Views language handler for the langcode database field on the base table.

Annnnd, there was much rejoicing!

I think it makes sense to special case the langcode field on the base table, but only to provide filter, contextual filter, and sort handlers. This would allow to support use cases like:

• List all translations of nodes having fr as default langcode
• List all term translations sorted by default langcode and name

Exposing a field handler not being an instance of the usual Entity Field API handler would trigger the usual issues with field access. What about having a subclass of the Entity Field API field handler dedicated to the entity langcode, exposing an option to use the default language as value? If not selected it would behave as the usual translation language. If needing to list both, one could add two langcode fields: one with the option enabled and the other with the option disabled.

I don't see it that way: we don't have a "default/original language" field, so there's nothing to pick. It's a common practice to have an option that affects the rendered value, so having a checkbox saying "Display default/original language" does not sound that weird to me.

An alternative could be providing a fake "Default language" field, not tied to any actual column, to be used only for display.

It is absolutely true, that we have quite some places in core (for example field formatters), which affect the shown value depending on some option.
I don't care much honestly, whether we want to follow that particular pattern here, as it all comes down to whether the site builders think about default language/translation language as something distinct or similar and hey, I'd say just go with one solution.

Given that for base/data table we have a split up in the database, I'd go with two fields instead of one.

I think it makes sense to special case the langcode field on the base table, but only to provide filter, contextual filter, and sort handlers. This would allow to support use cases like:

+1

To access the default language of a given entity, the entity either needs to be loaded or the base table for the entity needs to be joined.

We already load the entity, so it should be accessible.

Do we need to revive this issue? If not we should close it as Won't Fix.

OK we still need to revive this issue...

I tested this today:
- Install with Standard profile in English.
- Turn on the 4 multilingual modules.
- Add Spanish language.
- Make Node translatable (both content types).
- Add an English Article node translated to Spanish, and a Spanish Page node
translated to English.
- Add another node, originally marked as being in English. Edit it, changing the marked language to Spanish. Edit it again, this time turning on revisions, and change it back to English.

Then I made a Node view. When I go to add fields, I see this (filtering to things with "lang" in them):

So some of the descriptions may be slightly lame (like maybe they shouldn't have the word Node in them), but they are all OK and there is definitely a field for Original Language. Filter looks identical.

So then I added the 3 language fields to a Node view. Each row in the results is a node translation. The output is:

a) English article
Original: English
Translation: English
Translation source: Not specified

b) Spanish translation of (a)
Original: Spanish
Translation: Spanish
Translation source: English

c) Spanish page
Original: Spanish
Translation: Spanish
Translation source: Not specified

d) English translation of (c)
Original: English
Translation: English
Translation source: Spanish

e) Node whose language has changed a few times, and is now English:
Original: English
Translation: English
Translation source: Not specified
[Note: I also looked at the field values after I had changed the language to Spanish, and they were Spanish / Spanish / Not specified]

So. It looks like what these actually mean is:

- Original language is supposedly node.langcode. This should be the current language of the base node, but ... it isn't working right on output. I think the problem is that we are now using entity rendering for all field output, not results from a database query. So it says "Oh, langcode, I know what field that is." and renders langcode, which is the translation language, instead of getting the value of node.langcode, which would be the current language of the base node.
- Translation language node_field_data.langcode. It should be the language of the translation, and this is working right.
- Translation source is apparently node_field_data.content_translation_source. This seems to be working as I expect it to.

Now. On Filter, I can successfully use Original Language to filter based on what the original language of the node is. That works as I expect it -- when I filter to Spanish, I get only my originally-Spanish page. Translation language works too on Filter -- I get the translations/originals that are Spanish. And Translation Source works fine too.

But on Field, Original Language doesn't work. The only way to fix it would be to make it so we can access the node.langcode field as an Entity Field, and also somehow also have a way to render it in Views.

I will repeat this test with other entities, but I think this comment is long enough for Node.

I tested Taxonomy terms - similar test to #17 with similar content/translations (except they are Tag terms not Article nodes). Results:

a) The language fields need better descriptions. They are:
- Original language - The term language code.
- Translation language - The term language code.
- Translation source - The source language from which this translation was created.

===> Translation should say "of term or translation" like Node does, and none of them should say "language code", since what they are outputting is definitely the language not a "code".

b) The output using fields and filters works the same as for Node. So filters work; field Original Language doesn't.

Next test: User.

I found a bug (one of many I filed while testing today, sigh): when I made a view of users, the links shown were not localized to the languages. That is, both the translations linked to user/1 and one of them should have gone to es/user/1 to display in Spanish. This works correctly for Node and Taxonomy term views, but User isn't working right. But that's a separate issue.

So. The field labels are OK for User language fields, except that the Original one has "code" in it: "The user language code.". There are 5 fields:

Fields and filters work same as Node and Taxonomy.

Issue filed for the user link localization problem: #2582499: User views - user links are not localized

Oh holy .... OK. Now that I have made User translatable, and added a translation for user 1 (the author of all my content), when I go to the admin/content page I am getting 2 copies of everything. Filed: #2582535: Duplicated rows on admin/content if author is translated

Tested comment. Has the same duplication problem as User, which is generic and not really a bug per se. Being discussed a bit on #2582535: Duplicated rows on admin/content if author is translated.

Field labels need some tweaking. Functionality same as node/taxonomy/user as far as the language fields and filters.

And finally I looked at the fields for custom blocks, but I got tired and didn't make content and fully test the functionality... it should be the same as the other entities. The field descriptions do need some tweaking:

So. Updating the summary.

I took a look at the 'Original language' field not working, and it looks like the only problem is that the base 'field' views field handler is used, and not the 'language' views field handler. Switching the 'original language' over to the right handler seems to do the trick.

Not sure if this is the right place to set this, but since this is the place where the titles get set I figured it was a good place to start.

I haven't done a lot of manual testing, just a content view displaying both the translated and original language next to each other, and this is completely different then the discussed fixes . But I thought I'd share anyway.

So this still needs tests.

@Gabor Hojtsy, yeah was looking around for some kernel tests that set up translated content to display in a view, but couldn't find any, so no easy copy/paste tests for now. Moved the fix to a more logical spot I think and that clears up some of the tests fails locally, but not all, and lets see if it gives some new fails.

Still works with manual testing.

Ok some more cleanup. Commenting out some of the tests feels weird, but since the langcode now no longer uses the 'field' handler when using the base table, running tests for it like it does use the 'field' handler doesn't make sense (and there are a number of commented assertions in those test for some reason).

Moving the tests in EntityTestViewsFieldAccessTest over to entity_test_mul so the langcode can use the data table makes those tests pass. Nice illustration that the base/data table switching works for the langcode field.

This should move this closer to being green for existing tests, but still needs tests for the fix.

Missed one, not sure if the update tests are related, lets find out.

Now with a test, lets see what happens.

Interdiff is the test-only file.

+++ b/core/modules/views/src/EntityViewsData.php
@@ -436,7 +436,13 @@ protected function mapSingleFieldViewsData($table, $field_name, $field_type, $co
+        if ($table == $this->entityType->getBaseTable() || $table == $this->entityType->getRevisionTable()) {
+          // Use the language handler for the original language.
+          $views_field['field']['id'] = 'language';
+        }
+        else {
+          $views_field['field']['id'] = 'field';
+        }

Cann't we also use it for the translation langcode in the data table?

@dawehner well we could but then you would run into the issues that @Gabor describes in #36 for that field as well.

@Gabor well if I understand this correctly, the current fix only fixes the Views problem. The larger problem seems to be

As far as I know, you cannot access the original language of the node currently via Entity field rendering.

The current fix is actually a option c) for fixing this

c) Use a Views specific 'language' handler to render the original language field. This will not allow the use of field formatters but will allow the value to be shown.

Option b) is still the right way to go, but I do feel it's outside the scope of Views and probably more a global entity thing.

So the question would be, do we want a Views specific fix for now and follow up with a entity issue?

I have noticed that with this patch we lost the {{ langcode__value }} replacemenet pattern which was usefull to create a link back to the original language of the entity.
Is this something that can be avoided?

Is this issue still being worked on? It looks like it has been 9 months since the last activity but I still see this problem in the main content view when the Original language field is included. Instead of displaying the node's original language, it always displays the same value as the translation language.

Is this still a problem?

I tested this on 9.4.x, standard install with Italian and Spanish added, and was not able to reproduce this error. I had nodes and terms in a variety of original languages and translations. I setup a view that had fields to display the original and translation language for each entity. In all cases the original language and translation language were correct.

There hasn't been work on this problem for 6 years and maybe it has been fixed in the meantime. However, one year ago, @laraortiz, reports that they still have the problem, although the version of Drupal wasn't provided. So, instead of closing this as cannot reproduce I am setting the status to Postponed (maintainer needs more info) so that anyone experiencing this problem can confirm and provide complete steps to reproduce the issue (starting from "Install Drupal core").

Thanks!

Another seven months have passed and there has been no confirmation that this problem still exists. Therefor, I am closing this issue.

If you are experiencing this problem on a supported version of Drupal reopen the issue, by setting the status to 'Active', and provide complete steps to reproduce the issue (starting from "Install Drupal core").

Thanks!

Reopening.

Here is a video showing the issue starting from a Standard install of 10.3.9 on SimplyTest.me

https://www.youtube.com/watch?v=CZBmAEukp5g

Actually setting the status to Active rather than Needs work as explicitly requested by @quietone in #54

@eli-t, thanks for providing proof of the problem. Do you know if this happens on 11.x as well?

@quietone I've just tested this on Drupal 11.0.9 and can confirm the problem exists there too.

@eli-t, thanks.

Updating version because changes are made on on 11.x (our main development branch) first, and are then back ported as needed according to the Core change policies.