At least in the case of a solr backend translated entities are always rendered in the default language instead of the translation return by the index.

The issue is cause by missing meta data in the faked "search api table information" for the SQL flavored architecture of the views module.

I prepared a first (somewhat hackish) patch to fix the entity and field translation rendering of views rows generated via search_api.

With the patch applied you can create translation aware views including exposed language language filters.

To demonstrate the patch I created 3 articles: node 1 consists of an English and a Dutsch translation. Node 2 only exists in Dutch, node 3 only in English.

Original bug report:

I created two content Article in two languages EN(English) and NL(Dtuch). Indexed the same on to Solr. Everything is fine till this time.

Now When I am using views to show indexed data it is only showing English content even if language chosen in NL. But when I cross verified content at Solr everything is indexed.

Can anyone suggest where am I wrong or did I miss any configurations?

Comments

chera.jaswinder created an issue. See original summary.

chera.jaswinder’s picture

StatusFileSize
new15.1 KB
mkalkbrenner’s picture

Can you export your search_api_* configs as well? this will speed up my debugging.

chera.jaswinder’s picture

Added required files.

mkalkbrenner’s picture

It seems that the issue is not related to search_api_solr_multilingual itself.
The rendering of search results search_api_solr seems to be erroneous in search_api.
That needs some more debugging ...

chera.jaswinder’s picture

StatusFileSize
new15.11 KB

Yes even to me it seems to be not related to search_api_solr_multilingual itself.

Yesterday while looking into views settings I was able to figure out, if we unchecked option "Use entity field rendering" appearing during field configuration, which says "If checked, Drupal's built-in field rendering mechanism will be used for rendering this field's values, which requires the entity to be loaded. If unchecked, a type-specific, entity-independent rendering mechanism will be used."

So after making this option unchecked data value started appearing from Solr as required. I would highly recommend for views to make it by default unchecked for now it appears as checked by default which should not happen.

Also uploaded the correct views configuration file.

mkalkbrenner’s picture

I discovered the same behavior yesterday and discussed it in irc with @berdir.

But this is just a workaround because search_api's views plugin doesn't contain a specific renderer for every field type.
Think about a "translated" image field, having a different image for every translation of an article. I guess we want to have the right image displayed in the search result view.

I already continued my debugging and found the reason why "Use entity field rendering" doesn't display the right translation. I hope to have a patch later today ...

chera.jaswinder’s picture

Great, thanks for all your help.

mkalkbrenner’s picture

Title: Views Solr Indexed Issue » Views Integration doesn't work correctly with entity translations.
Project: Search API Multilingual Solr Search » Search API
Version: 8.x-1.0-alpha1 » 8.x-1.x-dev
Component: Miscellaneous » Views integration
Assigned: chera.jaswinder » Unassigned
Priority: Normal » Major
Issue summary: View changes
Status: Active » Needs review
StatusFileSize
new80.49 KB
new100.01 KB
new47.2 KB
new15.47 KB
new14.36 KB
new13.98 KB
new3.56 KB

OK, I got it to work basically. All the trouble is caused by search_api and it's views integration itself.

Status: Needs review » Needs work

The last submitted patch, 9: 2668898_views_integration_entity_translations.patch, failed testing.

The last submitted patch, 9: 2668898_views_integration_entity_translations.patch, failed testing.

mkalkbrenner’s picture

Status: Needs work » Needs review
StatusFileSize
new3.74 KB
new581 bytes
drunken monkey’s picture

Status: Needs review » Needs work
Issue tags: -Search API, -search api solr, -views

Thanks for creating this issue, Jaswinder, and thanks a lot for your detailed analysis, Markus!

However, I fear your current approach is much too hack-y to be committed to the module. It might work reasonably well in some scenarios, but that's of course not enough.
It would be great if you could find a way to fix this that won't require as many assumptions, and will fix this for as many scenarios (i.e., as generically) as possible.

My problems with the patch in detail:

  1. +++ b/search_api.views.inc
    @@ -97,6 +97,20 @@ function search_api_views_data() {
    +        // @todo that approach just works in the case where one index contains
    +        //    one datasource that covers one entity type. But that's what views
    +        //    expects! If we want to cover multiple datasources and multiple
    +        //    entity types we might have to simulate multiple joined tables.
    +        //    Who is an expert for non-SQL views integrations?
    +        $data[$key]['table']['entity type'] = $datasource->getEntityTypeId();
    

    As the @todo already explains, this is much too fragile to be committed.
    If the entity field renderer needs it for translations to work, can we not just override the entity field renderer class we use? It seems insane, if we already know the language, to add a wrong entity type to the Views table just so Views can get it with its default code.

  2. +++ b/src/Plugin/views/query/SearchApiQuery.php
    @@ -480,6 +480,9 @@ class SearchApiQuery extends QueryPluginBase {
    +    /** @var \Drupal\Core\Entity\EntityTypeManagerInterface $entity_type_manager */
    +    $entity_type_manager = \Drupal::service('entity_type.manager');
    

    Seems like this should use dependency injection.

  3. +++ b/src/Plugin/views/query/SearchApiQuery.php
    @@ -524,6 +527,19 @@ class SearchApiQuery extends QueryPluginBase {
    +      list($raw_item_id, $langcode) = Utility::splitPropertyPath($item_id);
    +      list(, $combined_entity_id) = Utility::splitPropertyPath($raw_item_id);
    +      list($entity_type_id, ) = Utility::splitCombinedId($combined_entity_id);
    

    You can't do that, almost none of the item ID's structure is in any way guaranteed. This will work for our own entity datasources, yes, but will break in more or less interesting and/or horrible ways for all others.
    Instead, you can get the entity type by calling the datasource's getEntityTypeId() method (which might return NULL). For the language code, it seems something like I suggested in #2641392: Review our language/translation support would be needed – then you could just call $item->getLanguage().

I might have time to look into this in the next weeks, if you don't have the time or get stuck. Just pretty busy at the moment clearing away my backlog of new and updated issues from two months of D8-only.

However, what would be very helpful in any case, is writing a failing test that demonstrates this problem (and could then be used to verify it is fixed).

mkalkbrenner’s picture

Issue summary: View changes
drunken monkey’s picture

Status: Needs work » Needs review
StatusFileSize
new4.61 KB

The attached solution seems to work fine, too, and is much cleaner.
The only remaining problem (though a major one) is this:

+++ b/src/Plugin/views/query/SearchApiQuery.php
@@ -556,6 +556,9 @@ protected function addResults(array $results, ViewExecutable $view) {
+      // @todo Temporary hack until language is a first-class property of our
+      //   items, like the others above. See #2641392.
+      list(, $values['search_api_language']) = Utility::splitPropertyPath($item_id, TRUE);

So we should probably get going with #2641392: Review our language/translation support and either postpone this issue or commit it with the hack-y workaround and then come back later to fix it.

drunken monkey’s picture

Status: Needs review » Postponed
Related issues: +#2641392: Review our language/translation support
drunken monkey’s picture

Status: Postponed » Needs review
Issue tags: +Needs tests
StatusFileSize
new863 bytes
new3.95 KB

The other issue was committed, so re-rolled the patch here and removed our hack-y workaround from it.

However, we should also definitely add a test for this, so this isn't ready to be committed. (Just setting to NR for the test bot.)

borisson_’s picture

Status: Needs review » Needs work

Back to NW for tests, I have no issues about the rest of the code.

mkalkbrenner’s picture

+++ b/src/Plugin/views/EntityTranslationRenderer.php
@@ -0,0 +1,25 @@
+    return isset($row->search_api_language)
+      ? $row->search_api_language
+      : $this->languageManager->getDefaultLanguage()->getId();

+++ b/src/Plugin/views/field/SearchApiEntity.php
@@ -204,7 +205,13 @@ public function preRender(&$values) {
+            $langcode = $values[$i]->search_api_language;

+++ b/src/Plugin/views/field/SearchApiFieldTrait.php
@@ -370,6 +371,9 @@ public function preRender(&$values) {
+                  if ($value instanceof ContentEntityInterface && $value->hasTranslation($row->search_api_language)) {
+                    $typed_data = $value->getTranslation($row->search_api_language)->getTypedData();
+                  }

It seems that search_api_language always has to be part of the search result. I recently added it for search_api_solr_multilingual. But it's not part of search_api_solr's result. Looks like I have to move the code.
In general, are there any other fields that are expected to be part of the search result. Beside a test for this feature here, a test for the existence for all required search result fields is required.

drunken monkey’s picture

It seems that search_api_language always has to be part of the search result.

No, that's not what's happening here. search_api_language is set on the Views result row in \Drupal\search_api\Plugin\views\query\SearchApiQuery::addResults(). It being present in the returned fields might even break this. (It's a reserved field ID, so I don't think it should ever be present.)

mkalkbrenner’s picture

It being present in the returned fields might even break this. (It's a reserved field ID, so I don't think it should ever be present.)

In search_api_solr_multilingual we require to have search_api_language to be part of the raw search result.
Do we have to remove it before it is turned into a search_api result?

mkalkbrenner’s picture

In search_api_solr_multilingual we require to have search_api_language to be part of the raw search result.
Do we have to remove it before it is turned into a search_api result?

Due to the refactoring that happened throughout the last month, this comment is now obsolete. So let's continue with Thomas' proposed solution.

drunken monkey’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new5.7 KB
new5.7 KB
new9.65 KB

Finally managed to get the tests working, I hope.

The last submitted patch, 23: 2668898-23--views_translated_fields--tests_only.patch, failed testing.

mkalkbrenner’s picture

+++ b/src/Plugin/views/EntityTranslationRenderer.php
@@ -0,0 +1,25 @@
+  public function getLangcode(ResultRow $row) {
+    return isset($row->search_api_language)
+      ? $row->search_api_language
+      : $this->languageManager->getDefaultLanguage()->getId();
+  }

Under which circumstances will there be a result row without a search_api_language property?

Depending on the answer I wonder if getCurrentLanguage(LanguageInterface::TYPE_CONTENT) won't be the better fallback.

drunken monkey’s picture

Under which circumstances will there be a result row without a search_api_language property?

None, I guess, unless someone overrides the query plugin or manually adds result rows. So, this is just very defensive coding on my part, and the fallback shouldn't really matter. We can still change it, though, if you think that one's more appropriate. (If the current just gives you the site-wide default language, that does seem to be the case.)

borisson_’s picture

+++ b/src/Tests/ViewsTest.php
@@ -21,7 +23,7 @@ class ViewsTest extends WebTestBase {
+  public static $modules = array('search_api_test_views', 'views_ui', 'language');

This makes this array go over 80 cols, let's reflow it.

Otherwise this looks good, so RTBC from my pov.

drunken monkey’s picture

StatusFileSize
new1.76 KB
new5.9 KB
new9.98 KB

Needed a re-roll, so I fixed both of these small issues. If the tests pass, I'll commit it.

The last submitted patch, 28: 2668898-28--tests_only.patch, failed testing.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

Great, committed.
Thanks again, everyone!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.