Updated: Comment #N

Problem/Motivation

This was spun off from #2212321-2: Add highlighted search result snippet as "field" for Views

Function node_views_data() provides integration with the core NodeSearch for Views of Nodes. Namely, there is a search keywords filter and argument that you can add to your Views, and if you do so, you also get the ability to sort by the search keyword score.

However, this filter is not language-aware. When nodes are indexed in NodeSearch, each translation is added separately to the search index. But when the keywords filter is used, the search results are not correlated correctly to the translations where the matches are found.

For example, if you make a site with English, French, and Spanish enabled, and make 1 node about food in Paris, and translate it into in each language (so each node has the word "food" translated in the title, and the word "Paris", same in all languages, as well).

If you have a View of nodes with no filtering, you will see 3 results. In the Views results, if you are using Teasers as your row output, the teasers are rendered in the 3 different languages; if you use Fields, you'll see the node title in each language.

If you filter by "title contains Paris", you will still see 3 results (same as with no filtering).

If you filter by "title contains comida", you will see only the Spanish result.

The same thing will happen if you use Core Search (outside views) and search for "Paris" or "comida".

So that is all good.

However, if you use the Search filter inside Views, you'd expect it to work the same as "title contains" filtering and as Core Search page (no Views):
- If you type "Paris" in as your keyword, you should get the 3 translations as your results.
- If you type "comida" you should get just the Spanish translation.

But that is not happening currently. If you search for "comida" you are still getting all 3 translations.

(As a note, a similar problem happens with the Body Contains filter -- see #2218065: Need to join fields to the entity field data tables, not entity tables, or filtering increases number of results).

Proposed resolution

Join the search tables to node_field_data on the langcode column as well as nid. That should fix the problem.

Remaining tasks

Make a patch.

User interface changes

Search filters in Views will work like Title Contains filters and like Search does outside of Views.

API changes

None.

Comments

jhodgdon’s picture

Status: Active » Needs review
StatusFileSize
new3.29 KB

Here's a failing test that illustrates this problem. If you run it interactively, you'll see at the last step that the Views output page has both translations shown, even though only one matches the keyword filter.

Status: Needs review » Needs work

The last submitted patch, 2: 2118057-test-only.patch, failed testing.

jhodgdon’s picture

Issue summary: View changes
Status: Needs work » Needs review
Related issues:
StatusFileSize
new4.86 KB
new758 bytes

OK, here's a patch along with the test. I edited the test assert messages so they are less cryptic and more descriptive, but otherwise the test is the same as the one that failed (as expected) in #2. Interdiff attached.

I think this works... For some reason I am unable to run tests in my current D8 install (something about not having an active config directory? I probably need to reinstall or something). But in my testing of the bug itself, the patch fixes the problem as stated in the issue summary. Let's see what the bot says.

jhodgdon’s picture

StatusFileSize
new2.48 KB

Whoops. That wasn't an interdiff. Try this one.

jhodgdon’s picture

4: 2218057-4.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, 4: 2218057-4.patch, failed testing.

jhodgdon’s picture

Hm. That looks like a legit test failure. Will need some work in that case.

jhodgdon’s picture

OK, I have no idea what is going on here.

I can reproduce the test failure on my local machine. When the test goes to the "test filter" page, which is supposed to find nodes with "pizza" in them, the page is empty when you look at the debug output of the test -- no links to either node.

However, when I try to do the steps the test is doing on my local site (outside of the test environment), after importing the test view configuration into my local site, the "filter page" works just fine (I can see the "pizza" node link). I thought it might be a permissions problem, but on my local site I can even see this result if I log out, so I don't think that is the problem.

I also added some lines to the test to debug what is happening, where I search for the word "pizza" to make sure it got into the search index, and that works fine (insert these just after the cron run line):

    $this->drupalPostForm('search/node', array('keys' => 'sandwich'), t('Search'));
    $this->assertLink('sandwich', 0, 'English node translation is searchable');
    $this->drupalPostForm('search/node', array('keys' => 'pizza'), t('Search'));
    $this->assertLink('pizza', 0, 'Spanish node translation is searchable');

The debug output for these lines shows the right search results.

jhodgdon’s picture

The really odd thing is that this test was passing on March 4. Something changed between then and now and now it is failing. :(

Also, just as a note: without the code patch here, as of March 4, the test I wrote for this bug was finding both the "pizza" and the "sandwich" nodes on the filter page (if you look at the test results output in comment #2, you'll see it fails on the line where it's verifying that the "sandwich" link should not be on the page).

When I unpatch my local site and look at the test view page output on my local site (outside of testing), I am still seeing that same behavior: it is supposed to be filtering to nodes containing "pizza", but both the "pizza" ("Spanish") translation and the "sandwich" ("English") translations are being shown (which is the bug that is being reported in this issue). But inside the testing environment, if I run the test it just shows the English "sandwich" node, and not the Spanish "pizza" node.

So, something is definitely different in the test environment and my actual local site, and apparently this change in the testing environment also changed between March 4 and now (and it didn't also change in my local site, which I just reinstalled today from scratch). I wonder what is going on?

jhodgdon’s picture

OK, I realize the last two comments are fairly confusing. Here's the situation:

a) What the test does:
- Adds Spanish as a language
- Adds a content type, and sets it up for translation
- Adds a node with title "sandwich" in English and "translates" it into Spanish, using title "pizza" in Spanish.
- Runs cron to index for searching.
- Visits a test view page from the view "test_search" (config file: core/modules/views/tests/modules/views_test_config/test_views/views.view.test_search.yml). This view page has a filter for "search keywords: pizza".
- Attempts to verify that the "pizza" Spanish node translation is visible, and that the "sandwich" English node translation is not.
- I also added a few extra lines to the test to verify that using the core search/node page, I can successfully search for both "pizza" and "sandwich" and I see the correct node translations in both cases.

b) On my local site, I reproduced similar steps... The only difference is that I installed using the Standard install profile, and used the built-in "Page" content type instead of a newly-created content type. And I went to Config Import to import the test view. I also tested viewing the test view page as both Anonymous and User 1.

c) As of March 4th, without the code patch, the behavior of both the tests and local site was that on the test view page, both the "sandwich" and "pizza" node variants were visible. And with the code patch, the behavior of both the tests and local site was that on the test view page, just the "pizza" node was visible.

d) As of today, my local test site behaves the same as described in (c).

e) As of today, when running the test: without the code patch, the test view page output shows just the "sandwich" translation. And with the code patch, the test view page output shows no results (just the page title). However, the test lines show that search/node is still working properly -- it is only within Views that things have changed for the worse.

Is that clear? So what I'm trying to figure out is:

1. Why, as of today, does the test behavior not match what I see on my local site?

2. Why is the test behavior different between March 4 and today -- it appears that I cannot see the Spanish node at all when I go to the test view page, although there is no language filter in the view.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new4.89 KB
new1.09 KB

The queyr seems a bit wrong:

 SELECT node_field_data.title AS node_field_data_title, node.nid AS nid, node_field_data.created AS node_field_data_created, SUM(node_search_index.score * search_total.count) AS score, MIN(node.nid) AS nid_1, node_search_index.langcode as node_search_index_langcode, node_field_data.langcode as node_field_data_langcode FROM  simpletest897819node node INNER JOIN simpletest897819node_field_data node_field_data ON node.nid = node_field_data.nid LEFT JOIN simpletest897819search_index node_search_index ON node_field_data.nid = node_search_index.sid AND (node_search_index.type = 'node_search'    ) LEFT JOIN simpletest897819search_total search_total ON node_search_index.word = search_total.word WHERE (( (node_field_data.status = 1) AND( (node_search_index.type = "node_search") AND( (node_search_index.word = "pizza") )))) GROUP BY node_search_index.sid, score, node_field_data_title, nid, node_field_data_created, node_field_data_langcode HAVING (( (COUNT(*) >= 1) )) ORDER BY node_field_data_created DESC;
+-----------------------+-----+-------------------------+--------------------+-------+----------------------------+--------------------------+
| node_field_data_title | nid | node_field_data_created | score              | nid_1 | node_search_index_langcode | node_field_data_langcode |
+-----------------------+-----+-------------------------+--------------------+-------+----------------------------+--------------------------+
| sandwich              |   1 |              1398528895 | 0.4261508360505104 |     1 | es                         | und                      |
+-----------------------+-----+-------------------------+--------------------+-------+----------------------------+--------------------------+

... this is missing the condition added in the patch.

... a couple of steps in between ... the node was not published, maybe this is actually a different bug, let's ask the entity people.

jhodgdon’s picture

OK, that solves the mystery! Probably prior to March, created translations were published by default, and now they are not. And on my local test site, the translation was definitely published. So that would explain why (a) the results changed over time and (b) they are different between the test and my site.

So now we have a viable patch and a passing test. I think this patch is good to go. dawehner do you still think it has problems?

jhodgdon’s picture

12: 2218057-12.patch queued for re-testing.

dawehner’s picture

StatusFileSize
new4.89 KB
new577 bytes

Just added a new line at the end ... given that I just changed a single line it seems fine to RTBC this myself.

jhodgdon’s picture

Status: Needs review » Reviewed & tested by the community

I think that you meant to set this to RTBC? If so, I'm +1. If not, set back. Thanks!

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed fe26eb6 and pushed to 8.x. Thanks!

  • Commit fe26eb6 on 8.x by alexpott:
    Issue #2218057 by jhodgdon, dawehner: Search Views integration - filter...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.