Hello,

I am making a fork of https://www.drupal.org/project/search_api_attachments named https://www.drupal.org/project/file_extractor.

I initially wanted to re-architecture the Search API Attachments module to go over some limitations. But thinking about this new architecture there was no more dependency on Search API, that's why I decided to make a fork. Everything will be detailed in #3126845: Version 2.0.0.

Search API attachments include a views filter plugin to allow to exclude fulltext fields when searching. If the plugin is configurable to be able to select which fulltext fields to exclude, there is no more link to Search API Attachments or Text Extractor, so I open this issue and I will post a patch that add a such configurable filter.

Comments

Grimreaper created an issue. See original summary.

grimreaper’s picture

Assigned: grimreaper » Unassigned
Status: Active » Needs review
StatusFileSize
new5.15 KB

Here is the patch.

I have not made test yet. If I can get some feedbacks before and some guidelines on how the maintainers want the tests to be done.

Thanks for the review.

drunken monkey’s picture

Thanks a lot for posting this! Please apologize my delay in responding.
Before going into the details of the patch, I have a number of questions/remarks:

  1. Wouldn‘t it be possible to just make this an (optional) part of the SearchApiFulltext filter’s exposed form? After all, there is already a setting there to set the fields to search.
  2. Is there a reason to make this a negative filter, and not just let the user select the fields they do want to search on? Seems unnecessarily complicated.
  3. The current code doesn’t look like it would work when no fulltext fields were explicitly set. Please be aware that getFulltextFields() can also return NULL (meaning “all available fields”).
  4. If they’d remain separate, it seems this filter would be dependent on running after the SearchApiFulltext does, or risk being just overridden by its settings. Is there any way to defend against that, or would this be the user’s responsibility?
grimreaper’s picture

Hello @drunken monkey,

No problem for the delay. :)

  1. Yes, I think. But how will you see that? There will be a loss of flexibility to not be able to place form part how you want, but that is ok. especially regarding the point 4, it will help avoid side effect.
  2. Initially, in Search API Attachments, the feature was to have a checkbox to exclude search in attachments. I guess an exposed checkbox is more user friendly than an exposed list of fields to search among. I am open to suggestion :)
  3. Thanks for pointing this. I will handle that when agreed on the other points.
  4. I was not aware of that. Is there a way to have exposed filters triggered in an order different from their display order?

Thanks for your feedbacks.

drunken monkey’s picture

Initially, in Search API Attachments, the feature was to have a checkbox to exclude search in attachments. I guess an exposed checkbox is more user friendly than an exposed list of fields to search among. I am open to suggestion :)

Yes, agreed, the UX for this is probably not optimal. Most sites would probably need to apply custom stlying/scripts to make this behave the way they want it to. Still, I think it’s an improvement to have this option available right in the filter, to expose the searched fields. That would at least eliminate one part of custom code this use case currently needs.

I was not aware of that. Is there a way to have exposed filters triggered in an order different from their display order?

No, I don’t think so. So yes, making this part of the SearchApiFulltext filter is probably the best way to go.

drunken monkey’s picture

Status: Needs review » Needs work
grimreaper’s picture

Status: Needs work » Needs review
StatusFileSize
new4.48 KB

Hello,

Here is an updated patch from the discussion we had.

And while uploading the patch. I realized that it is still a "negative" filter (regarding point 2 of comment 3).

Waiting for your feedback to rework the patch if needed and implement automated tests.

Status: Needs review » Needs work
grimreaper’s picture

Status: Needs work » Needs review

Back to needs review to get maintainers feedback before fixing tests.

drunken monkey’s picture

Title: Exposed filter to exclude search in fulltext fields » Add option to expose searched fields in Views fulltext filter
Status: Needs review » Needs work
StatusFileSize
new1.86 KB
new3.9 KB

Thanks for your continued work on this! Attached are some small adaptions by me.

Yes, please do change the logic of the form field to make the selection positive.

Also, I saw these problems in the admin UI:

  1. “Expose searched fields” is available even if the filter itself isn’t exposed.
  2. Once you enable the option, the “Exclude from search” select box is also present when configuring the filter in the admin UI. It should only be present when the form is exposed.
grimreaper’s picture

Status: Needs work » Needs review
StatusFileSize
new3.79 KB
new5.67 KB

Thanks a lot for your review.

And nice catchs on Views UI!

I changed the methods where the new form elements are provided.

And changed the logic of selecting searched field and not excluded fields.

If this is ok for you, next step will be writing an automated test. Please say what you want in the test and regarding Search API architecture how you want it to be done.

drunken monkey’s picture

Thanks a lot, looks pretty much flawless now! Also works well, in a quick manual test.
So, thanks, and sorry for the huge delay!

To be honest, I’m not too strict regarding automated tests for smaller Views features, I think I would commit this without an automated test, too. If you want to be extra-awesome, though, I’d of course also be glad about test coverage. If you do want to add some, I’d suggest adding a functional test in the following way:

In \Drupal\Tests\search_api\Functional\ViewsTest::testSearchView(), after the call to regressionTests(), add code to enable this feature for the search_api_test_view view and then test it in some way. (You can find the test content used in this test in \Drupal\Tests\search_api\Functional\ExampleContentTrait::insertExampleContent() – there, just pick a word where it makes a difference which fields are searched and then test that via $this->checkResults().

Status: Needs review » Needs work

The last submitted patch, 12: 3127099-12--expose_searched_fields.patch, failed testing. View results

grimreaper’s picture

Hello @drunken monkey,

No problem for the delay :),

Thanks for the guidance about automated tests.

I will try to add automated tests. I just don't know when :p

grimreaper’s picture

Status: Needs work » Needs review
StatusFileSize
new8.35 KB
new4.21 KB
new189.75 KB

Hello,

Here is an updated patch with automated tests.

I have also updated existing config to have the new config option.

And two notes:

  1. Should we make an update hook to update existing views to have config updated with the default option value?
  2. When testing on the search_api_test_view, I found that the new exposed "search fields" select list is not grouped with the fulltext exposed filter with the option enabled. And if there are multiple fulltext exposed filters it applies to all the fulltext exposed filters because the machine name of the exposed element is not changed automatically to match the filter id. I don't know if the case of multiple fulltext exposed filters is a common case. Personnally I almost only encountered a single fulltext exposed filters with facets. See attached screenshot.

Thanks for the review,

drunken monkey’s picture

StatusFileSize
new7.97 KB
new5.66 KB

Wow, that was quick! Thanks!

Should we make an update hook to update existing views to have config updated with the default option value?

I think with the entry in \Drupal\search_api\Plugin\views\filter\SearchApiFulltext::defineOptions(), missing values should be handled fine, so no need for that. It also wouldn’t have been necessary for the exported test views – but probably no harm in that, either.

When testing on the search_api_test_view, I found that the new exposed "search fields" select list is not grouped with the fulltext exposed filter with the option enabled. And if there are multiple fulltext exposed filters it applies to all the fulltext exposed filters because the machine name of the exposed element is not changed automatically to match the filter id. I don't know if the case of multiple fulltext exposed filters is a common case. Personnally I almost only encountered a single fulltext exposed filters with facets. See attached screenshot.

Ah, OK, that’s of course unfortunate. While you are of course correct that multiple exposed fulltext filters are rather rare, they aren’t completely unheard-of. Probably, the right way to implement this is like an exposed operator, with the option to choose the used GET parameter (and a check that it is unique). Maybe we should just go through \Drupal\views\Plugin\views\filter\FilterPluginBase and copy everything related to exposed operators … (E.g., defaultExposeOptions() also looks useful. Plus, we might want to nest this option under 'expose', for the sake of consistency.)

Actually, let’s do that last one in any case – patch attached. All the others we can also just fix if someone complains, in case you are tired of working on this. I think I’d still just commit this now as-is – it’s still an improvement, even though not perfect.
On the other hand, if you have the nerve to still work on this, I’d of course be happy to hold off and help you polish this further.

In any case, thanks a lot again!

PS: Where the heck did those 114 fails come from for my patch, and why are they gone without any related changes?!? Maybe a Core bug?
Can probably ignore as long as they don’t appear again …

grimreaper’s picture

Assigned: Unassigned » grimreaper

Hello,

Thanks for the reply and the updated patch!

Yes, I would like to do it properly, no hurry to merge, I am learning some Views stuff in the process :p.

Still don't know when I will have time to do that, and this time I have other stuff to do in priority :)

Regards and happy festive season!

grimreaper’s picture

Assigned: grimreaper » Unassigned
StatusFileSize
new7.88 KB
new4.45 KB

And here is a new patch. I want to do this issue before forgetting about it.

I found why the searched field was at the end of all the form elements, because of an hardcoded weight (**facepalm**).

I found that there was only one exposed searched fields element because the form element id was hardcoded, so only one element remaining.

Now that the identifier for the searched fields element can be configured it is ok.

In FilterPluginBase, I found defaultExposeOptions() but also buildGroupOptions(), I have not understood completely the difference between both so only implementing defaultExposeOptions() for the moment, because in defineOptions(), impossible to get the default filter ID.

Hope it will be ok now.

PS: And happy new year!

drunken monkey’s picture

Status: Needs review » Fixed

Thanks, happy new year to you, too! And sorry for the long delay in getting back to you.

Regarding the patch: Excellent work, thanks a lot once again! I just changed a bit of whitespace and removed the unnecessary isset() calls (!empty() already contains isset()), otherwise this looked flawless.
So: Committed.
Thanks a lot again!

grimreaper’s picture

No problem.

Thanks for the merge!

:)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.