Steps to reproduce:

  1. You enable a fields processor, like "Stemmer". By default, it's enabled for all fulltext fields.
  2. You add a new fulltext field.
  3. The processor won't be enabled by default for that field.

It's not entirely clear what the correct behavior here should be, but I argue that in a lot of cases you'll want the enabled fields processors (that support that field) also enabled for a newly added field.

As a possible compromise, we could also think about only adding the field if the processor previously had all fields enabled – a bit more complex to implement, but less prone to surprises for site admins, I'd say. (Though, arguably, more complex behavior is harder to understand, being a potential UX problem on its own.)

Any opinions here?

(This issue was split off #2859683: Processors don't correctly preprocess keywords per field.)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

drunken monkey created an issue. See original summary.

ShaunDychko’s picture

Thanks for looking into this issue. I was mired by it for a little while today.

How about adding a "select all fields" checkbox on a processor config, with a description below the checkbox saying "by selecting all, any new fields added to the index which are eligible for this processor will be automatically included in this processor." Choosing "select all fields" disables the checkboxes on each field, if that's conveniently possible. This simplifies a tiny bit the logic for checking whether a new field should be added to the processor automatically, since this will be done only if "select all" is chosen. It also gives site admins the ability to explicitly control this behaviour, and things will work the way they expect it to, one way or the other.

drunken monkey’s picture

Ah, yes, that would be an option, too. Thanks for the suggestion!
I went ahead and implemented it, patch attached. Please test/review!

Status: Needs review » Needs work

The last submitted patch, 3: 2867809-3--fields_processor_all_fields_option.patch, failed testing.

TuWebO’s picture

Hello,
Patch in #3 worked for me.

  1. Go to Fields tab: Added field body as fulltext
  2. Go to Processors tab: Checked HTML Filter and the option "Enable on all supported fields"
  3. Go to Fields tab, and added body_1 (again)
  4. Go to Processors tab. The option "Enable on all supported fields" is checked and if I unchecked it, it will show that the processor is running on all fields, including the new one
  5. I runned the "Index now" and checked the index. Both body fields shows in Solr document properly filtered when enabled
"ts_body":"<p>PEtererasdascas <strong>asiodfuh</strong> aoispjf a</p>\n",
        "terms_ts_body":"<p>PEtererasdascas <strong>asiodfuh</strong> aoispjf a</p>\n",
        "its_uid":1,
        "itm_field_user_likes":[39,
          41],
        "its_field_user_likes":39,
        "ts_body_1":"<p>PEtererasdascas <strong>asiodfuh</strong> aoispjf a</p>\n",
        "terms_ts_body_1":"<p>PEtererasdascas <strong>asiodfuh</strong> aoispjf a</p>\n",

Converts to

"ts_body":"PEtererasdascas asiodfuh aoispjf a",
        "terms_ts_body":"PEtererasdascas asiodfuh aoispjf a",
        "its_uid":1,
        "itm_field_user_likes":[39,
          41],
        "its_field_user_likes":39,
        "ts_body_1":"PEtererasdascas asiodfuh aoispjf a",
        "terms_ts_body_1":"PEtererasdascas asiodfuh aoispjf a",

Thanks!

drunken monkey’s picture

OK, good to hear it worked for you, too. Thanks for testing!
Then we just need to fix all those test failures …

Status: Needs review » Needs work

The last submitted patch, 6: 2867809-6--fields_processor_all_fields_option.patch, failed testing.

drunken monkey’s picture

Component: Framework » Plugins
Status: Needs work » Needs review
FileSize
490 bytes
13.72 KB

That should do it.

borisson_’s picture

I have some nits to pick, comments only. Not setting to needs work, because it's only comments.

  1. +++ b/src/Processor/FieldsProcessorPluginBase.php
    @@ -90,11 +96,48 @@ public function setDataTypeHelper(DataTypeHelperInterface $data_type_helper) {
    +    // If the "all supported fields" option is checked, we simply reset the
    +    // "fields" option to contain all supported fields. No need to explicitly
    +    // check for renames, or anything else.
    

    I had to go to the preIndexSave method to have enough context for this comment, but I think we can improve this.

    If the "all supported fields"
     option is checked, we need to reset the fields array and fill it with all fields defined on the index.
  2. +++ b/src/Processor/FieldsProcessorPluginBase.php
    @@ -125,40 +168,107 @@ public function preIndexSave() {
    +    // @todo Add "fields" default here, too, and figure out how to replace
    +    //   current "magic" code dealing with unset option (or whether we even need
    +    //   to).
    

    Should we already open an issue for this (and link to it) or do we fix this in this issue?

  3. +++ b/src/Processor/FieldsProcessorPluginBase.php
    @@ -125,40 +168,107 @@ public function preIndexSave() {
    +    // Unfortunately, Form API doesn't seem to automatically add the default
    +    // "#pre_render" callbacks to an element if we set some of our own. We
    +    // therefore need to explicitly include those, too.
    

    We need to add pre-render callbacks to checkbox elements because form api doesn't add the default callbacks.

  4. +++ b/tests/src/Unit/Plugin/Processor/FieldsProcessorPluginBaseTest.php
    @@ -98,6 +99,47 @@ public function testFieldRenaming() {
    +   * Tests whether the "Enable on all supported fields" option works correctly.
    

    I think we should describe the test a little here.

    Tests whether the "Enable on all supported fields" option works correctly.
    
    We want the all_fields option to make sure that all string and text fields are automatically added to the processor.
    
  5. +++ b/tests/src/Unit/Plugin/Processor/FieldsProcessorPluginBaseTest.php
    @@ -98,6 +99,47 @@ public function testFieldRenaming() {
    +    // Need to create a new index mock, since overwriting getFields() doesn't
    +    // work correctly otherwise.
    

    Create a new mock for the Index, so we can override getFields method.

    sounds better?

drunken monkey’s picture

Thanks a lot once again for your thorough review!
You're right, some of these comments could definitely be improved.

Should we already open an issue for this (and link to it) or do we fix this in this issue?

Added #2881665: Get rid of "magic" isset() checks for field settings in FieldsProcessorPluginBase and referenced it in the @todo comment. It's an existing, separate issue, and not trivial (I think), so I wouldn't tackle that in this one.

We need to add pre-render callbacks to checkbox elements because form api doesn't add the default callbacks.

Are you really sure that's better? It seems more vague and not completely correct (Form API just doesn't add the default in this case, not generally).
Though I admit, the current comment also isn't perfect. Maybe like this?
Form API doesn't add the default "#pre_render" callbacks to an element if we set some of our own. We therefore need to explicitly include the default ones, too.

I think we should describe the test a little here.

Tests whether the "Enable on all supported fields" option works correctly.

We want the all_fields option to make sure that all string and text fields are automatically added to the processor.

I now used this:
We want the option to make sure that all supported fields are automatically added to the processor, without adding unsupported ones or keeping old ones that were removed.
Sounds good?

Create a new mock for the Index, so we can override getFields method.

sounds better?

Counter-offer:
Since it's not possible to override an already specified method on a mock object, we need to create a new mock object for the index in this test.

See the attached patch for my revisions.

borisson_’s picture

The changes you suggested in the comments sound good, but it looks like you forgot the patch. Waiting for that before setting to rtbc.

drunken monkey’s picture

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

Looks great.

drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

Good to hear, thanks for reviewing!
Committed.
Thanks again, everyone!

  • drunken monkey committed b0996c3 on 8.x-1.x
    Issue #2867809 by drunken monkey, borisson_: Added an option to enable a...
drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.