Problem/Motivation

The Search API module has a special cache tag that gets invalidated when when search index is updated. The invalidation is detected by Purge, but the tag does not end up in the purge queue. This tag can be used to invalidate views build on search index results.

To reproduce:
- Set up Search API with a server and one index. We use it in combination with Seach API Solr, but this is probably not relevant.
- Create a view based on a Search API index. Set the view caching to Search API tag based caching.
- Edit a node that is listed by the view, or create a new node that gets listed.
- Check that the Search API tag (format: search_api_list:{search index name}) is added/increased in the cachetags table.
- See that this tag is not added in the purge_queue table. The node_list tag is added.

Methods called when search_api_list tag is invalidated:

  • \Drupal\search_api\Entity\Index::indexSpecificItems (this is where the search_api_list tag is invalidated)
  • \Drupal\purge_queuer_coretags\CacheTagsQueuer::invalidateTags
  • \Drupal\purge\Plugin\Purge\Queue\QueueService::add (last method called in Purge related to this tag)

The methods called when other tags (e.g. node_list) are invalidated.

  • \Drupal\purge\Plugin\Purge\Queue\QueueService::add
  • \Drupal\purge\Plugin\Purge\Queue\QueueService::commit
  • \Drupal\purge\Plugin\Purge\Queue\QueueService::commitAdding
  • \Drupal\purge\Plugin\Purge\Queue\QueueService::commitReleasing

The reason that QueueService::commit is not called after Index::indexSpecificItems, is because both are executed during the service destruction phase. In \Drupal\Core\EventSubscriber\KernelDestructionSubscriber::onKernelTerminate the following services are due for desctruction in the given order. Note that search_api.post_request_indexing is called after purge.queue.
The service destruction itself has no priority per service. I tried increasing the purge module's weight to 10, but that did solve the destruction order, but still the tag is not added to the purge_queue table.
Also tried to change the pruge services destruct priority ({ name: needs_destruction, priority: -100 }) but without success. This seems to have no effect on the destruction order.

Initial destruction order (purge module weight = 0):

‌array (
  0 => 'state',
  1 => 'path.alias_whitelist',
  2 => 'menu.active_trail',
  3 => 'theme.registry',
  4 => 'library.discovery.collector',
  5 => 'block_content.uuid_lookup',
  6 => 'string_translator.locale.lookup',
  7 => 'purge.logger',
  8 => 'purge.queue',
  9 => 'purge.queue.stats',
  10 => 'purge.purgers.tracker.runtime_measurement',
  11 => 'search_api.post_request_indexing',
  12 => 'drupal.proxy_original_service.router.builder',
)

Proposed resolution

t.b.d.

Issue fork purge-3008776

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

Sutharsan created an issue. See original summary.

sutharsan’s picture

Issue summary: View changes

Added further investigation results.

sutharsan’s picture

Issue summary: View changes

Added more test results.

As a work-around I will now add the node_list tag to the Search API views

grayle’s picture

Status: Active » Needs review
StatusFileSize
new1.6 KB

Priority won't work because the class that manages setting up the destruction calls doesn't look for it. Priority doesn't seem to be a globally supported key inside a service tags array, it's handled individually for each manager in charge of that specific tag. So it's up to core to fix that part. Once it's in, simply give purge.queue a low priority (or all purge services that need_destruction).

related issue: https://www.drupal.org/project/drupal/issues/3038040

Status: Needs review » Needs work

The last submitted patch, 4: 3008776.add-priority-to-needs_destruct.patch, failed testing. View results

eddie_c’s picture

@Sutharsan, would you mind explaining the work around you mention? How did you add the node_list tag to the Search API views?

sutharsan’s picture

@eddie_c, not sure, it's been a long time. Perhaps https://www.drupal.org/project/views_custom_cache_tag

grayle’s picture

Reroll

dealancer’s picture

Patch #8 does not work for me it even breaks Drupal cache invalidation on the Drupal side for some tags.

swentel’s picture

#3038040: Support priority for needs_destruction service tag has a patch that should work so if someone can test it, then it can go to RTBC :)

alorenc’s picture

Reroll to 8.x-3.2 version

alorenc’s picture

Added missing priority for #11

alorenc’s picture

timwood’s picture

Status: Needs work » Needs review

The patch from #12 works for us!

nvaken’s picture

Same here, #12 works for us in combination of the patch in https://www.drupal.org/project/drupal/issues/3038040

jorgik made their first commit to this issue’s fork.

claudiu.cristea’s picture

The core issue has tests and is ready for review #3038040: Support priority for needs_destruction service tag