Problem/Motivation

Currently, the actual store used to provide the expirable KV storage is supposed to be defined by the factory.keyvalue.expirable parameter, which defaults to default: keyvalue.expirable.database.

However, this fallback is duplicated in KeyValueFactory::get(), which does this:

    if (!isset($this->stores[$collection])) {
      if (isset($this->options[$collection])) {
        $service_id = $this->options[$collection];
      }
      elseif (isset($this->options[static::DEFAULT_SETTING])) {  // 'keyvalue_expirable_default', not 'default'
        $service_id = $this->options[static::DEFAULT_SETTING];
      }
      else {
        $service_id = static::DEFAULT_SERVICE;
      }

As a result, if a specific store is not provided when trying to instantiate an expirable KV store, the first and second checks fail because the default parameter is called default in core.services.yml but used as keyvalue_expirable_default in code, so the logic falls back to the value of KeyValueExpirableFactory::DEFAULT_SERVICE.

AFAICS, we have five use cases in core for this: form_cache (see #2183275: Use cache for $form, kv for $form_state), update.manager, update.manager, update.processor, user.private_tempstore and user.shared_tempstore.

There are two possible fixes: the simplest one is to change the factory.keyvalue.expirable section in default.services.yml, the better one is to fix that logic to use default.

Steps to reproduce

TBD

Proposed resolution

TBD

Remaining tasks

Determine if this is a fix we want to add
If yes determine best patch forward
BC layer + tests
Code review

If no close as works as designed

User interface changes

TBD

API changes

TBD

Data model changes

TBD

Release notes snippet

TBD

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

fgm created an issue. See original summary.

fgm’s picture

In addition, it seems that the SPECIFIC_PREFIX constant defined in both factories is never used as described.

fgm’s picture

Status: Active » Needs review
FileSize
1.3 KB

Let's see what happens when removing this.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jhominal’s picture

While I agree that replacing the DEFAULT_SETTING key from 'keyvalue_expirable_default' to 'default' is the logical long-term fix, what are the backwards compatibility constraints on minor Drupal releases?

I mean, there could be existing installations that override factory.keyvalue.expirable by supplying keyvalue_expirable_default - changing the name of that key would "break" these installations (by forcing them to immediately update their services.yml file).

Is that an acceptable breakage in 8.x?

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

DuaelFr’s picture

As this key is not/wrongly documented in default.services.yml and misused in core.services.yml. I don't think a lot of people are relying on it.
For consistency reasons, I think we should replace the default key by default instead of keyvalue_expirable_default.
If we really want to maintain BC, we can still override the \Drupal\Core\KeyValueStore\KeyValueFactory::get() method to look for the keyvalue_expirable_default key if there is nothing defined under the default key.

pguillard’s picture

I guess this is RTBC :
- SPECIFIC_PREFIX not found elsewhere in core code
- This is what I get on a page load before and after patch

$collection | $service_id BEFORE PATCH | $service_id AFTER PATCH
______________________________________________________________________________
state | keyvalue.database | keyvalue.database
user.private_tempstore.views_bulk_operations_content_page_1 | keyvalue.expirable.database | keyvalue.expirable.database
config.entity.key_store.block | keyvalue.database | keyvalue.database
update_available_releases | keyvalue.expirable.database | keyvalue.expirable.database
update | keyvalue.expirable.database | keyvalue.expirable.database
update_fetch_task | keyvalue.database | keyvalue.database
config.entity.key_store.tour | keyvalue.database | keyvalue.database

DuaelFr’s picture

I'd feel more confident if we added the BC layer even if it's not likely to be used anywhere.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

TR’s picture

Priority: Normal » Major
Issue tags: +Bug Smash Initiative

I think the fix needs to go a little further, because currently KeyValueExpirableFactory can't be used to create an expirable store - it only creates non-expirable KeyValueStore stores.

For example, in core, if you do this:
drupal-check -a modules/update/src/UpdateManager.php
You will see (among other things):

  101    Property Drupal\update\UpdateManager::$keyValueStore                  
         (Drupal\Core\KeyValueStore\KeyValueStoreExpirableInterface) does not  
         accept Drupal\Core\KeyValueStore\KeyValueStoreInterface.              
  103    Property Drupal\update\UpdateManager::$availableReleasesTempStore     
         (Drupal\Core\KeyValueStore\KeyValueStoreExpirableInterface) does not  
         accept Drupal\Core\KeyValueStore\KeyValueStoreInterface.

I encountered this while working on Honeypot. Honeypot has this code:

\Drupal::service('keyvalue.expirable')->get('honeypot_time_restriction')
  ->setWithExpire($identifier, \Drupal::time()->getCurrentTime(), 3600 * 24);

Which looks reasonable. This code tries to:

  1. Get a KeyValueExpirableFactory using \Drupal::service('keyvalue.expirable')
  2. Get a store implementing KeyValueStoreExpirableInterface from that factory
  3. Call setWithExpire() on that KeyValueStoreExpirableInterface

But this doesn't work because KeyValueExpirableFactory::get does NOT return KeyValueStoreExpirableInterface as expected, it returns a non-expirable KeyValueStoreInterface which does not support the setWithExpire() method.

Note that if you use keyvalue.expirable.database instead (implemented by KeyValueDatabaseExpirableFactory), then this all DOES work, because KeyValueDatabaseExpirableFactory::get returns the expected KeyValueStoreExpirableInterface.

Berdir’s picture

This is definitely working, otherwise core wouldn't function. I think drupal-check is just confused about the trickery that's going on with the subclass and the extra interface that overrides the return type.

This issue is about the fact that you can't change the returned store to use a non-database implementation based on how it is documented.

TR’s picture

I linked as a related issue the original issue which removed the use of (but not definition of) SPECIFIC_PREFIX and added factory.keyvalue.expirable as a service parameter.

Yeah you're right that my comments aren't directly related, but they're an indication that this 'trickery' you referred to is fragile and could be broken by this patch.

Specifically, right now the factory.keyvalue.expirable parameter (which is keyvalue.expirable.database) is being ignored, and as long as no-one tries to override that in core.services.yml then things still work as expected because the default value in core.services.yml is the same value as the fallback DEFAULT_SERVICE, which is also keyvalue.expirable.database.

I assume you mean the 'trickery' here is that the keyvalue.expirable.database service will provide an expirable store with an interface KeyValueStoreExpirableInterface, so that calling setWithExpire() will work even though the return type declaration of KeyValueFactory::get is technically the non-expirable KeyValueStoreInterface.

That's all well and good, and will currently always work because any override of that default service parameter is currently ignored.

But if you fix this issue so that the service parameter CAN be overridden, then there's nothing that guarantees that an expirable store will be returned from the overridden factory, so there's no guarantee that the setWithExpire() method will be available. And if it's not available, core breaks.

Static analysis is not really being fooled, it's just pointing out a problem where the return type is not guaranteed to be KeyValueStoreExpirableInterface therefore there is no contract and no promise that setWithExpire() method will be available.

andypost’s picture

Not sure if it's related but keyvalue storage test is what causes segfault for core+php8.1+sqlite in CI https://dispatcher.drupalci.org/job/drupal8_core_regression_tests/45252/...

andypost’s picture

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative, +Needs issue summary update

This issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.

Reading the comments seems there is still work/discussion to be had.

Updated the issue summary to include missing sections of the template.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Sweetchuck’s picture

Issue summary: View changes
Status: Needs work » Needs review
FileSize
3.62 KB

I am not sure about the comments and the deprecation message.

Status: Needs review » Needs work

The last submitted patch, 28: drupal-2769955-1100-28-keyvalue-factory.patch, failed testing. View results