Problem/Motivation

drupal_static() should ideally be obsolete, but there are many procedural places in core that use them.

Other properly injected OO code needs to use drupal_static_reset() because of that, which leads to include-hell for tests.

Proposed resolution

Deprecate drupal_static() and drupal_static_reset(). Provide no replacement as using static variables is a bad design. Document that new code should not use drupal_static().

Remaining tasks

None.

User interface changes

None.

API changes

Both drupal_static() and drupal_static_reset() are now deprecated.

CommentFileSizeAuthor
#96 2260187-96.interdiff.txt2.09 KBclaudiu.cristea
#96 2260187-96.patch1.95 KBclaudiu.cristea
#89 2260187_89.patch1.12 KBmile23
#81 interdiff.txt2.03 KBmile23
#81 2260187_81.patch18.66 KBmile23
#77 Screen Shot 2016-02-29 at 16.02.34.png953.77 KBdawehner
#75 interdiff.txt1.88 KBmile23
#75 2260187_75.patch18.68 KBmile23
#72 interdiff.txt2.26 KBmile23
#72 2260187_72.patch18.51 KBmile23
#60 interdiff.txt1.96 KBdawehner
#60 2260187-60.patch18.14 KBdawehner
#56 interdiff.txt3.12 KBdawehner
#56 2260187-56.patch18.22 KBdawehner
#53 replace_drupal_static-2260187-53.patch17.92 KBcilefen
#48 interdiff.txt1.86 KBdawehner
#48 2260187-48.patch18.7 KBdawehner
#46 replace_drupal_static-2260187-46.patch18.73 KBcilefen
#45 replace_drupal_static-2260187-44.patch16.96 KBcilefen
#40 replace_drupal_static-2260187-40.patch18.74 KBjeroent
#40 interdiff-37-40.txt1.45 KBjeroent
#37 replace_drupal_static-2260187-37.patch18.96 KBcilefen
#37 interdiff.txt463 bytescilefen
#31 interdiff.txt834 bytesdawehner
#31 2260187-31.patch18.96 KBdawehner
#30 replace_drupal_static-2260187-30.patch18.11 KBcilefen
#30 interdiff.txt4.47 KBcilefen
#26 replace_drupal_static-2260187-26.patch18.06 KBcilefen
#26 interdiff.txt23.45 KBcilefen
#24 replace_drupal_static-2260187-24.patch13.31 KBcilefen
#20 replace_drupal_static-2260187-20.patch17.84 KBcilefen
#18 replace_drupal_static-2260187-18.patch20.27 KBajits
#12 drupal_static-2260187-12.patch20.7 KBtim.plunkett
#1 static-2260187-1.patch16.54 KBtim.plunkett

Issue fork drupal-2260187

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

tim.plunkett’s picture

Status: Active » Needs review
StatusFileSize
new16.54 KB
dawehner’s picture

Priority: Major » Minor

Seriously .. there is really little value in doing this here, if you ask me, because most instances should be already converted and the other should be converted.

msonnabaum’s picture

@dawehner what about FormBuilder? Is there a clear path to removing drupal_static_reset from it?

If not, I'd argue this is still worthwhile.

tim.plunkett’s picture

Unless we make the parent issue a release blocker, I think this has merit.

dawehner’s picture

@dawehner what about FormBuilder? Is there a clear path to removing drupal_static_reset from it?

If not, I'd argue this is still worthwhile.

What about cats? Aren't they worthwhile to be added to core? Yes I think so. FormBuilder for example just uses a reset on drupal_html_id, which will be dropped anyway.

ParisLiakos’s picture

i agree with dawehner
Moving on #2121713: Move drupal_html_id() and drupal_html_class() to Drupal\Component\Utility is certainly more valuable

tim.plunkett’s picture

That is one out of 39 statics, just the example I picked:

Drupal\Core\Cache\CacheBackendInterface::tagCache
Drupal\Core\Cache\DatabaseBackend::deletedTags
Drupal\Core\Cache\DatabaseBackend::invalidatedTags
_drupal_add_css
_drupal_add_js
_update_create_fetch_task
block_list
drupal_add_js
drupal_get_library
drupal_html_id
example_list
field_form_mode_settings
field_view_mode_settings
file_test_file_scan_callback
filter_formats
image_module_test_image_effect_info_alter
ip_address
locale
locale_get_plural
locale_get_plural:plurals
locale_translation_get_file_history
menu_link_get_preferred
node_access_view_all_nodes
options_allowed_values
shortcut_current_displayed_set
system_list
system_rebuild_module_data
taxonomy_get_tree
taxonomy_get_tree:parents
taxonomy_get_tree:terms
taxonomy_get_treeparent
taxonomy_get_treeterms
taxonomy_term_count_nodes
taxonomy_term_load_children
taxonomy_term_load_parents
taxonomy_term_load_parents_all
taxonomy_vocabulary_get_names
template_preprocess
user_roles

Status: Needs review » Needs work

The last submitted patch, 1: static-2260187-1.patch, failed testing.

ParisLiakos’s picture

Well, sooner all later most of them will go away. Opening issues to kill them one by one and assigning #1577902: [META] Remove all usages of drupal_static() & drupal_static_reset() as parent is probably a better course of action.
Dont get me wrong, i am just saying.. you can of course spend your time as you like :)

tim.plunkett’s picture

Spoke with @alexpott in IRC, he said removing drupal_static did not qualify for a beta or release blocker, but agreed it should be a beta target.

Therefore, I think we should continue here to ensure we're not stuck with that for another cycle.

sun’s picture

I don't really get the point of this issue... bootstrap.inc won't go away anytime soon, so it's not about auto-loading. The change makes DrupalStatic itself injectable, but it's still using a static variable itself, so it's still a global state construct, and thus the dependency still exists as-is, it's just moved from A to B. I'm not able to see the improvement of doing that.

Instead of duct-taping drupal_static(), I'd rather refactor the actual dependency (e.g. drupal_html_id()) into a class/service. Even if that would end up to just be a nonsense/mini/stateful service, it would at least be a step in the right direction.

But of course I agree with @ParisLiakos, this is just some sanity feedback; you can obviously use your time to work on whatever you want ;)

  * @param $name
  *   Name of the static variable to reset. Omit to reset all variables.
...
 function drupal_static_reset($name = NULL) {
-  drupal_static($name, NULL, TRUE);
+  \Drupal::service('event_dispatcher')->dispatch('drupal_static_reset', new StaticResetEvent($name));
 }

+++ b/core/lib/Drupal/Core/DrupalStatic/StaticResetEvent.php
@@ -0,0 +1,43 @@
+  public function __construct($name) {

The reason this fails is that the NULL behavior for resetting all statics is not covered.

To my knowledge, the NULL case is actually the primary use-case in HEAD now. (although it's possible that I've been working on too much low-level stuff recently)

tim.plunkett’s picture

Status: Needs work » Needs review
StatusFileSize
new20.7 KB

Okay, after more though, ditching the over-architected Event-based approach.
All of these statics should ideally be replaced, but let's at least provide a way around this for now.

No interdiff since I kinda rewrote it.

Thanks for the pointer @sun, that was indeed what I missed.

Also I wrote a unit test, and then realized we had a web test, so I just removed that.

sun’s picture

This looks much better and simpler now, but I still don't think this is helpful, so I don't want to RTBC this change.

jibran’s picture

I think we should profile this patch as well.

tim.plunkett’s picture

Assigned: tim.plunkett » Unassigned

If you want to compare static variables and static properties, and profile the internals of PHP, feel free.

This issue is done, it just needs to be committed. Otherwise, the parent issue can be bumped to a release blocker, and this can be won't fixed.

jibran’s picture

It was just a suggestion but according to @tim.plunkett in IRC

i don't think profiling a language level difference is worth holding the issue up

so I withdraw my suggestion. :)

jhedstrom’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll

This needs a reroll. Not sure if it's too late for 8.0?

ajits’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new20.27 KB

Rerolling patch from #12. Had to do a little cleanup in core/testsDrupal/Tests/Core/Form/FormTestBase.php

Basically I removed the following code:

namespace {

  function test_form_id() {
    $form['test'] = array(
      '#type' => 'textfield',
      '#title' => 'Test',
    );
    $form['options'] = array(
      '#type' => 'radios',
      '#options' => array(
        'foo' => 'foo',
        'bar' => 'bar',
      ),
    );
    $form['value'] = array(
      '#type' => 'value',
      '#value' => 'bananas',
    );
    $form['actions'] = array(
      '#type' => 'actions',
    );
    $form['actions']['submit'] = array(
      '#type' => 'submit',
      '#value' => 'Submit',
    );
    return $form;
  }

}

Please let me know, if I did it right. The code did not make sense to me and looked like a test code. If this is wanted, I can add it back. Rest of the conflicts were pretty easy to fix.

Status: Needs review » Needs work

The last submitted patch, 18: replace_drupal_static-2260187-18.patch, failed testing.

cilefen’s picture

Status: Needs work » Needs review
StatusFileSize
new17.84 KB

@AjitS test_form_id() is needed. The changes in FormTestBase from #12 are irrelevant now that there is an Html utility class. Some of the methods in TestFormBuilder should be using it. However, I think those methods are never called. They could possibly be removed in a separate issue.

dawehner’s picture

Let's improve the patch a bit:

  • Move that into a component, called StaticStorage. It can be reused
  • Call it from without UnitTestCase::setup()
  • use it for the two most usecases in core: SafeMarkup, Html
cilefen’s picture

StatusFileSize
new13.31 KB

Reroll.

cilefen’s picture

I think we wanted the drupal_static() docblock mostly emptied. Could whoever rolls this next take care of that and let's do #23.

cilefen’s picture

StatusFileSize
new23.45 KB
new18.06 KB

This is #25 and #23-1. We still need to:

  • Call it from without UnitTestCase::setup()
  • use it for the two most usecases in core: SafeMarkup, Html
cilefen’s picture

  1. +++ b/core/lib/Drupal/Component/Utility/StaticStorage.php
    @@ -0,0 +1,179 @@
    + * All functions requiring a static variable to persist or cache data within
    + * a single page request are encouraged to use this function unless it is
    + * absolutely certain that the static variable will not need to be reset during
    + * the page request. By centralizing static variable storage through this
    + * function, other functions can rely on a consistent API for resetting any
    + * other function's static variables.
    

    "function" should be "class".

  2. +++ b/core/lib/Drupal/Component/Utility/StaticStorage.php
    @@ -0,0 +1,179 @@
    + * Example:
    + * @code
    + * function example_list($field = 'default') {
    + *   $examples = &drupal_static(__FUNCTION__);
    + *   if (!isset($examples)) {
    + *     // If this function is being called for the first time after a reset,
    + *     // query the database and execute any other code needed to retrieve
    + *     // information.
    + *     ...
    + *   }
    + *   if (!isset($examples[$field])) {
    + *     // If this function is being called for the first time for a particular
    + *     // index field, then execute code needed to index the information already
    + *     // available in $examples by the desired field.
    + *     ...
    + *   }
    + *   // Subsequent invocations of this function for a particular index field
    + *   // skip the above two code blocks and quickly return the already indexed
    + *   // information.
    + *   return $examples[$field];
    + * }
    + * function examples_admin_overview() {
    + *   // When building the content for the overview page, make sure to get
    + *   // completely fresh information.
    + *   drupal_static_reset('example_list');
    + *   ...
    + * }
    + * @endcode
    

    The examples are demonstrating the global functions instead of this class.

The StaticStorage docblock needs a good looking-over.

@dawehner After that is done, I think changing usages should be follow-ups. There are a lot.

dawehner’s picture

@dawehner After that is done, I think changing usages should be follow-ups. There are a lot.

I think we should try to ensure that the fast pattern can actually used in an OO version

cilefen’s picture

@dawehner Fair enough - could you please explain comment #23 in more detail?

cilefen’s picture

StatusFileSize
new4.47 KB
new18.11 KB

I fixed up some of the documentation before moving on.

dawehner’s picture

StatusFileSize
new18.96 KB
new834 bytes

Who else thinks this is evil :)

fabianx’s picture

#31: I think this looks pretty genius actually.

Discussed with msonnabaum and dawehner in IRC:

This allows two things:

a) Reset
b) Scoping

Both are important.

While it would be way better to not have any statics and have everything in classes, this is not gonna happen and there is also legacy code.

=> Lets get this in ...

pwolanin’s picture

Do we want to use it in SafeMarkup? it seems like the current class static is good enough?

dawehner’s picture

Do we want to use it in SafeMarkup? it seems like the current class static is good enough?

No, you want to have a central place to reset those statics ... see the comment from fabianx

Status: Needs review » Needs work

The last submitted patch, 31: 2260187-31.patch, failed testing.

The last submitted patch, 31: 2260187-31.patch, failed testing.

cilefen’s picture

Status: Needs work » Needs review
StatusFileSize
new463 bytes
new18.96 KB
pwolanin’s picture

I don't see how this provides any scoping?

Also - do we know if there is a performance difference in calling the static method rather than a function?

tim.plunkett’s picture

  1. +++ b/core/includes/bootstrap.inc
    @@ -983,47 +892,18 @@ function drupal_classloader_register($name, $path) {
    + * @deprecated in Drupal 8.x-dev, will be removed before Drupal 8.0.
    
    @@ -1031,11 +911,12 @@ function &drupal_static($name, $default_value = NULL, $reset = FALSE) {
    + * @deprecated in Drupal 8.x-dev, will be removed before Drupal 8.0.
    

    8.0.x, 8.0.0

  2. +++ b/core/tests/Drupal/Tests/Component/Utility/StaticStorageTest.php
    @@ -0,0 +1,98 @@
    +  /**
    +   * {@inheritdoc}
    +   */
    +  public static function getInfo() {
    +    return array(
    +      'name' => 'Tests \Drupal\Component\Utility\StaticStorage',
    +      'description' => '',
    +      'group' => 'Bootstrap',
    +    );
    +  }
    

    obsolete

jeroent’s picture

StatusFileSize
new1.45 KB
new18.74 KB

Addressed feedback in #39. Patch attached.

cilefen’s picture

Issue tags: +Documentation

A documentation specialist ought to look over the docblock.

Status: Needs review » Needs work

The last submitted patch, 40: replace_drupal_static-2260187-40.patch, failed testing.

anavarre’s picture

Issue tags: +Needs reroll
cilefen’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new16.96 KB

A reroll.

cilefen’s picture

StatusFileSize
new18.73 KB

Oops, ResettableStaticUnitTest crept in.

mikeryan’s picture

Version: 8.0.x-dev » 8.1.x-dev

Well, I'd like to see this, but it seems clear it isn't going to make 8.0.

dawehner’s picture

Version: 8.1.x-dev » 8.0.x-dev
StatusFileSize
new18.7 KB
new1.86 KB

I think this certainly can still land in 8.0.x as it solves problems we have.

fabianx’s picture

I agree it could land in 8.0.x to make our tests easier, etc.

dawehner’s picture

Issue summary: View changes

Adapted the issue summary

joelpittet’s picture

Possibly documentation duplication here: #422352: Improve code comments for static caching API

dawehner’s picture

Version: 8.0.x-dev » 8.1.x-dev

This is 8.1.x material now, this is for sure.

cilefen’s picture

StatusFileSize
new17.92 KB

Trying the reroll against 8.1.x.

mile23’s picture

Title: Replace drupal_static() and drupal_static_reset() with OO versions » Create OOP replacement for drupal_static() and drupal_static_reset()
Status: Needs review » Needs work

Currently 46 calls to drupal_static() in core, according to api.drupal.org.

I like StaticStorage as an OOP replacement for our static pattern. Just one thing:

+++ b/core/lib/Drupal/Component/Utility/StaticStorage.php
@@ -0,0 +1,180 @@
+  public static function reset($name = NULL) {
+    if (!isset($name)) {
+      // Reset all: ($name == NULL). This needs to be done one at a time so that
+      // references returned by earlier invocations of StaticStorage::get() also
+      // get reset.
+      foreach (static::$default as $name => $value) {
+        static::$data[$name] = $value;
+      }
+    }

If you pass in a generated name that doesn't exist, you end up losing the benefit of caching for everything. Could we add resetAll() and be explicit? And then make the $name argument non-optional. If it doesn't exist, do nothing.

benjy’s picture

I wonder if a better name would be GlobalStorage or even GlobalStaticStorage? That way it's clearer that the storage is global to the entire request?

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new18.22 KB
new3.12 KB

If you pass in a generated name that doesn't exist, you end up losing the benefit of caching for everything. Could we add resetAll() and be explicit? And then make the $name argument non-optional. If it doesn't exist, do nothing.

Great idea!

I wonder if a better name would be GlobalStorage or even GlobalStaticStorage? That way it's clearer that the storage is global to the entire request?

Having the static storage in there is IMHO an important bit. I'm happy with GlobalStatic, do you agree?

benjy’s picture

Yes, I like GlobalStatic. +1 from me.

Status: Needs review » Needs work

The last submitted patch, 56: 2260187-56.patch, failed testing.

mile23’s picture

Nice.

Not sure about the test failures... I guess everything uses drupal_static(), eh?

Couple things:

  1. +++ b/core/lib/Drupal/Component/Utility/StaticStorage.php
    @@ -158,23 +158,26 @@ public static function &get($name, $default_value = NULL) {
       public static function reset($name = NULL) {
    

    The only reason to have a default of NULL is so that you can call reset() and have nothing happen. :-)

  2. +++ b/core/lib/Drupal/Component/Utility/StaticStorage.php
    @@ -158,23 +158,26 @@ public static function &get($name, $default_value = NULL) {
    +  public static function resetAll() {
    +    // Reset all: ($name == NULL). This needs to be done one at a time so that
    

    Don't need the $name==null part in the documentation.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new18.14 KB
new1.96 KB

Good points @mile23

mile23’s picture

Title: Create OOP replacement for drupal_static() and drupal_static_reset() » Deprecate drupal_static()/_reset(), replace with StaticStorage class w/ BC
Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs issue summary update +@deprecated, +Needs change record

The patch in #60 addresses concerns in #59, and still applies.

The new static class fills a gap, though I hate statics. :-)

We have BC with the old global methods.

We have tests, and can migrate towards using this new class as needed.

I've added the 'Needs change record' tag, since it's a deprecation.

Updated the issue summary.

Generally RTBC-worthy except for some paperwork.

dawehner’s picture

Thank you @mile23
Filed the change record

dawehner’s picture

Issue tags: -Needs change record
mile23’s picture

Status: Needs work » Reviewed & tested by the community

Looks good. Let's go. :-)

catch’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +needs profiling

This was already discussed on #1577902: [META] Remove all usages of drupal_static() & drupal_static_reset() (edit: hmm partly on that issue, but I think more on another one), but the result of that discussion was just to gradually convert the procedural code using drupal_static() to services. Then any 'static caching' could be on protected properties of those classes then, if it's still needed.

I can't yet see the benefit of making drupal_static() itself 'nicer' - it was never supposed to be nice, it was a horrible hack to get around static caching issues in SimpleTest.

If there's a use case for it other than 'code that's not services yet', then it'd be good to see one. #33-35 mention SafeMarkup, but that's gone now.

Also is there any profiling done here? Without conversions that's hard to do, and I guess if there's a hit we could make the threshold for using the drupal_static_fast pattern lower, but also if it's sufficiently bad it starts to make the static caching in some cases pointless anyway.

dawehner’s picture

This was already discussed on #1577902: [meta] Remove all drupal_static()s + drupal_static() itself (edit: hmm partly on that issue, but I think more on another one), but the result of that discussion was just to gradually convert the procedural code using drupal_static() to services. Then any 'static caching' could be on protected properties of those classes then, if it's still needed.

Throughout the cycle in 8.0.x we have actively added some static caches in places which didn't used to be some, like the filecache component, the extension discovery etc., so the base of the discussion changed over time.

Those bits currently have a normal static, so those caches cannot be cleared which already bites us in some places.

alexpott’s picture

Those bits currently have a normal static, so those caches cannot be cleared which already bites us in some places.

Yep a static in a service is not really any better. Having the ability to clear static caches makes it easier to test stuff.

dawehner’s picture

Yep a static in a service is not really any better. Having the ability to clear static caches makes it easier to test stuff.

OH yeah no quesiton. Its horrible that we have those kind of workarounds. Having though one central place for them, makes it really obvious how bad it actually is.

mile23’s picture

So the problem is: Items use static caching which we can't control, therefore we're complicating the testing process because there's a bunch of static data keeping the code from using our mock. We're also eating up memory that won't be garbage-collected during the request.

If the policy is that we shouldn't migrate away from drupal_static() (to StaticStorage), then we can still use this patch which has a BC layer leaving drupal_static() in place. Code that uses drupal_static() can have its cache cleared by a test using StaticStorage::reset(), without having to jump through hoops to autoload an .inc file.

The deprecation message could be to stop using statics rather than to start using StaticStorage, which could then be documented as a backend for drupal_static().

mile23’s picture

Issue summary: View changes
Status: Needs review » Needs work

Hmm. I guess I should have marked this 'needs work.'

Updated the issue summary based on #65 onward.

If there's documentation on how to prove things with profiling, I'd love to see it. :-)

mile23’s picture

Title: Deprecate drupal_static()/_reset(), replace with StaticStorage class w/ BC » Deprecate drupal_static()/_reset(), add StaticStorage class w/ BC
mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new18.51 KB
new2.26 KB

The patch in #60 needed a reroll, so I did that, and also updated the docblocks to reflect #69.

If this is acceptable we'll still need to work on the change record.

Also, @catch #65:

If there's a use case for it other than 'code that's not services yet', then it'd be good to see one.

Services are really supposed to be stateless, but I think that boat has sailed.

dawehner’s picture

These changes look great! Now its just about time to find someone to review it.

alexpott’s picture

I can't yet see the benefit of making drupal_static() itself 'nicer' - it was never supposed to be nice, it was a horrible hack to get around static caching issues in SimpleTest.

As long as the test runner for things that use WebTestBase or BrowserTestBase are able to access the database and services of the site under test we have this problem. Saying that services shouldn't use this seems premature. The test runner makes far too many API calls at present in the name of speed but not sanity and proper test coverage.

mile23’s picture

StatusFileSize
new18.68 KB
new1.88 KB

Updated the change record to reflect that we shouldn't use statics at all, and that this change is for testing legacy code. https://www.drupal.org/node/2661204

+++ b/core/includes/bootstrap.inc
@@ -902,11 +784,23 @@ function &drupal_static($name, $default_value = NULL, $reset = FALSE) {
+ * @deprecated in Drupal 8.x-dev, will be removed before Drupal 9.0.0. Do not

Updated this to be marked as deprecated in 8.1.x.

Also marked StaticStorage as deprecated with the same message.

dawehner’s picture

+1 for telling people that statics aren't a good idea.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new953.77 KB

This difference would totally disappear when once we would have converted the entries over, but of course, for all other static caches, this would be worse again.

### FINAL REPORT

=== 8.1.x..8.1.x compared (56d454b883f33..56d45cf1ddec3):

ct  : 27,653|27,653|0|0.0%
wt  : 68,071|68,447|376|0.6%
mu  : 13,942,216|13,942,216|0|0.0%
pmu : 14,078,592|14,078,592|0|0.0%

=== 8.1.x..2260187 compared (56d454b883f33..56d45d040c011):

ct  : 27,653|27,665|12|0.0%
wt  : 68,071|67,917|-154|-0.2%
mu  : 13,942,216|14,029,256|87,040|0.6%
pmu : 14,078,592|14,166,128|87,536|0.6%

=== SUM: 2260187-summary..8_1_x-summary compared (56d45d12e6ef3..56d45d16d0e3d):

ct  : 2,768,348|2,766,979|-1,369|-0.0%
wt  : 7,138,631|7,308,247|169,616|2.4%
mu  : 1,398,126,040|1,396,466,312|-1,659,728|-0.1%
pmu : 1,412,306,272|1,410,145,608|-2,160,664|-0.2%

---
ct = function calls, wt = wall time, cpu = cpu time used, mu = memory usage, pmu = peak memory usage

### XHPROF-LIB REPORT

+-------------------+------------+------------+------------+------------+------------+
| namespace         |        min |        max |       mean |     median |       95th |
+-------------------+------------+------------+------------+------------+------------+
| Calls             |            |            |            |            |            |
|                   |            |            |            |            |            |
| 2260187           |     27,665 |     29,513 |     27,683 |     27,665 |     27,665 |
| 8_1_x             |     27,653 |     29,332 |     27,670 |     27,653 |     27,653 |
|                   |            |            |            |            |            |
| Wall time         |            |            |            |            |            |
|                   |            |            |            |            |            |
| 2260187           |     67,915 |     89,821 |     71,386 |     70,703 |     76,392 |
| 8_1_x             |     68,447 |    102,529 |     73,082 |     71,402 |     86,068 |
|                   |            |            |            |            |            |
| Memory usage      |            |            |            |            |            |
|                   |            |            |            |            |            |
| 2260187           | 13,952,528 | 14,367,328 | 13,981,260 | 13,952,608 | 14,318,240 |
| 8_1_x             | 13,942,008 | 14,351,880 | 13,964,663 | 13,942,216 | 14,307,504 |
|                   |            |            |            |            |            |
| Peak memory usage |            |            |            |            |            |
|                   |            |            |            |            |            |
| 2260187           | 14,088,880 | 15,028,464 | 14,123,063 | 14,089,088 | 14,454,672 |
| 8_1_x             | 14,078,288 | 14,525,408 | 14,101,456 | 14,078,592 | 14,443,856 |
|                   |            |            |            |            |            |
+-------------------+------------+------------+------------+------------+------------+

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

alexpott’s picture

+++ b/core/lib/Drupal/Component/Utility/StaticStorage.php
@@ -0,0 +1,191 @@
+ * @deprecated in Drupal 8.1.x, will be removed before Drupal 9.0.0. Do not use
+ *   global static PHP variables. For globally-accessible in-memory caching, use
+ *   services.

I'm not sure what is meant by global static variables. A variable in the global PHP scope is not static ie something like global $user... this seems an odd turn of phrase.

Also isn't something like ExtensionDiscovery::$files shouldn't that be using StaticStorage.

And adding something already deprecated seems premature. I don't think this is a promise we can keep.

dawehner’s picture

Status: Reviewed & tested by the community » Needs work

I agree, we should update the comment

mile23’s picture

Status: Needs work » Needs review
Issue tags: -needs profiling
StatusFileSize
new18.66 KB
new2.03 KB

'Global static PHP variables' is a bit awkward but it's supposed to mean that you can use statics as long as they're not global-scope, like those wrapped by drupal_static() or just global static $foo.

I suppose telling people to use StaticStorage is the best solution right now, since the problem we're trying to solve is being able to reset the static during tests without having to include an .inc file.

So:

Un-deprecated StaticStorage.

Changed the deprecation messages to be 8.2.x.

Hopefully clarified the language in the deprecation messages.

Removed 'needs profiling' tag re: #77

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Thank you @mile23!

catch’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +Needs issue summary update

I'm still missing the use-cases for this, as asked for in #65.

ExtensionDiscovery looks like it has a bug where it's not keying the $files static by hash of $this->searchDirectories and whether tests are included. Also ExtensionDiscovery is the opposite of 'properly injected OOP code', see ExtensionDiscovery::setProfileDirectoriesFromSettings()

I don't know of any other classes in core that are using static properties that are causing problems in tests, and none are listed here apart from ExtensionDiscovery and FileCache.

The only non-test classes that are calling drupal_static_reset() in core are in migrate, shortcut and taxonomy - none of these are good examples either. For example TermStorage::resetCache() calls drupal_static_reset('taxonomy_term_count_nodes') - that function doesn't even exist in 7.x let alone 8.x

BrowserTestBase has three calls for non-existent statics too.

dawehner’s picture

These are examples of static properties ... which all could be problematic in tests or for example when someone would run a PHP daemon.

core/lib/Drupal/Component/FileCache/FileCache.php
core/lib/Drupal/Component/FileCache/FileCacheFactory.php
core/lib/Drupal/Component/Utility/Html.php
core/lib/Drupal/Component/Utility/Timer.php
core/lib/Drupal/Component/Utility/Unicode.php
core/lib/Drupal/Component/Utility/UrlHelper.php
core/lib/Drupal/Component/Utility/Xss.php
core/lib/Drupal/Core/DrupalKernel.php
core/lib/Drupal/Core/Composer/Composer.php
core/lib/Drupal/Core/Datetime/Element/Datetime.php
core/lib/Drupal/Core/Extension/ExtensionDiscovery.php
core/lib/Drupal/Core/Extension/InfoParser.php
core/lib/Drupal/Core/Form/FormState.php
core/lib/Drupal/Core/Mail/MailFormatHelper.php
core/lib/Drupal/Core/Password/PhpassHashedPassword.php
core/lib/Drupal/Core/Render/Renderer.php
core/lib/Drupal/Core/Utility/Error.php
catch’s picture

Thanks for the list. Had a look through all of them.

core/lib/Drupal/Component/FileCache/FileCache.php
core/lib/Drupal/Component/FileCache/FileCacheFactory.php

See below.

core/lib/Drupal/Component/Utility/Html.php

Already has an issue to remove it, StaticStorage doesn't help with that.

    // @todo Remove all that code once we switch over to random IDs only,
    // see https://www.drupal.org/node/1090592.
    if (!isset(static::$seenIdsInit)) {
      static::$seenIdsInit = array();
    }
    if (!isset(static::$seenIds)) {
      static::$seenIds = static::$seenIdsInit;
    }

#1305882: drupal_html_id() considered harmful; remove ajax_html_ids to use GET (not POST) AJAX requests

core/lib/Drupal/Component/Utility/Timer.php

Has its own API to clear the static. Should not be globally reset since that would break timers.

core/lib/Drupal/Component/Utility/Unicode.php

Horrible, but also has a setter for testing. Should not be globally reset since it is set during bootstrap and not on demand, and there's no useful default.

core/lib/Drupal/Component/Utility/UrlHelper.php

Has a setter, should not be globally reset since the value is set in DrupalKernel::preHandle() based on a container parameter, meaning there's no useful default if it gets reset.

core/lib/Drupal/Component/Utility/Xss.php

These are only statics because PHP < 5.6 doesn't allow array class constants. We don't ever want people to be able to change that list globally.

core/lib/Drupal/Core/DrupalKernel.php

Should not be globally reset since it exists very specifically to ensure that it only gets run once per PHP process. Method could possibly be made re-entrant so it doesn't need the static, or possibly already is and the static is cruft.

core/lib/Drupal/Core/Composer/Composer.php

'array constant' again.

core/lib/Drupal/Core/Datetime/Element/Datetime.php

This is a static so that the example always shows the exact same date for a request - i.e. so that two elements don't show 17:35:32 and 17:35:33

Could add a setter. Should probably however get request time from a request object and not be a static at all, or be replaced in JavaScript or something different entirely, or just remove it and accept the second rollover issue, since right now it'd break with render caching anyway.

StaticStorage doesn't help long running processes here, since in those cases you'd still want all date elements rendered from the master request to have a consistent date example, assuming we don't fix it properly above. Also no memory implications since it's a single object.

core/lib/Drupal/Core/Extension/ExtensionDiscovery.php

Discussed above and below.

core/lib/Drupal/Core/Extension/InfoParser.php

Discussed below.

core/lib/Drupal/Core/Form/FormState.php

There are three calls to FormState::hasAnyErrors() in core.

#1863020: View's build fails when an unrelated form on the same page has validation errors and #1305882: drupal_html_id() considered harmful; remove ajax_html_ids to use GET (not POST) AJAX requests would remove two.

The third is in FormBuilder:

      // If there are no errors and the form is not rebuilding, submit the form.
      if (!$form_state->isRebuilding() && !FormState::hasAnyErrors()) {
     

Looks dodgy in the same way that the call in ViewExecutable is - it's checking if there are any errors in any forms, not just this one.

So needs an issue for that last one, and should be @deprecated.

core/lib/Drupal/Core/Mail/MailFormatHelper.php

Makes my eyes hurt. But from a quick look could probably be replaced by passing in $urls as an array and adding to it by reference, since it is reset then checked within the same method and nowhere else.

core/lib/Drupal/Core/Password/PhpassHashedPassword.php

Static was added in 2012 when the class went in, as far as I can see should be a constant like the other constants for that class.

core/lib/Drupal/Core/Render/Renderer.php

   * This must be static as long as some controllers rebuild the container
   * during a request. This causes multiple renderer instances to co-exist
   * simultaneously, render state getting lost, and therefore causing pages to
   * fail to render correctly. As soon as it is guaranteed that during a request
   * the same container is used, it no longer needs to be static.
core/lib/Drupal/Core/Utility/Error.php

Another 'array constant'.

Of those, several should definitely not under any circumstances be changed to either drupal_static() or StaticStorage. Three main categories for these:

1. protected static = array() because we don't require PHP 5.6, after which they could be constants. Just needs a @todo. In the case of Xss tag lists, converting those to StaticStorage would allow contrib/custom code to introduce security issues where they currently can't (if one altered the global list to save passing in a method param, much worse things have been done before).

2. timers, environment status, re-entrance checks, all things which can't be globally reset, and have setters for testing. Ugly, but StaticStorage doesn't help.

3. Bugs/travesties, for which StaticStorage will not make them less of a bug/travesty, some of those have had issues open to refactor for some time. Conversion won't help those issues move faster, might hold them up.

The borderline ones for me are ExtensionDiscovery, InfoParser and FileCache, which are all maintaining static caches - so actually storing things that they explicitly don't want to recalculate during a request.

FileCache already injects a cache backend. So I wondered why it doesn't inject a chained cache backend fronted by MemoryCache and drop the static property, or use MemoryCache directly and do it's own chaining internally. Then I read the issue and found this comment #2395143-86: YAML parsing is very slow, cache it with FileCache by a certain @dawehner. So not sure, but I think that could use its own issue to revisit if we need to. For example we could potentially replace that with a 'statically caching memory backend' that has lots of comments explaining why it does it. Cache backends have a deleteAll() method which would cover the long-running process/memory reclaim issue.

ExtensionDiscovery and InfoParser are both mostly called by https://api.drupal.org/api/drupal/core%21modules%21system%21system.modul...

That entire area is very messy, and if we continue refactoring it, could probably end up with class properties rather than statics, although we'd likely run into the same issue with container rebuilds then in the same way that FileCache does.

Those three, very specific and similar cases in more or less the same subsystems do not really support adding a generic API for static storage, especially not one that is described as a replacement for drupal_static().

mile23’s picture

Given #185 should we rescope this issue to deprecate drupal_static() and pals, without adding StaticStorage? Then make follow-ups to do all the work suggested?

catch’s picture

Linked to one wrong issue in #85, the Html::id() issue that's actually open is #1090592: [meta] Use HTML5 data-drupal-* attributes instead of #ID selectors in Drupal.settings, #85 links to the previous one.

@Mile23 there's a meta for drupal_static() at #1577902: [META] Remove all usages of drupal_static() & drupal_static_reset() , but as discussed in that issue, usually the drupal_static() removal is a by-product of a wider refactoring, so a single meta to track them is hard.

However in #83 I found at least two calls to drupal_static() that are dead code and could be removed, we could open a single issue to find and remove those calls since there may be others. That would then make it easier to see what's left.

Some of the cases in #85 could definitely have individual issues opened, like the array class constants and MailHandler etc.

I think it's worth an issue to talk about ExtensionDiscovery/FileCache/InfoParser more - potentially this issue could stay open for that?

berdir’s picture

+1 to #86.

For example, we already have a static cache bin that's just a memory backend. We could use that for some cases. One of the benefits we have there is that we have cache tag based invalidation. So maybe we can also get rid of some manual cache invalidations through that.

mile23’s picture

Issue tags: +Needs change record updates
StatusFileSize
new1.12 KB

The use case for StaticStorage is admittedly weak: We wanted a replacement for drupal_static() that could be reset without loading .inc files or without booting a kernel in order to load an .inc file. This was for testing purposes.

Generally, use of statics like this, especially hidden behind a non-autoloaded global function, is bad form, so the desire was to provide an alternative.

However, it's clear the StaticStorage part is a no-go, which is why I asked about rescoping for deprecation in #86.

This issue would not be a meta for deprecation, but to apply the following patch which would mark drupal_static() and drupal_static_reset() as @deprecated for removal in Drupal 9.0.0.

That way, we can signal to contrib that they should stop using it, and also have a clear reason for core maintainers to reject its use. This would also make all those other issues implied in #85 actionable.

donquixote’s picture

Related issue I just opened, which is hopefully not too confusing:
#2729725: Reset API / ResettableInterface (brainstorming)

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

claudiu.cristea’s picture

Title: Deprecate drupal_static()/_reset(), add StaticStorage class w/ BC » Deprecate drupal_static() and drupal_static_reset()
Issue summary: View changes
Status: Needs review » Postponed
Issue tags: -Documentation, -Needs issue summary update, -Needs change record updates +Kill includes
StatusFileSize
new1.95 KB
new2.09 KB

The question now is how to remove the usage from procedural code. If we cannot, we can't unpostpone this one.

claudiu.cristea’s picture

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

claudiu.cristea’s picture

Status: Postponed » Needs review

I think we still can deprecate the usage for contrib and custom code callers. Then we gradually deprecate core usages by progressing with #1577902: [META] Remove all usages of drupal_static() & drupal_static_reset() . Some may argue that the solution is ugly but I find it effective.

The MR is not a continuation of previous PRs, even we can already add the `@deprecation` tags to both functions.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

daffie’s picture

Should we not remove all usage of the two functions before we deprecate them?

catch’s picture

Title: Deprecate drupal_static() and drupal_static_reset() » [PP-1] Deprecate drupal_static() and drupal_static_reset()
daffie’s picture

Status: Needs review » Postponed

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.