Problem/Motivation
CurrentPathStack caches one path per request in an \SplObjectStorage keyed by the Request object. SplObjectStorage holds a strong reference to every key, and nothing ever removes one, so every Request that passes through getPath() or setPath() stays in memory, together with its parameter and header bags, until the process ends.
In a web request that is one or two objects and does not matter. It matters in any long-running process that matches synthetic requests against the router, because RouteProvider::getRouteCollectionForRequest() calls $this->currentPath->setPath($path, $request) for every request it matches.
Entity Usage does exactly that: UrlToEntity::findEntityIdByUrl() creates a Request::create() for every link it resolves, and its EntityRouting listener passes it to router.no_access_checks. Re-tracking usage for a site's content from drush therefore keeps one Request per link for the whole run.
Measured on a Drupal 11.3 site with about 357,000 entities, re-tracking entity usage from drush:
30 chunks of 200 taxonomy terms (1,602 links resolved)
as is: +21.2 MB, 1,603 requests held by path.current
$paths replaced by a WeakMap: +9.7 MB
(the rest is memcache 2.8's static statistics, a separate issue)
Full run: the drush process reached 1.25 GB.
The remaining growth is memcache 2.8.0 recording every call in a static array, reported as memcache #3503083 and fixed in memcache #2996054.
Root cause: \SplObjectStorage keeps its keys alive. The cache only needs to know the path of a request while something else still uses that request.
Steps to reproduce
- In
drush php:eval, create and route-match many synthetic requests:
$router = \Drupal::service('router.no_access_checks'); for ($i = 0; $i < 5000; $i++) { try { $router->matchRequest(\Symfony\Component\HttpFoundation\Request::create('/node/' . $i)); } catch (\Exception $e) {} } $paths = new \ReflectionProperty(\Drupal\Core\Path\CurrentPathStack::class, 'paths'); print count($paths->getValue(\Drupal::service('path.current'))); - It prints 5000 or more (plus the requests drush created itself), and
memory_get_usage()grows by the size of those requests. None of them is released while the process runs. With 2,000 requests on Drupal 11.3.17: 2,000 held, +13.6 MB.
Proposed resolution
Store the paths in a \WeakMap. It supports the same isset($this->paths[$request]) and $this->paths[$request] = $path access, so no other line changes, and it drops an entry as soon as nothing else references its request. A request that is still in use, such as the current one on the request stack, keeps its path exactly as before.
/**
* Static cache of paths.
*
- * @var \SplObjectStorage
+ * Weak, so that a request nothing else references any more is released
+ * together with its path. Code that matches many synthetic requests, such
+ * as a long-running process resolving URLs, would otherwise keep every one.
+ *
+ * @var \WeakMap<\Symfony\Component\HttpFoundation\Request, string>
*/
protected $paths;
...
public function __construct(RequestStack $request_stack) {
$this->requestStack = $request_stack;
- $this->paths = new \SplObjectStorage();
+ $this->paths = new \WeakMap();
}
With a unit test that fails before the change and passes after it:
public function testRequestNothingElseHoldsIsReleased(): void {
$stack = new CurrentPathStack(new RequestStack());
$request = Request::create('/node/1');
$stack->setPath('/node/1', $request);
$this->assertSame('/node/1', $stack->getPath($request));
$reference = \WeakReference::create($request);
unset($request);
$this->assertNull($reference->get());
}
Remaining tasks
- Confirm nothing serializes
CurrentPathStack:\WeakMapcannot be serialized,\SplObjectStoragecould. Services are not serialized directly (DependencySerializationTraitreplaces them with their service id), and no core code accesses$pathsoutside this class.
User interface changes
None.
API changes
None. $paths is protected; a subclass that relied on it being an \SplObjectStorage (for example calling count(), attach() or detach() on it) would need to adapt. count() and array access still work on a \WeakMap.
Data model changes
None.
Issue fork drupal-3625547
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
siegristComment #3
siegristComment #5
siegristComment #6
amitgoyal commentedReviewed the patch and confirmed it addresses the problem described: CurrentPathStack::$paths is switched from \SplObjectStorage to \WeakMap so a Request nothing else references is released along with its cached path, instead of being held for the lifetime of the process.
Checked out the MR branch locally and verified:
- The new CurrentPathStackTest passes, including testRequestNothingElseReferencesIsReleased, which fails against the old SplObjectStorage-based code and passes with the WeakMap change.
- All existing Core/Path unit tests (31 tests) still pass, no regressions.
- PHPCS with Drupal,DrupalPractice standards is clean on both changed files.
- No Change Record needed: $paths is a protected property, and per the issue's own API changes note, only a subclass relying on it being an SplObjectStorage (calling count()/attach()/detach() directly) would need to adapt; normal usage via getPath()/setPath() is unaffected.
The pipeline initially showed 3 failed jobs. On inspection, two are the PHPUnit Unit (Core/Component) jobs on the "next PHP major" (8.6) lane, which are configured with allow_failure (exit_codes: 100) in .gitlab-ci.yml and failed on unrelated Extension/InfoParser/Update tests, not on anything in this change. The third was PHPUnit Functional Javascript 2/3, which errored in Drupal\Tests\field_ui\FunctionalJavascript\ManageFieldsTest::testAddField with WebDriver\Exception\MoveTargetOutOfBounds - a Selenium drag-and-drop timing issue unrelated to this patch. Retried just that job and it passed on the second run, confirming it was a flaky test rather than a regression. Pipeline is now green (passing with the two known/allowed 8.6 warnings).
Moving to RTBC.
Comment #7
alexpottWhat a nice fix. As maintainer of entity_usage as well I can see how this is going to be really useful.
Committed and pushed 396e5b8bcad to main and 561f06170ae to 12.0.x and 4450cfd8b37 to 11.x and 997c5f893ef to 11.4.x. Thanks!
Comment #13
alexpott@siegrist FYI the memcache stats issue has just been fixed too... see #3621859: Memcache statistics are enabled by default... hopefully that module will get a release soon.
Comment #14
siegristWow, that was quick! Thank you! Yes, I saw the memcached issue, hoping for a quick release there too.
Comment #15
ayalon commentedAwesome work!