If NestedArray::setValue() is called with an array structure that does not support adding the value at the given position, the page crashes with a

Warning: Illegal string offset 'id' in Drupal\Component\Utility\NestedArray::setValue()

and

Fatal error: Cannot create references to/from string offsets nor overloaded objects in /[..]/core/lib/Drupal/Component/Utility/NestedArray.php

The method does have a $force parameter which can prevent such situations.
But in some cases it is called without this parameter.

E.g. in FormBuilder::handleInputElement() we find this code:

          NestedArray::setValue($form_state->getUserInput(), $element['#parents'], NULL);

For me this blew up when I changed a form structure in PHP, and then resubmitted the form.
So it could happen when a user resubmits a form at the moment of a deployment.
It's a bit rare maybe but still..

A way to artificially reproduce this with custom code:

$array = ['x' => 'X'];
$parents = ['x', 'y'];
$value = 5;
$force = false;
\Drupal\Component\Utility\NestedArray::setValue($array, $parents, $value, $force);

My main point here is that the NestedArray::setValue() method is not programmed in a robust way.
It has no well-defined behavior in a case like above.

My first solution would be to let it throw an exception instead.
E.g. like this:


  public static function setValue(array &$array, array $parents, $value, $force = FALSE) {
    $ref = &$array;
    foreach ($parents as $i => $parent) {
      // PHP auto-creates container arrays and NULL entries without error if $ref
      // is NULL, but throws an error if $ref is set, but not an array.
      if (NULL !== $ref && !is_array($ref)) {
        if ($force) {
          $ref = [];
        }
        else {
          $conflict_parents = [];
          foreach ($parents as $j => $theparent) {
            if ($i === $j) {
              break;
            }
            $conflict_parents[] = $theparent;
          }
          $parents_str = '$[' . implode('][', $parents) . ']';
          $conflict_parents_str = '$[' . implode('][', $conflict_parents) . ']';
          $type_at_ref = gettype($ref);
          throw new \RuntimeException("Tried to set nested value at $parents_str, but found a $type_at_ref at $conflict_parents_str.");
        }
      }
      $ref = &$ref[$parent];
    }
    $ref = $value;
  }

This may be a bit overkill, but most of the added logic is only executed if the exception is going to be thrown.
The logic takes into account that $parents might not be cleanly indexed.

In the above artificial example, the output would be:

RuntimeException: Tried to set nested value at $[x][y], but found a string at $[x]. in Drupal\Component\Utility\NestedArray::setValue()

This is already better, isn't it?

Comments

donquixote created an issue. See original summary.

donquixote’s picture

From the doc comment:

   * @param bool $force
   *   (optional) If TRUE, the value is forced into the structure even if it
   *   requires the deletion of an already existing non-array parent value. If
   *   FALSE, PHP throws an error if trying to add into a value that is not an
   *   array. Defaults to FALSE.

So yeah this is kinda "works as advertised". But I think this is a flawed design, and simply letting PHP crash for a predictable error is not the correct way of doing things.

dawehner’s picture

This is a tough issue, isn't it? At least for me PHP is a language where being strict is hard, as its not designed for that, on purpose.

What about providing something like "setValueForced" / "setForcedValue" or so, which enables you to use that parameter easier / in a nicer way?

donquixote’s picture

This is a tough issue, isn't it? At least for me PHP is a language where being strict is hard, as its not designed for that, on purpose.

I think modern PHP is moving more and more to opt-in strictness. I read the term "gradually typed language".
And modern frameworks more and more choose to use it in the strict way where possible.

But I don't think this is a language problem, in this case.
E.g. the following example could be in any language:

function div($a, $b, $return_false_if_b_is_zero) {
  if ($b !== 0) return $a / $b;
  if ($return_false_if_b_is_zero) return FALSE;
  return $a / 0;  // -> Ouch.
}

Instead we would do this:

function div($a, $b, $return_false_if_b_is_zero) {
  if ($b !== 0) return $a / $b;
  if ($return_false_if_b_is_zero) return FALSE;
  throw new InvalidArgumentException('$b may not be zero.');
}

So, instead of just letting the language crash natively, we throw an exception. Ideally with some contextual information, which can be logged or reported somewhere.

What about providing something like "setValueForced" / "setForcedValue" or so, which enables you to use that parameter easier / in a nicer way?

You mean dedicated functions? It would look nicer, perhaps.

But the fact remains that FormBuilder::handleInputElement() calls this with $force = FALSE, which can cause WSOD depending on user input.

So the questions I see:
- Why does FormBuilder call it with $force = FALSE?
- Would $force = TRUE result in undesired behavior?
- Should we throw an exception instead of letting the error just happen?
- If we do so, where would we catch this exception?
- Would changing any of this be a BC break in FormBuilder?

I can think of two reasons why this error would occur:
1. Some form components on server side are misbehaving -> programmer error.
2. The form receives POST data that does not have the expected structure. This could be due to misbehaving client-side components, or due to a code change on server side between form build and form submission, or because a malicious user is sending manipulated POST data.

Both of these cases are "exceptional", so we should let the developer know that something is wrong.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

donquixote’s picture

donquixote’s picture

Issue tags: +php7, +PHP 7.0 (duplicate)

This behavior changed from PHP 5.6 to PHP 7.
https://3v4l.org/dDDi0 - check "eol versions".

For non-empty strings this was always broken.

Empty strings were treated the same as NULL in PHP 5.

liam morland’s picture

Issue tags: -php7

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

neclimdul’s picture

Priority: Normal » Major
Issue tags: +PHP 8.0

I'm trying to come up with a simplified test case but using this process from #3160406: Exposed forms with checkboxes don't work correctly if the checkbox value is in the URL I did trigger this.

  1. Create a node listing view that is a page.
  2. Add a filter like "status". Make the filter exposed and grouped so it has some depth to the for structure.
  3. Visit the page and append the filter identifier to the url as a query argument. Like /test-page?status

The form builder is populating from user input(which did honestly surprise me coming from the query) and the input structure doesn't match the form structure because of the nesting triggering a fatal error.

Because this is a fully fatal error, repeatable out of the box, and doesn't require any weird user interactions like POST manipulation I've bumped this to major. Still sort of an edge case but easily repeatable crash.

Additionally in case its helpful, my case the trace ended up looking like this:

The website encountered an unexpected error. Please try again later.
Error: Cannot create references to/from string offsets in Drupal\Component\Utility\NestedArray::setValue() (line 155 of core/lib/Drupal/Component/Utility/NestedArray.php).

Drupal\Component\Utility\NestedArray::setValue(Array, Array, NULL) (Line: 1249)
Drupal\Core\Form\FormBuilder->handleInputElement('views_exposed_form', Array, Object) (Line: 1000)
Drupal\Core\Form\FormBuilder->doBuildForm('views_exposed_form', Array, Object) (Line: 1070)
Drupal\Core\Form\FormBuilder->doBuildForm('views_exposed_form', Array, Object) (Line: 1070)
Drupal\Core\Form\FormBuilder->doBuildForm('views_exposed_form', Array, Object) (Line: 574)
Drupal\Core\Form\FormBuilder->processForm('views_exposed_form', Array, Object) (Line: 320)
Drupal\Core\Form\FormBuilder->buildForm('\Drupal\views\Form\ViewsExposedForm', Object) (Line: 135)
Drupal\views\Plugin\views\exposed_form\ExposedFormPluginBase->renderExposedForm() (Line: 1238)
Drupal\views\ViewExecutable->build() (Line: 395)
Drupal\views\Plugin\views\display\PathPluginBase->execute() (Line: 196)
Drupal\views\Plugin\views\display\Page->execute() (Line: 1630)
Drupal\views\ViewExecutable->executeDisplay('page_1', Array) (Line: 81)
Drupal\views\Element\View::preRenderViewElement(Array)
call_user_func_array(Array, Array) (Line: 101)
Drupal\Core\Render\Renderer->doTrustedCallback(Array, Array, 'Render #pre_render callbacks must be methods of a class that implements \Drupal\Core\Security\TrustedCallbackInterface or be an anonymous function. The callback was %s. See https://www.drupal.org/node/2966725', 'exception', 'Drupal\Core\Render\Element\RenderCallbackInterface') (Line: 772)
Drupal\Core\Render\Renderer->doCallback('#pre_render', Array, Array) (Line: 363)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 201)
Drupal\Core\Render\Renderer->render(Array, ) (Line: 241)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 564)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 242)
Drupal\Core\Render\MainContent\HtmlRenderer->prepare(Array, Object, Object) (Line: 132)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object)
call_user_func(Array, Object, 'kernel.view', Object) (Line: 142)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object, 'kernel.view') (Line: 163)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 80)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 708)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)
neclimdul’s picture

Version: 9.3.x-dev » 9.4.x-dev
Status: Active » Needs review
StatusFileSize
new2.06 KB
new3.95 KB

Turns out this is pretty easy to recreate inside out test suite. Just adding a query to one of the exposed filters tests for views triggers it right away.

Here's a patch with some tests that _mostly_ fixes the problem based on the solution in the d7 patch but throwing an exception instead of triggering an error. I figure the exception will allow forms to fail a bit more gracefully though I haven't started digging into the validation logic to confirm that. The case where the value is set will still WSOD a site so we should still have a failure on this patch.

neclimdul’s picture

A bit of a self review, open question. This preserves the BC with old version of PHP which treated the empty string as null. But had this been fixed earlier, that would have likely been removed with 9 when the requirements where raised. Additionally because it fails, this has never been supported with D9 so... is it really BC?

So either this patch needs to properly deprecate the behavior, remove it, or document why its needed. Leaning toward just removing it.

neclimdul’s picture

Version: 9.4.x-dev » 9.3.x-dev
Status: Needs review » Active
StatusFileSize
new2.29 KB
new4.18 KB

Woops...

neclimdul’s picture

Version: 9.3.x-dev » 9.4.x-dev
Status: Active » Needs review

neclimdul credited Lendude.

neclimdul’s picture

Component: base system » forms system
StatusFileSize
new6.84 KB

Found a good test for the Form component of this problem on #2932604: Fatal error when providing a string value to a form field that expects an array value by Lendude. Added it here and merged the issues since they're the same problem.

The last submitted patch, 18: 2901127-18-FAIL.patch, failed testing. View results

The last submitted patch, 18: 2901127-18.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 21: 2901127-20.patch, failed testing. View results

neclimdul’s picture

Goodness, this bug/hole in NestedArray trickled out to a lot of things. The config tests are even asserting and working around the PHP 8 in the test... weird.

neclimdul’s picture

StatusFileSize
new2.8 KB
new9.29 KB

Pulling in the fix for config adjusted to use the exception.

karavkars’s picture

I was getting error when passing filter value in the url for multiselect exposed filter.
I could solve this with small alteration in the url parameter,
posting this if you find helpful.
Drupal Version: 9.2.x
Problem:

URL- http://localhost.com/vews-page?field_tag_target_id=261
Error - Error: Cannot create references to/from string offsets in Drupal\Component\Utility\NestedArray::setValue() (line 155 of core\lib\Drupal\Component\Utility\NestedArray.php).

Solution:
Changed in url - http://localhost.com/vews-page?field_tag_target_id[261]=261

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

bwoods’s picture

Thank you for this patch. My use case was a Views Exposed Filter was not being set due to a malformed link. I kept seeing this error in watchdog, but didn't see anything wrong with the filter itself. Since the Exposed Filter existed in the query string, but the value was blank, NestedArray blew up with an empty string. I was temporarily able to get it working by adding an empty check as an OR in the if $ref check, but this patch looks a lot more comprehensive.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

ahmad khader’s picture

Reroll to 10.2.x

ahmad khader’s picture

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.