Problem/Motivation

Doing a composer upgrade on drupal 8.8.x on php 7.1+ installs symfony/dom-crawler 4.3.3. At some point between 4.2.11 and 4.3.3, it began to cause test failures of the type: Behat\Mink\Exception\ElementNotFoundException : Element matching xpath "./ancestor::form" not found in around a dozen tests.

Okay, So the root of this comes down to a few form render arraysputting html into the #field_prefix and #field_suffix keys.
Those are rendered inside of tags, so it creates invalid html that the html parser can't parse correctly.

    $element['min_resolution'] = [
      '#type' => 'item',
      '#title' => t('Minimum image resolution'),
      '#element_validate' => [[get_class($this), 'validateResolution']],
      '#weight' => 4.2,
      '#field_prefix' => '<div class="container-inline">',
      '#field_suffix' => '</div>',
      '#description' => t('The minimum allowed image size expressed as WIDTH×HEIGHT (e.g. 640×480). Leave blank for no restriction. If a smaller image is uploaded, it will be rejected.'),
    ];

This ends up outputting

<span class="field_prefix"><div class='container-inline'></span>
...stuff...
<span class="field_suffix"></div></span>

There seems to be a few examples of this throughout core.

Proposed resolution

Adjust the render arrays for Drupal 8.8.
file a follow-up to add a composer conflict for dom-crawler >= 4.3.3 for 8.7

Remaining tasks

- get release manager review
- commit

User interface changes

no

API changes

no

Data model changes

no

Release notes snippet

Comments

mikelutz created an issue. See original summary.

mikelutz’s picture

Here's a composer.lock patch showing the relevant failures.

mikelutz’s picture

Issue summary: View changes

mikelutz’s picture

The code to find the xpath doesn't seem to have changed. What has changed is how the Dom is loaded into the crawler. Starting in 4.3, dom-crawler is using Masterminds/html5 to load the dom. I assume the issue is there somewhere, but it's tough to inspect these dom objects in xdebug, so I haven't yet figured out exactly what has changed.

mikelutz’s picture

Okay, So the root of this comes down to the image field settings form putting html into the #field_prefix and #field_suffix keys.
Those are rendered inside of tags, so it creates invalid html that the html parser can't parse correctly.

    $element['min_resolution'] = [
      '#type' => 'item',
      '#title' => t('Minimum image resolution'),
      '#element_validate' => [[get_class($this), 'validateResolution']],
      '#weight' => 4.2,
      '#field_prefix' => '<div class="container-inline">',
      '#field_suffix' => '</div>',
      '#description' => t('The minimum allowed image size expressed as WIDTH×HEIGHT (e.g. 640×480). Leave blank for no restriction. If a smaller image is uploaded, it will be rejected.'),
    ];

There seems to be a few examples of this throughout core. Curious as to what breaks vs what is fixed if we just take it out.

mikelutz’s picture

Yeah, so that fixed the ones I expected it to. The The CK errors likely correspond to the same thing in editor.admin.inc

On the Views table one at the moment I seem to be unable to get it to pass after reverting to HEAD, so I must be missing something in my development environment right now, maybe due to all the recent composer changes. I may need to clear out my vendor directory, I'm not quite sure yet.

amateescu’s picture

Status: Needs work » Needs review
StatusFileSize
new5.26 KB
new10.14 KB

I think the #field_prefix and #field_suffix parts in ImageItem::fieldSettingsForm() and editor_image_upload_settings_form() are simply a misusage of Form API, and should be completely fine to be fixed in a minor release.

Here's a fix for them and also for StyleTableTest::testAccessibilitySettings().

mikelutz’s picture

I think this is right, tagging for RM review, and +1 to RTBC from me. The issue exists in 8.7 as well, but if the bugfix is determined to be too disruptive for a patch release, I recommend adding a conflict for symfony/dom-crawler >=4.3.0 to 8.7 to avoid using the new parser and to allow tests to pass with updated dependencies.

wim leers’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new27.62 KB
+++ b/core/modules/editor/editor.admin.inc
@@ -93,8 +93,6 @@ function editor_image_upload_settings_form(Editor $editor) {
-    '#field_prefix' => '<div class="container-inline clearfix">',

@@ -110,6 +108,7 @@ function editor_image_upload_settings_form(Editor $editor) {
+    '#prefix' => '<div class="form--inline clearfix">',

Nice.

form--inline was added in #2333719: Abstract Views Exposed Form styling out into a reusable class in 2015. This code was originally added in #1932652: Add image uploading to WYSIWYGs through editor.module in 2013.

I manually tested this and it continues to look the same:

Since all changes are in forms, and then even admin forms, and no form structure changes, only some styling-related markup changes, I am confident this change is acceptable, at least in Drupal 8.8.

wim leers’s picture

Issue tags: +Drupal 9 readiness
wim leers’s picture

Can you elaborate on why you think this needs a change record, @andypost?

andypost’s picture

Build arrays are not a part of API but affects every custom(probably contrib) fields which extends core field type for image

+++ b/core/modules/image/src/Plugin/Field/FieldType/ImageItem.php
@@ -209,8 +209,6 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
-      '#field_prefix' => '<div class="container-inline">',
-      '#field_suffix' => '</div>',

@@ -220,6 +218,7 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
       '#field_suffix' => ' × ',
+      '#prefix' => '<div class="form--inline clearfix">',

@@ -228,6 +227,7 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
       '#field_suffix' => ' ' . t('pixels'),
+      '#suffix' => '</div>',

@@ -236,8 +236,6 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
-      '#field_prefix' => '<div class="container-inline">',
-      '#field_suffix' => '</div>',

@@ -247,6 +245,7 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
+      '#prefix' => '<div class="form--inline clearfix">',

@@ -255,6 +254,7 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
+      '#suffix' => '</div>',

this formally not BC break but good to know... it's hard to catch if no regression testing

gábor hojtsy’s picture

@Wim Leers: In my understanding, despite being just admin forms, there still may be people form altering for field prefix / suffix and now will not override what they used to override. So I think that is where @andypost is coming from. Better be safe than sorry :D

mikelutz’s picture

Issue summary: View changes
mikelutz’s picture

Issue summary: View changes
xjm’s picture

Title: Fix Behat\Mink\Exception\ElementNotFoundException : Element matching xpath "./ancestor::form" not found. in symfony/dom-crawler:4.3+ » Invalid markup in ImageItem (causes test failures with symfony/dom-crawler:4.3+)
Issue tags: -Needs release manager review, -Needs change record +Needs followup

Thanks everyone!

In this case I think we can forego the CR since the markup change is purely removing broken markup from a render array in one element (without any changes required to templates etc.) It's sort of on the edge, but these changes are allowed in minors and typically don't need a CR. According to the frontend BC policy, this type of change is OK in a minor release.

That said, it's a bit too disruptive to backport. Therefore, @mikelutz is going to file a followup issue for 8.7 to add a conflict for dom-crawler 4.3 so that sites don't accidentally update to a library that will cause breakage.

mikelutz’s picture

Issue summary: View changes
xjm’s picture

xjm’s picture

Status: Reviewed & tested by the community » Fixed

This looks straightforward. Thanks also for the manual testing.

The one question I had about this is that the opening tag is now attached to one element as the prefix, and the closing tag is attached to the next element as a suffix, without any structure/nesting and so just relying on the default order of the two render elements. It's not a big deal since these essentially form one conceptual unit but it does cause psychological tension.

While we could change the render array to get around this (e.g. with #type => 'html_tag' or such), @mikelutz, @samuel.mortenson, and I discussed the issue and agreed that would actually be more disruptive. I thought of looking for other places in core where an opening tag and a closing tag are attached to sibling elements but that's difficult to grep for... and after all, markup that might be broken under theoretical alters is less bad than markup that's 100% broken now.

I think put more words into thinking about this than are in the patch. ;) Anyway, this seems alright, so committed to 8.8.x.

Thanks!

  • xjm committed 9d039b4 on 8.8.x
    Issue #3072752 by mikelutz, amateescu, Wim Leers, andypost, xjm, Gábor...
xjm’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: +mwds2019
xjm’s picture

wim leers’s picture

but it does cause psychological tension.

https://twitter.com/wimleers/status/1160569258256011264 😁

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

kristen pol’s picture

Per a Slack discussion with Gábor Hojtsy regarding usage of D9 tags (Drupal 9, Drupal 9 compatibility, Drupal 9 readiness, etc.), "Drupal 9 compatibility" should be used for contributed projects that need updating and "Drupal 9" was the old tag for D8 issues before the D9 branch was ready. Doing tag cleanup here based on that discussion.

jibran’s picture