Problem/Motivation

If en entity reference field is using a ViewsSelection handler, the underlying view is not initialised with the request of the page where the field is exposed. This is because the view is built in a subsequent request of the autocomplete widget. This is bad because that view cannot use default arguments from the original request (like path arguments or query parameters). We can imagine a lot of use-cases where the autocomplete list needs to be narrowed by a parameter from the URL.

There's a "test only" patch showing this bug.

Proposed resolution

Ideally we should store the entire original Symfony request when exposing the ER field and pass it later to the view. But this would lead to a very big variance that would had flood the storage, so I went with the next proposal:

  1. In EntityAutocomplete: save the request URI in expirable key/value store, keyed by a hash built from URI.
  2. Pass the hashed key along the URL of autocomplete request to EntityAutocompleteController::handleAutocomplete()
  3. In EntityAutocompleteController::handleAutocomplete() retrieve the original URI from the key/value expirable store.
  4. Pass the original URI to the matcher (EntityAutocompleteMatcher::getMatches()).
  5. The selection handler plugin will be instantiated also with the original URI.
  6. In ViewsSelection::initializeView() (right now only this handler has a legitimate interest on using this value) create a mock of the original request based only on the URI and pass is as request context to the view.

Remaining tasks

None.

User interface changes

None.

API changes

  • The route system.entity_autocomplete takes an additional, optional parameter {original_uri_key}
  • EntityAutocompleteController::handleAutocomplete accepts an additional, optional argument $original_uri_key
  • EntityAutocompleteMatcher::getMatches accepts an additional, optional argument $original_uri

Data model changes

SelectionInterface plugins receive also the original URI as original_uri config.

CommentFileSizeAuthor
#64 2776571-nr-bot.txt144 bytesneeds-review-queue-bot
#62 2776571-59.patch17.84 KBcgrader
#58 interdiff_57-58.txt844 bytesranjith_kumar_k_u
#58 2776571-58.patch17.77 KBranjith_kumar_k_u
#57 2776571-57.patch17.82 KBaludescher
#56 2776571-56.patch17.83 KBhchonov
#53 interdiff_51-53.txt665 bytesraman.b
#53 2776571-53.patch17.77 KBraman.b
#51 interdiff.txt635 bytesraman.b
#51 2776571-51.patch17.77 KBraman.b
#50 reroll_diff_48_50.txt7.53 KBraman.b
#50 2776571-50-reroll.patch16.97 KBraman.b
#48 interdiff_47-48.txt3.65 KBnikitagupta
#48 2776571-48.patch17.1 KBnikitagupta
#47 interdiff_44-47.txt2.04 KBnikitagupta
#47 2776571-47.patch17.43 KBnikitagupta
#44 interdiff-43-44.txt4.12 KBhchonov
#44 2776571-44-8.9.x.patch17.48 KBhchonov
#43 2776571-43-8.9.x.patch13.36 KBhchonov
#37 2776571-37.patch11.92 KBbircher
#36 2776571-36.patch7.65 KBbircher
#25 interdiff.txt3.1 KBclaudiu.cristea
#25 entityautocomplete-2776571-25.patch12.61 KBclaudiu.cristea
#23 interdiff.txt1.57 KBclaudiu.cristea
#23 entityautocomplete-2776571-23.patch12.63 KBclaudiu.cristea
#16 entityautocomplete-2776571-16.patch12.92 KBclaudiu.cristea
#16 interdiff.txt16.76 KBclaudiu.cristea
#9 entityautocomplete-2776571-9-test-only.patch12.75 KBclaudiu.cristea
#9 entityautocomplete-2776571-9.patch21.26 KBclaudiu.cristea
#7 entityautocomplete-2776571-7-8.1.x.patch17.31 KBclaudiu.cristea
#6 interdiff.txt8.43 KBclaudiu.cristea
#6 entityautocomplete-2776571-6.patch21.27 KBclaudiu.cristea
#2 entityautocomplete-2776571-2.patch14.38 KBclaudiu.cristea
er_viewsselector-test-only.patch5.86 KBclaudiu.cristea
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

claudiu.cristea created an issue. See original summary.

claudiu.cristea’s picture

claudiu.cristea’s picture

Issue summary: View changes

The last submitted patch, er_viewsselector-test-only.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 2: entityautocomplete-2776571-2.patch, failed testing.

claudiu.cristea’s picture

Obvious. Now all tests involving entity reference require the keyvalue.expirable service in KernelTestBase container... and it's not there. I added the parameter in KernelTestBase but the table schema is only created on demand. Also I temporary stole #2623568: Config schema of argument_default plugins is incorrect just to see this passing.

claudiu.cristea’s picture

Version: 8.2.x-dev » 8.1.x-dev
FileSize
17.31 KB

And for 8.1.x.

Status: Needs review » Needs work

The last submitted patch, 7: entityautocomplete-2776571-7-8.1.x.patch, failed testing.

claudiu.cristea’s picture

claudiu.cristea’s picture

Status: Needs review » Needs work

The last submitted patch, 9: entityautocomplete-2776571-9-test-only.patch, failed testing.

claudiu.cristea’s picture

Status: Needs work » Needs review

The "test only" patch failed, the other is green. So, back to NR.

amateescu’s picture

Just a question for now, why don't we pass the original URI as part of the existing selection settings entry from k/v?

claudiu.cristea’s picture

@amateescu, that was my first impulse. But on normal site, the actual implementation, will produce a reasonable variance in \Drupal::keyValue('entity_autocomplete'). How many specific autocomplete configurations you might have on a site? Dozens is already too much. But if we put there also the URI you can end up with hundreds or even thousands of items in \Drupal::keyValue('entity_autocomplete') (or even more). This is because we cannot predict on how many URIs with the same autocomplete will be displayed. Just imagine a page with an autocomplete where the URI has a node id as query parameter (e.g. /node/add/page?catalog={node}. For each {node} you'll have a new entry.

But adding the URI to \Drupal::service('keyvalue.expirable')->get('entity_autocomplete') with a 24 hrs expiry time can work as the garbage collection will keep a reasonable number of entries there. Of course the TTL we have to discuss and it's now an arbitrary value.

claudiu.cristea’s picture

@amateescu, alternative solution:

  1. We pack everything (selection settings + entire original symfony Request object) in a single array.
  2. Build a hash on that package
  3. Store the package keyed with the hash in session.
  4. Pass the hash in the URL (as we do now)
  5. Retrieve on the other side, in EntityAutocompleteController::handleAutocomplete(), the whole package
  6. Clear the session entry.

What about that?

EDIT: Ignore the whole comment, this doesn't work.

claudiu.cristea’s picture

This is another approach that doesn't require Expirable Key/Value. The URI is sent with a POST, instead of GET, in autocomplete.js. If it's reasonable we can expand and send also all the selection settings through the same post an drop completely the \Drupal::keyValue('entity_autocomplete') storage. What about that?

dawehner’s picture

What blocks you from sending the URI also as query parameter and don't change it from a GET to a POST request.

claudiu.cristea’s picture

@dawehner, that was my first thought. But now I think I can extend the POST by adding also the $selection_settings to be posted. In this way we can get rid of the 'entity_autocomplete' Key/Value storage. That key/value stuff seem to me a little bit weird because has no mechanism for garbage collection. For example when the settings of a field are changed, a dead entry will still remain in the storage. And conceptually, I don't think that the key/value is for passing data between requests. But GET or POST are exactly for passing data across the requests.

dawehner’s picture

Well the problem is that GET requests are cacheable, while POST requests arent'

claudiu.cristea’s picture

Well the problem is that GET requests are cacheable, while POST requests arent'

@dawehner, that counts!

In this case what is your opinion, should I:

  1. Go with #16 but switch to GET
  2. Go with #16, switch to GET and, additionally, pass also serialised (PHP, Json?) $selection_settings as query param?
Berdir’s picture

We changed this to key value on purpose, don't change that. The reason is that the client/browser should *not* be able to change those settings. We had a security issue there that was fixed before 8.0.0.

amateescu’s picture

I still think we should pass the URI as part of the selection settings, and maybe make it use the expirable one and set the expire time to match the Drupal form expiration time.

claudiu.cristea’s picture

Changed back to use GET request. Now the original URI is passed as query parameter.

@amateescu, why cluttering key/value storage, even the expirable one, when we can pass it along with a very light GET request?

claudiu.cristea’s picture

Credits.

claudiu.cristea’s picture

Small improvements and doc fixes.

amateescu’s picture

@amateescu, why cluttering key/value storage, even the expirable one, when we can pass it along with a very light GET request?

Because we're trading cluttering the key/value storage for cluttering cache-able GET requests :)

dawehner’s picture

Can we somehow make that an opt in behaviour for some form elements? By that we would not have to clutter up key_value nor change the GET url by default.

Another idea I just had, we could store the URI in session, if needed. This is by far still not ideal, but at least something.

claudiu.cristea’s picture

@amateescu,

Because we're trading cluttering the key/value storage for cluttering cache-able GET requests :)

You're right. But personally I like more to store them in the cache for 2 reasons: it has better invalidation mechanism and it benefits for faster storages (Redis, Memcache, etc)

@dawehner,

Can we somehow make that an opt in behaviour for some form elements?

Sure we can. But that means we need to add also an UI option in entity reference autocomplete widget so that we can take the benefit. There is the first place where this would be used.

Another idea I just had, we could store the URI in session, if needed.

I had also this idea in #15 but the values should live in the session till the end. The session data can increase dramatically for a single user.

claudiu.cristea’s picture

How about storing a flag "pass_original_uri" in selection handler settings? Then we do the whole logic only if a specific selection handler will require that. ViewsSelection will require the original uri while DefaultSelection not.

EDIT: At least this narrows the cases to the selection handlers that requires it. But adding also the opt-in could narrow more.

claudiu.cristea’s picture

If we let selection handlers to declare if they want to get the original URI (idea from #29) then we need a base class for all selection handlers in order to assure BC compatibility. I opened #2787873: Add a base class for entity reference selection handlers and fix the structure of their configuration also to deal with #2786841: Entity reference fields should add selection handler config dependencies. That will provide the foundation to add a method such as needsOriginalUri() to all selection handlers. Then we implement that method in the base class returning FALSE and we are covered as BC.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jibran’s picture

Issue tags: +VDC
Berdir’s picture

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

bircher’s picture

re-roll of #25 without any of the improvements in subsequent comments.

I agree that this would be best tackled together with the mentioned #2787873: Add a base class for entity reference selection handlers and fix the structure of their configuration.

bircher’s picture

FileSize
11.92 KB

sorry for forgetting the test class in the patch.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

hchonov’s picture

A simple re-roll for 8.9.x.

hchonov’s picture

I've updated the "Taxonomy tid default argument" to check also the parameters for the origin URI request. This makes it possible to use a contextual filter on a view reference selection on an entity reference field and filter by the current term ID.

The last submitted patch, 43: 2776571-43-8.9.x.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 44: 2776571-44-8.9.x.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

nikitagupta’s picture

worked on deprecation in EntityReferenceAutocompleteTest.

nikitagupta’s picture

Status: Needs work » Needs review
FileSize
17.1 KB
3.65 KB

Status: Needs review » Needs work

The last submitted patch, 48: 2776571-48.patch, failed testing. View results

raman.b’s picture

Version: 8.9.x-dev » 9.2.x-dev
Status: Needs work » Needs review
FileSize
16.97 KB
7.53 KB

Re-rolling for the latest dev branch

raman.b’s picture

Resolving failed test cases from #48

Status: Needs review » Needs work

The last submitted patch, 51: 2776571-51.patch, failed testing. View results

raman.b’s picture

Status: Needs work » Needs review
FileSize
17.77 KB
665 bytes

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

hchonov’s picture

aludescher’s picture

Current 9.3.x reroll.

ranjith_kumar_k_u’s picture

Status: Needs review » Needs work

The last submitted patch, 58: 2776571-58.patch, failed testing. View results

yogeshmpawar’s picture

Status: Needs work » Needs review

Marking Needs Review again because test failures are unrelated.

1) Drupal\Tests\layout_builder\FunctionalJavascript\BlockFormMessagesTest::testValidationMessage
Failed asserting that a NULL is not empty.

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:121
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/core/modules/layout_builder/tests/src/FunctionalJavascript/BlockFormMessagesTest.php:65
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

Getting above test failures in most of the issues.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

cgrader’s picture

Update patch from #58 so that it fits to 9.3.15

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
FileSize
144 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.