Problem/Motivation

If using Views Ajax History (update, same problem when just 'Ajax' enabled), a GET request is made instead of POST. Views Reference passes all configuration as part of the ajax request. This can result in a URI too long if there are a lot of settings. The amount of settings can be greatly reduced if only selected options are passed.

Steps to reproduce

Add an option to the Views Reference field like filtering by topic and have a higher amount of topics. Each topic is passed even if not selected.

Proposed resolution

Compress the views reference configuration

(For anyone coming across this still having 414 issues, there is now https://www.drupal.org/project/viewsreference_extras which reloads the views reference configuration from the entity instead of compressing the configuration as is)

Remaining tasks

Provide MR

User interface changes

None

API changes

None

Data model changes

None

Issue fork viewsreference-3396530

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

scott_euser created an issue. See original summary.

scott_euser opened merge request !36

scott_euser’s picture

scott_euser commented
Assigned: scott_euser » Unassigned
Status: Active » Needs review
scott_euser’s picture

scott_euser commented
Title: Views reference results in 414 if many options set and ajax history used » Views reference results in 414 if many options set and ajax used
Issue summary: View changes
seanb’s picture

seanb
Primary language Dutch
Location Netherlands
commented

I just ran into something similar. The request options we are passing in the URL is causing issue with the total URL length on some hosting providers. My initial thought is to implement something like #3348789: Compress ajax_page_state. We can compress the data in the URL and then decompress it when we need it to optimize this.

scott_euser’s picture

scott_euser commented

Sounds like a good idea. I am curious how compressed it becomes.

As an alternative, could the field storage be referenced instead of all the configuration passed in each ajax request, and the view reference configuration loaded from the field storage at each ajax request. Compared to the Views query itself, I imagine that would be pretty neglible impact on performance. Not sure if there are cases where things go into the view reference that different per user or differ based on current form state that could cause problems with this direction.

seanb’s picture

seanb
Primary language Dutch
Location Netherlands
commented

We definitely don't want an object to be passed in the URL. We could theoretically also just pass the parent entity type, ID/revision ID and langcode? We could then load the data in the AJAX request when we need it? This could potentially break any alters etc though.

In any case, the compression is quite significant for the AJAX page state.

heikkiy’s picture

heikkiy commented

We seem to be hitting a similar issue which might be related to this module. Our views Ajax calls broke after upgrading to Drupal 10.

We are now getting a 403 error from views/ajax request and it seems like the GET request is quite huge.
Object { message: "\nAn AJAX HTTP error occurred.\nHTTP Result Code: 403\nDebugging information follows.\nPath: /en/views/ajax?event_language=All&accessibility_options=All&target_group=1709\nStatusText: Forbidden\nResponseText: \n403 Forbidden\nForbidden\nYou don't have permissions to access /en/views/ajax?event_language=All&accessibility_options=All&target_group=1709&_wrapper_format=drupal_ajax&view_name=dynamic_listings&view_display_id=person&view_args=all%2Fall%2Fall%2F2018%2Fteaser%2F12_full_pager&view_path=%2Fnode%2F130768&view_base_path=&view_dom_id=a3b06dc96cec4f9caf7b345b59fd85ea9d043b4926127cda2f8c14c1b7558555&pager_element=0&event_language=All&accessibility_options=All&target_group=1709&page=1&_drupal_ajax=1&ajax_page_state%5Btheme%5D=jyu&ajax_page_state%5Btheme_token%5D=&ajax_page_state%5Blibraries%5D=admin_toolbar%2Ftoolbar.tree%2Cadmin_toolbar%2Ftoolbar.tree.hoverintent%2Cadmin_toolbar_search%2Fsearch%2Cadmin_toolbar_tools%2Ftoolbar.icon%2Cbetter_exposed_filters%2Fgeneral%2Cbig_pipe%2Fbig_pipe%2Ccookie_content_blocker%2Fcookie-content-blocker%2Ccore%2Fdrupal.active-link%2Cenvironment_indicator%2Fdrupal.environment_indicator%2Cextlink%2Fdrupal.extlink%2Cgin%2Fgin_accent%2Cgin%2Fgin_base%2Cgin%2Fgin_init%2Cgin%2Fgin_toolbar%2Cjyu%2Fglobal%2Cjyu%2Fhyphenation%2Cjyu%2Fjyu_cookie_content_blocker%2Cjyu%2Flanguage-dropdown%2Cjyu%2Fmobile-menu%2Cjyu%2Fmodal%2Cjyu%2Fsprite%2Cjyu%2Ftopbar%2Copen_readspeaker%2Fbasic%2Cparagraphs%2Fdrupal.paragraphs.unpublished%2Csearch_api_autocomplete%2Fsearch_api_autocomplete%2Cshortcut%2Fdrupal.shortcut%2Csocial_media_links%2Ffontawesome.component%2Csocial_media_links%2Fsocial_media_links.theme%2Csystem%2Fbase%2Ctoolbar%2Ftoolbar%2Ctoolbar%2Ftoolbar.escapeAdmin%2Cuser%2Fdrupal.user.icons%2Cviews%2Fviews.ajax%2Cviews%2Fviews.module on this server\n", name: "AjaxError", stack: "@https://preview.jyu.fi/sites/default/files/js/js_0nNCZ_HC3WMSFrI5-A5-uO0ptYPrGVQETYvV5i_lMxU.js?scope=footer&delta=2&language=en&theme=jyu&include=eJx1U32-4iAMvBCWM-xJ-AUa2ygQlg-f3n4Dtr6tz_dHJZmYZmZIHfOV0DiOFWM11rO7YtZuwKcNPm2wKuwIvAk4ExhP8Vr0T2iqKwZU5VEqBm2hoLo8ml48W_AjXB9pxQiVOI5cHuM-ElGOM-o5twR-Alfphqc-ROG99nMvbalaKOoKtoxAGHleRiiPGUz2BJyTMSN1fD7jsxJ5RpMy3gi_BpCxJI5Fxh7gwfJemwgfkgdoM8Lscgv2NaUyewv5lVOk-snEs7wPvrBwwMlxkJGdXLempEwVVYIMS4a0ll3xNzK1mJr1VFacVadY9PidAs_N49NCEsI5St_lb8P8mM6cw8HclHkRteUJiocW7N7sGeZLOd5FjNyiwwMYpB8WwTxk1nCB-1A-gv-JDcBiFUoG74kLzuZMXtKiF4yYZU84YTTd0ZIQ-krK9ZEbngS25PEUMLaRV07d5E86TSNVELJbDSQy0Cp3fz1W1L_g24hZOBzf2OIsBDv2VrDgrlauTG3XrbdzGgzfQNWKiNkM6_FEsk1FYbxR5hj68lOcyUHl1_8-FhXM4XvHzFPOpuqt1s_yotUHqrJyrq7VfcSeHztfPTXLJ_J7aVr5hpnGt_vDBiwOEv7pzcrSYhIl1HvwDzYvwfc:34:2411\n@https://preview.jyu.fi/sites/default/files/js/js_0nNCZ_HC3WMSFrI5-A5-uO0ptYPrGVQETYvV5i_lMxU.js?scope=footer&delta=2&language=en&theme=jyu&include=eJx1U32-4iAMvBCWM-xJ-AUa2ygQlg-f3n4Dtr6tz_dHJZmYZmZIHfOV0DiOFWM11rO7YtZuwKcNPm2wKuwIvAk4ExhP8Vr0T2iqKwZU5VEqBm2hoLo8ml48W_AjXB9pxQiVOI5cHuM-ElGOM-o5twR-Alfphqc-ROG99nMvbalaKOoKtoxAGHleRiiPGUz2BJyTMSN1fD7jsxJ5RpMy3gi_BpCxJI5Fxh7gwfJemwgfkgdoM8Lscgv2NaUyewv5lVOk-snEs7wPvrBwwMlxkJGdXLempEwVVYIMS4a0ll3xNzK1mJr1VFacVadY9PidAs_N49NCEsI5St_lb8P8mM6cw8HclHkRteUJiocW7N7sGeZLOd5FjNyiwwMYpB8WwTxk1nCB-1A-gv-JDcBiFUoG74kLzuZMXtKiF4yYZU84YTTd0ZIQ-krK9ZEbngS25PEUMLaRV07d5E86TSNVELJbDSQy0Cp3fz1W1L_g24hZOBzf2OIsBDv2VrDgrlauTG3XrbdzGgzfQNWKiNkM6_FEsk1FYbxR5hj68lOcyUHl1_8-FhXM4XvHzFPOpuqt1s_yotUHqrJyrq7VfcSeHztfPTXLJ_J7aVr5hpnGt_vDBiwOEv7pzcrSYhIl1HvwDzYvwfc:34:19298\n" }

It seems to also include unrelated information. I can see the error in console after each view Ajax change like pagination or exposed filter.

Tested the patch from this issue in our staging environment and still get the error. We don't seem to get a similar error in local environment. I suspect it might be the staging environment proxy which fails us. I did try the merge request from #3348789: Compress ajax_page_state but that seems to be for 10.2 and the merge request does not apply against 10.1.6.

I do recognize that our error is actually 403 Forbidden but it seems really similar to this issue. Also no other view seems to be misbehaving expect the ones using Embedded view field type.

heikkiy’s picture

heikkiy commented

This definitely seems to be related to #3348789: Compress ajax_page_state. I did try using the patch for 10.x from that core issue but I have mixed results. Pagination works but exposed forms seem to just reload the page and not actually use Ajax. But since this seems to be a core issue partly, it's probably best to continue the discussion there.

scott_euser’s picture

scott_euser commented

We definitely still have this issue even with #3348789: Compress ajax_page_state. We are mostly using #3364798: Views reference is adding many query string variables to the pager URLs though to solve this rather than this patch though. This patch is opinionated, whereas the other approach gives the control to the developer, so perhaps more likely to get merged? Needs maintainer opinion.

seanb’s picture

seanb
Primary language Dutch
Location Netherlands
commented

The change in #3348789: Compress ajax_page_state only affects the ajax_page_state parameter. We should apply the same solution to the viewsreference parameter we are adding to the URL.

We can compress it in viewsreference_views_pre_render using
UrlHelper::compressQueryParameter(json_encode($view->element['#viewsreference']))

And the uncompress it in viewsreference_views_pre_view using
json_decode(UrlHelper::uncompressQueryParameter($view->getRequest()->query->all('viewsreference')), TRUE)

scott_euser’s picture

scott_euser commented
Assigned: Unassigned » scott_euser

Makes sense, thank! Will give this a shot. Existing functional JS test coverage should help here.

scott_euser opened merge request !59

scott_euser’s picture

scott_euser commented

Okay proof of concept here: this sorts it for me with unlimited length URL, BUT it does not work with compression alone. With compression we get a lot of the way there, but the URL limit according to https://stackoverflow.com/a/417184 should be 2000. If we leave space for exposed filters and other bits in the URL, with a lot of possible configuration options, it still gets hit.

I suggest we default to compression, and fallback to state if that fails. Would like to get other maintainer buy-in on this before I:

  1. Add test coverage properly for this (ie, I just grabbed my tests from Views Ajax History just for quick POC)
  2. Make the threshold of switching to state configurable or override-able somehow?
scott_euser’s picture

scott_euser commented
Status: Needs review » Needs work

Hmmm I think actually state becomes problematic if someone shares a particular URL. Seems better to implement this suggestion from #7:

We could theoretically also just pass the parent entity type, ID/revision ID and langcode. We could then load the data in the AJAX request when we need it? This could potentially break any alters etc though.

scott_euser’s picture

scott_euser commented

Okay that's implemented. Opted for also compressing the reload of the options to make it less tempting for someone to want to modify the options in the URL + checking entity access on the reloaded entity.

@seanB if you are happy with this approach I'll get some test coverage properly in for it (right now its not covered).

scott_euser changed the visibility of the branch 3396530-views-reference-results to hidden.

scott_euser’s picture

scott_euser commented
Assigned: scott_euser » Unassigned
Status: Needs work » Needs review

Okay added test coverage as well to help get this in faster.

scott_euser opened merge request !62

scott_euser’s picture

scott_euser commented

Okay after discussion with @seanB in Slack:

  1. Removing the reloading the settings directly from the entity when the compression query string is still too long and will shift that to a new separate module.
  2. Converted to a service so that new separate module can then handle larger compression via reloading from the entity.

For those coming across this issue after it is marked as Fixed here, if you still have this issue, please see this module homepage for a link to the extra module.

seanb’s picture

seanb
Primary language Dutch
Location Netherlands
commented
Status: Needs review » Fixed

Merged! Thanks.

scott_euser’s picture

scott_euser commented
Issue summary: View changes

Thanks! For anyone coming across this still having 414 issues, there is now https://www.drupal.org/project/viewsreference_extras which reloads the views reference configuration from the entity instead of compressing the configuration as is.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

daniel kulbe’s picture

daniel kulbe
he/him
Primary language German
Location Berlin
commented

The used methods from \Drupal\Component\Utility\UrlHelper are only available in D11, afaik - so this is a breaking change for version support!

scott_euser’s picture

scott_euser commented

It does seem to be in Drupal 10 docs too https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Component%21Util...

scott_euser’s picture

scott_euser commented

Yep 10.1 so well before any currently supposed versions https://www.drupal.org/project/drupal/issues/3303067#comment-14888060

daniel kulbe’s picture

daniel kulbe
he/him
Primary language German
Location Berlin
commented

You're right ... prior my statement I took a look at 10.0 - it has been added in 10.1.x:

https://www.drupal.org/project/drupal/issues/3303067

Using viewsreference 8.x-2.0-beta9 ... which still has ^9.3 support.

Also the current current release uses ^10 || ^11 ... which includes 10.0 as well.