Problem/Motivation

In Drupal 9.5.11, the Media Library AJAX pagination used the POST method. In Drupal 10.1.x, it switched to using the GET method.
With multiple pages in the Media Library, the URLs for these pages become longer and longer. Consequently, these lengthy URLs are being blocked by AWS WAF when following the core ruleset https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-g...
---
SizeRestrictions_QUERYSTRING
Inspects for URI query strings that are over 2,048 bytes.
Rule action: Block
Label: awswaf:managed:aws:core-rule-set:SizeRestrictions_QueryString
--
The specific rule causing the block is the "SizeRestrictions_QUERYSTRING" rule, which checks for URI query strings over 2,048 bytes and blocks them accordingly.

This issue is related to
- #3348789: Compress ajax_page_state
- #3380486: Extremely long Views AJAX query string triggers 403 in AWS

Despite applying a patch to compress libraries from #3348789: Compress ajax_page_state, the URL length remains too large. The URL length was reduced from approximately 2600 to 2200 characters, and it still triggers blocks by AWS WAF.

This issue may also be connected to the discussion on allowing AJAX to use GET requests
- #956186: Allow AJAX to use GET requests

Proposed resolution

Wouldn't it be better to handle view ajax pagination with the POST method to avoid such URL length issues altogether?

CommentFileSizeAuthor
#13 3421255-13.patch4.03 KBadebruin

Issue fork drupal-3421255

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

vesnag created an issue. See original summary.

vesnag’s picture

vesnag commented
Issue summary: View changes
cilefen’s picture

cilefen commented

Can you verify on 10.2 please?

vesnag’s picture

vesnag commented

With the clean install of 10.2.3, the Media Library URL may extend beyond 2000 characters when navigating through pages.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Category: Bug report » Support request

This isn't a bug in Drupal, just a false positive in the WAF. I've also run into this as I have sites behind AWS WAF and have had to add exceptions for this rule and the media library URLs (as an aside, editing content in Drupal often triggers false positives in other WAF rules such as XSS, this is not a bug either).

There are lots of different resources to refer to here, but the most up to date standard from https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of... says

It is RECOMMENDED that all senders and recipients support, at a minimum, URIs with lengths of 8000 octets in protocol elements.

so Drupal is not technically doing anything wrong here. WAFs are never one-size-fits-all and always need tuning for complex applications.

vesnag’s picture

vesnag commented

As already reported here https://www.drupal.org/project/drupal/issues/3403077, GET requests also break Varnish.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented

Well yes, but that's a separate issue; the AWS WAF does not use Varnish.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

johnv’s picture

johnv
Primary language Dutch
commented
Title: AWS WAF Blocks Media Library AJAX Pagination URLs » Pager with ajax on Media Library makes longer pagination URL, getting blocked by AWS WAF
Issue summary: View changes
Related issues: +#3348789: Compress ajax_page_state, +#3380486: Extremely long Views AJAX query string triggers 403 in AWS, +#956186: Allow AJAX to use GET requests

Better title for non-techies, better links tot issues, adding related links

adebruin made their first commit to this issue’s fork.

adebruin opened merge request !14116

adebruin’s picture

adebruin commented

It is possible to reproduce this problem by using the media library, especially the pager, in a node edit form when using at least 70+ libraries.

We made an attempt at further reducing the length of the ajax_page_state libraries. For our attempt to work, the theme has to be loaded so we had to move the priority of the ajax_page_state middleware to run only after the pre_handle middleware has already ran.

We made changes to the UrlHelper::compressQueryParameters and UrlHelper::uncompressQueryParameters methods. These methods were not specifically created for only handling ajax_page_state (un)compression but right now they only seem to be used for this. A better solution might be to move the compression to a separate service.

The change we made reduced the size of the libraries token from 900 to about 360 characters which is quite a lot less.

adebruin’s picture

adebruin commented
StatusFileSize
new 3421255-13.patch4.03 KB

This is a patch file of the latest state of the MR for save usage with composer patches.

cilefen’s picture

cilefen commented
Issue tags: +Possible duplicate

We should consider closing this as a duplicate of #3403077: media_library_opener leads to massive GET requests that break varnish etc..

adebruin’s picture

adebruin commented

These tickets do seem to want to fix the same issue but with a different approach.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

drale01’s picture

drale01 commented

Same issue

quietone’s picture

quietone commented
Status: Active » Closed (duplicate)
Issue tags: +Bug Smash Initiative

This was changed to a support request by @longwave who has experienced the same problem. He also pointed out that this issue is about "AWS WAF" which does not use Varnish. Whereas the possible duplicate issue is specifically about Varnish. He also explained what he did to workaround the problem.

Based on the that, this issue can be closed.

I also asked in #bugsmash about this being a duplicate. acbramley replied that this does appear to be a duplicate. Since there are several people that think so I will close this as a duplicate and comment on the other issue that there is an MR here they may wish to review.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

quietone closed merge request !14116