Determine how much POST truncation protection is needed and add tests for it

There is/was a check in FileWidgetAjaxController which protected against broken POST requests.

Firstly, curious if there's any test coverage for it. If not, this will pass, which I suspect is the case.

Secondly, I manually tested (on PHP 5.5 on MAMP) uploading a file that's bigger than php.ini's post_max_size. Regardless of whether upload_max_filesize is bigger, equal, or smaller than post_max_size, the behavior is the same, which is that all of $_POST is empty, regardless of whether the other elements are before or after the file element. That is consistent with PHP's documented behavior.

So, retitling this issue, since it's not necessarily a settled conclusion that the order of the element matters.

However, I did not test situations of a request body truncated prior to PHP, such as from an aborted connected from the client or something like suhosin on the server. It's possible that order matters in such a case, in which case, we need to decide whether we want the element coming through successfully (by ordering it before the file element) or not (by ordering it after). Or maybe, we want 'form_build_id' early in the form and some other element (e.g., 'form_truncation_check') late in the form. Before going down that road of messing with order through, let's see if PHP takes care of things for us, by e.g., automatically detecting if the request body length is less than the Content-Length header and giving us an empty $_POST for us (wouldn't that be nice).

If the only purpose of checking $form_build_id is to cover the case of an empty $_POST due to a file larger than post_max_size, then it's unnecessary to compare it against the one from the query string. This patch should provide the same protection, but allow removal of the query string one, which is necessary for #2500527: Rewrite \Drupal\file\Controller\FileWidgetAjaxController::upload() to not rely on form cache.

Patch for #2.

Also, the current protection being only in FileWidgetAjaxController is problematic due to:

• It's possible to submit forms that exceed the post_max_size, even without files, though I guess that's much less common.
• This code does not run for people without JS. I tried creating an article with JS disabled, and when I try to upload a file > post_max_size, I just get back a redisplay of an empty form, with no friendly message of any kind, because without anything in $_POST, the form is processed identically as during the GET.

so there is no not enough bytes left for the full other rest of the form data

No, I think even there PHP will empty out all of $_POST. The only thing I'm unsure of is what happens when it's not PHP that applies post_max_size, but something prior to PHP truncating the request body. Then the request body length will be less than the 'Content-Length' header, but I don't know if PHP gives you an empty $_POST for that, or ignores the header and parses the incomplete body anyway. If the latter, then we may need a hidden input element at the end of the form to detect that.

Adding related issue. A long time since I've looked at that one, but that also results in silent truncation (or did at the time that issue was active).

There seem to be a lot of misunderstandings here. An HTTP server really should not process an incomplete HTTP message. If something in the chains allows an incomplete HTTP message to be processed, it is a bug in that something.

Can we understand what is happening and identify which infrastructure that are affected? Do we have a setup somewhere that is affected?

For information, RFC7230 defines an incomplete message as:

A message body that uses the chunked transfer coding is incomplete if the zero-sized chunk that terminates the encoding has not been received. A message that uses a valid Content-Length is incomplete if the size of the message body received (in octets) is less than the value given by Content-Length. A response that has neither chunked transfer coding nor Content-Length is terminated by closure of the connection and, thus, is considered complete regardless of the number of message body octets received, provided that the header section was received intact.

About @dawehner#8:

Removed Content-Length: PHP did not complained about it. Then I removed the last 50 chars from the POST data (ajaxPageData[libraries]). PHP then still returned you everything before it.

^ This is not possible. You cannot submit an HTTP message body without either a Content-Length or using a chunked transfer encoding (which carries the length information by itself).

Replaced Content-Length with a higher value: PHP did not complained about it either, at all.

^ This is not possible, or it means that you are actually using chunked transfer encoding without knowing it. If you were not using chunked transfer encoding, the request would just block until there is sufficient data.

Replaced Content-Length with a lower value: (10000 => 1000) In that case the request is indeed truncated.

^ That is expected.

This has gone a long time without comment, and there is much discussion about the direction since the last posted patch.