Problem/Motivation

If you take Drupal 8 out-of-the box you will see that the Content-Length header is never being sent to the user.

The symfony component that is delivering the page content is doing this:

        $this->sendHeaders();
        $this->sendContent();

        if (function_exists('fastcgi_finish_request')) {
            fastcgi_finish_request();
        } elseif ('cli' !== PHP_SAPI) {
            static::closeOutputBuffers(0, true);
        }

Both Kernel termination events and any registered shutdown functions will execute after this. Not only core but contrib rely on defering code execution with register_shutdown_function() so that it will not slow down the request.

A more or less universal rule for any platform running PHP vía fastCgi is that the connection to the client will be closed:

  • a) If fastcgi_finish_request() is called explicitly.
  • b) If the Content-Length header is sent, and content matching the Content-Length header is flushed to the output.

See the issues regarding terminating the connection to the client and then being able to run code in the comments here:

http://php.net/manual/en/features.connection-handling.php
http://php.net/manual/en/function.register-shutdown-function.php

As it is now, only case (a) is being covered. So on some platforms/setups the response is being ransomed by shutdown functions and kernel termination events.

Just as a way of demonstrating this situation, if you add a sleep() call after sending the response in Index.php, you shold not expect your request to be ransomed by this:

$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();

sleep(5);

$kernel->terminate($request, $response);

But... response is being delayed 5 seconds in the example before being sent to the user! Adding the Content-Length header will make the output to be inmediately sent to the user after calling $response->end():

$request = Request::createFromGlobals();
$response = $kernel->handle($request);

$response->headers->add(array('Content-Length' => strlen($response->getContent())));
$response->send();

sleep(5);

Proposed resolution

Probably making delivery handlers explictly set the Content-Length header.

Remaining tasks

User interface changes

API changes

Data model changes

Issue fork drupal-2579775

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

david_garcia created an issue. See original summary.

david_garcia’s picture

Title: Missing Content-Length header blocks request until all registered shutdown functions execute and the Kernel terminate events run » Missing Content-Length header blocks response until all registered shutdown functions execute and the Kernel terminate events run

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Category: Bug report » Task
Status: Active » Needs review
Issue tags: +front-end performance
StatusFileSize
new2.73 KB

I can't believe this. You're totally right. I was certain you were wrong, and I was looking for where this was being set, but could not find it… because it never gets set!

We should definitely do this. But we need to make sure that we don't set this for streamed responses. Attached patch does this.

However, this is not a bug, it's a task. Nothing is broken. This merely improves (front-end) performance.

david_garcia’s picture

Status: Needs review » Reviewed & tested by the community

Thanks for taking on this. Tested and verified that works as advertised.

This makes a huge difference for non streamed responses. I.E. in the supercache module cache tags are persistend in a kernel terminate event, so the actual performance gains where "masked" by this "bug".

To me this is more an issue with the FastCgi implementations, ALL environments should have a fastcgi_finish_request() but we've got what we've got.....

After patching you need to rebuild, otherwise the new event subscriber will not get registered.

david_garcia’s picture

Status: Reviewed & tested by the community » Needs review

Too fast... looks like something broke with bigpipe, after enabling the module I am getting fiddler to complain about HTTP protocol violations where content length sent by server is greater than what was specified in content-length header.

david_garcia’s picture

Status: Needs review » Reviewed & tested by the community

The issue was me having an outdated version of bigpipe in /modules/ instead of the one in core.

wim leers’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests

The patch is not ready yet. It needs test coverage, and it needs to be cleaned up.

david_garcia’s picture

GOTCHA: If you send the content-length header, but for any reason, the output sent to the buffer is "smaller" thant the reported content-length header, then you get the exact opposite effect: the request is "ransomed" by fast-cgi indefinitely (until request times out) as it is expecting the missing content to arrive.

I am seeing this in the "extend" (priorly known as modules) page. Maybe due to encoding issues the flushed output does not match reported content-length.

wim leers’s picture

Maybe due to encoding issues the flushed output does not match reported content-length.

Sounds feasible.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

sassafrass’s picture

Hi... I need to access the Content_Length in the HTTP Client Header and tried applying this patch to Drupal core 8.9.10, but, not surprisingly, it failed. Was wondering if this functionality will be added to current and future versions of Drupal, or should I try a work-around? Thanks!

pobster’s picture

StatusFileSize
new3.05 KB

Rerolled for latest 8.9.x.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

bradjones1 made their first commit to this issue’s fork.

ravi.shankar’s picture

StatusFileSize
new2.51 KB
new2.93 KB

Added reroll of patch #20 on Drupal 9.4.x., still needs work for tests.

pooja saraah’s picture

StatusFileSize
new2.49 KB
new705 bytes

Fixed failed commands on #25
Attached interdiff patch

bradjones1’s picture

Status: Needs work » Closed (duplicate)

I'm marking this as a duplicate of #3295790: Post-response task running (destructable services) are actually blocking; add test coverage and warn for common misconfiguration which fixes a broader range of post-response termination concerns. Credit should be given there for meaningful contributors to this issue.