Problem/Motivation

During review of #2122761: SessionHttpsTest only runs half the tests it was discovered that the $form variable was populated but never used in several places of SessionHttpsTest. Close inspection of the matter revealed that this was the case when enforcing a login through one of the http.php and https.php mock front-controllers. Nevertheless, the current code should be improved in order to improve readability.

Proposed resolution

Let's take a second look on SessionHttpsTest and clean it up:

  • Remove usage of the globals $base_secure_url and $base_insecure_url from bootstrap.inc
  • Remove the globals $is_http_mock and $is_https_mock.
  • Turn $secure_session_name and $insecure_session_name into instance variables on SessionHttpsTest and populate them during setUp().
  • Extract the new helper methods SessionHttpsTest::loginHttp() and SessionHttpsTest::loginHttps() in order to make the test-code more self-documenting
  • Make testHttpsSession() work when the test runs in an HTTPS environment by consequently using httpUrl() and httpsUrl() if necessary.
  • Fix redirect to non-existing URL after POST requests (see #9)

Remaining tasks

Review.

User interface changes

None.

API changes

None.

History of the SessionHttpsTest class

The SessionHttpsTest was introduced by #1577: Mapping privilege separation on non-SSL/SSL (opened in 2003 and committed in 2009) along with most of the SSL mixed mode functionality. The patch brought in the globals $is_https, $base_secure_url, $base_insecure_url and the variable $conf['https']. Also the https.php mock was added and in order to enforce redirects to go through HTTP from within a mocked HTTPS request, hook_goto_alter was introduced as well. In order to be able to store both, secure and insecure session ids, the session table got an additional column (ssid).

Subsequently it was discovered that sites operating in mixed mode SSL were vulnerable to an impersonation attack #575280: Impersonation when an https session exists.. In addition to fixing the vulnerability test coverage was extended. Special attention went into making the tests work when the test-runner was a HTTPS environment. In order to make this possible, the http.php mock front-controller was added.

Also noteworthy is #813492: HTTPS sessions use an invalid merge query because it intermittedly roled back changes introduced in #575280: Impersonation when an https session exists. and also left a bogus assortion which is still found in current code: The headers argument is third in drupalGet(), not second. The assertion fails if the parameters are correct (opened #2330279: When operating in mixed mode SSL: data from anonymous non-HTTP session is still available after login).

// Test that session data saved before login is not available using the 
// pre-login anonymous cookie.
$this->cookies = array();
$this->drupalGet('session-test/get', array('Cookie: ' . $anonymous_cookie));
$this->assertNoText($session_data, 'Initial anonymous session is inactive after login.');

The redirect bug (see #9) was introduced in #1668866: Replace drupal_goto() with RedirectResponse. The problem is that RedirectResponse::getTargetUrl() returns an absolute URL. However due to the fact that SessionHttpsTest did not examine response HTTP status, and probably due to #2122761: SessionHttpsTest only runs half the tests this slipped through the cracks.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug
Issue priority Major because it affects security related end-to-end tests
Unfrozen changes Unfrozen because it only changes tests. (SessionHttpsTest)
Prioritized changes The main goal of this issue is security.
CommentFileSizeAuthor
#66 2265099-66.diff14.32 KBznerol
#66 interdiff.txt831 bytesznerol
#60 2265099-60.patch15.13 KBrpayanm
#56 cleanup-2265099-56.patch15.28 KBneclimdul
#49 2265099-cleanup-session-https-test-49.diff15.32 KBznerol
#48 interdiff.txt809 bytesznerol
#48 2265099-cleanup-session-https-test-48.diff25.57 KBznerol
#46 interdiff.txt3.28 KBznerol
#46 2265099-cleanup-session-https-test-46.diff25.56 KBznerol
#44 interdiff.txt1.06 KBznerol
#44 2265099-cleanup-session-https-test-44.diff24.35 KBznerol
#39 interdiff.txt1.68 KBznerol
#39 2265099-cleanup-session-https-test-39.diff24.42 KBznerol
#38 2265099-cleanup-session-https-test-38.diff26.09 KBznerol
#33 2265099-cleanup-session-https-test-33.diff25.98 KBznerol
#32 2265099-cleanup-session-https-test-32.diff26.44 KBznerol
#30 interdiff.txt7.15 KBznerol
#30 2265099-cleanup-session-https-test-30.diff24.62 KBznerol
#22 interdiff.txt8.27 KBznerol
#22 2265099-cleanup-session-https-test-22.diff25.14 KBznerol
#20 interdiff.txt2.32 KBznerol
#20 2265099-cleanup-session-https-test-20.diff23.02 KBznerol
#19 interdiff.txt1.4 KBznerol
#19 2265099-cleanup-session-https-test-19.diff22.09 KBznerol
#13 2265099-cleanup-session-https-test-13.diff22.05 KBznerol
#11 2265099-cleanup-session-https-test-11.diff21.93 KBznerol
#9 redirect-to-nowhere.png107.24 KBznerol
#9 interdiff.txt7.37 KBznerol
#9 2265099-cleanup-session-https-test-9.diff21.8 KBznerol
#8 interdiff.txt1.44 KBznerol
#8 2265099-cleanup-session-https-test-8.diff18.05 KBznerol
#5 interdiff.txt957 bytesznerol
#5 2265099-cleanup-session-https-test-5.diff17.16 KBznerol
#3 interdiff.txt5.68 KBznerol
#3 2265099-cleanup-session-https-test-3.diff17.15 KBznerol
#2 interdiff.txt3.36 KBznerol
#2 2265099-cleanup-session-https-test-2.diff13.75 KBznerol
#1 2265099-cleanup-session-https-test-1.diff11.11 KBznerol
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

znerol’s picture

Status: Active » Needs review
FileSize
11.11 KB
  • No need to alter the request-service if ::http[s]Url() is used wherever its necessary
  • Make ::testMixedModeSslSession() work on HTTPS test environment
  • Move common code to ::setUp().
znerol’s picture

  • Remove the global variables $base_insecure_url and $base_secure_url
  • No need use the return an absolute URL from ::http[s]Url().
znerol’s picture

Extract inlined login code via mock http/https to dedicated methods. This reveals two different uses of $form[0]['action']:

  • Modify the form action such that drupalPost(NULL, ...); will submit to one of the mock front controllers (user login form)
  • Checking whether the form action has the correct protocol on a rendered form

That essentially addresses the concern raised by @dawehner in #2122761-9: SessionHttpsTest only runs half the tests:

test needs more research, given that $form is modified but not used in multiple places.

The last submitted patch, 2: 2265099-cleanup-session-https-test-2.diff, failed testing.

znerol’s picture

Fix missing $path variable in SessionTestSubscriber.

The last submitted patch, 3: 2265099-cleanup-session-https-test-3.diff, failed testing.

znerol’s picture

Issue summary: View changes
znerol’s picture

No need to enforce authentication from within session test request-subscriber.

znerol’s picture

Do not tamper with redirect responses inside the system under test. Instead assign the responsibility of modifying the protocol of the location header appropriately to the test case itself.

This also fixes a problem with the current https test where the browser was redirected to an in-existing URL after form submission. However this went unnoticed because the HTTP status of the final response is not checked (see screenshot):

znerol’s picture

Issue summary: View changes
znerol’s picture

znerol’s picture

Title: Cleanup SessionHttpsTest » Cleanup SessionHttpsTest and fix redirect to non-existing URL after POST requests
znerol’s picture

Priority: Major » Critical
Issue tags: +Security
FileSize
22.05 KB

Reroll. I'm bumping this to critical because this fixes a serious defect in an important test (security-wise).

Status: Needs review » Needs work

The last submitted patch, 13: 2265099-cleanup-session-https-test-13.diff, failed testing.

cosmicdreams’s picture

+++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
@@ -281,6 +264,84 @@ protected function testCsrfTokenWithMixedModeSsl() {
+    $base_url . 'index.php/';

This line is suspect as we're concatenating 'index.php' to $base_url but we're not storing that result. Effectively this does nothing.

Does the fact that it does nothing break assumptions of what this test is doing? Looks like the following functions may all be wrong as a result;
loginHttp
loginHttps
testCsrfTokenWithMixedModeSsl

Not a full answer but curious.

+++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
@@ -28,44 +43,36 @@ class SessionHttpsTest extends WebTestBase {
-    $this->assertTrue($this->cookies[$secure_session_name]['secure'], 'The secure cookie has the secure attribute');

Even though cookies is a sensical name for this. There isn't any container named cookies in this class or the base class WebTestBase.

however this is a cookieFile. Should that be used?

dawehner’s picture

@znerol
Do you think you can fix the failures? Would be great to not have the security problem for too long.

znerol’s picture

Assigned: Unassigned » znerol

Yes, in the meantime I discovered some additional problems with the tests. My working copy is still a big mess though, I'll try to put together a patch soon.

znerol’s picture

Status: Needs work » Needs review

Fixes the bad reroll from #19.

Note that the patch still contains bugs, e.g.:

+++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
@@ -281,6 +264,84 @@ protected function testCsrfTokenWithMixedModeSsl() {
+  /**
+   * Log in a user via HTTPS.
+   *
+   * Note that the parents $session_id and $loggedInUser is not updated.
+   */
+  protected function loginHttps(AccountInterface $account) {
...
+    $this->drupalGet($this->httpUrl($path));

Should be httpsUrl().

Also issues pointed out in #15 not yet addressed.

znerol’s picture

znerol’s picture

This fixes the obvious bugs pointed out in #15 and #19. Regrettably I still see failures when the test-runner is a HTTPS environment. Will have to set-up wireshark in order to understand what is going on there.

Status: Needs review » Needs work

The last submitted patch, 20: 2265099-cleanup-session-https-test-20.diff, failed testing.

znerol’s picture

Status: Needs work » Needs review
FileSize
25.14 KB
8.27 KB
cosmicdreams’s picture

  1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
    @@ -793,7 +793,6 @@ protected function initializeRequestGlobals(Request $request) {
    -    global $base_secure_url, $base_insecure_url;
    

    Yay! fewer globals.

  2. +++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
    @@ -28,81 +43,75 @@ class SessionHttpsTest extends WebTestBase {
    +    $this->curlCookies = array();
    

    Does nulling out the curlCookies make more sense inside the curlClose(); function?

    Is this something we want to optionally do?

+++ b/core/modules/system/tests/modules/session_test/src/EventSubscriber/SessionTestSubscriber.php
@@ -68,9 +52,9 @@ public function onKernelResponseSessionTest(FilterResponseEvent $event) {
+    $events[KernelEvents::RESPONSE][] = array('onKernelResponseSessionTest');
+    $events[KernelEvents::REQUEST][] = array('onKernelRequestSessionTest');

why are we removing priority?

znerol’s picture

Issue summary: View changes

Added some historical information i digged up from old issues (git log --patch ftw).

znerol’s picture

Issue summary: View changes
cosmicdreams’s picture

OK, in reviewing the interdiffs it appears that the priority ensured that the tests get access to specialized logic that was altering the request. It appears that this was a special case that the original developers introduced because they thought it was necessary. Apparently it isn't anymore so great. This patch seems RTBC to me.

znerol’s picture

@cosmicdreams: Give me some time, there is still commented out code in the patch:

+++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
@@ -281,6 +278,105 @@ protected function testCsrfTokenWithMixedModeSsl() {
+    //$session_cookies = [];
+    //foreach ([$this->insecureSessionName, $this->secureSessionName] as $cookie) {
+    //  if (isset($this->cookies[$cookie]['value']) && $this->cookies[$cookie]['value'] != 'deleted') {
+    //    $session_cookies[] = $cookie . '=' . $this->cookies[$cookie]['value'];
+    //  }
+    //}
+    //$cookie = implode('; ', $session_cookies);

Also I like to check whether we can get rid of the curlClose() calls now that cookies are managed manually (via curlCookies property).

znerol’s picture

znerol’s picture

+++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
@@ -28,81 +43,75 @@ class SessionHttpsTest extends WebTestBase {
+    // Prevent that curl parses and records cookies.
+    $this->additionalCurlOptions = array(
+      CURLOPT_COOKIEJAR => FALSE,
+    );

Regrettably this does not have the desired effect. In fact it is not possible to disable the internal cookie handling mechanism of curl after either CURLOPT_COOKIEJAR or CURLOPT_COOKIEFILE has been set to any value. Therefore the only way to get rid of cookies which were picked up by curl during previous requests is closing the curl handle.

znerol’s picture

Switching back to curl built-in cookie handling. Cleaned up the patch such that it is hopefully review-worthy now.

xjm’s picture

Priority: Critical » Major

Thanks @znerol!

Downgrading to major based on discussion with @znerol; this is important but not release-blocking, and there isn't an actual vulnerability here, just a need for hardening and better test coverage for safe refactoring later.

znerol’s picture

Reroll after #2330279: When operating in mixed mode SSL: data from anonymous non-HTTP session is still available after login landed in core. This version now successfully runs all tests regardless of whether the test-runner is HTTP or HTTPS.

znerol’s picture

Status: Needs review » Needs work

The last submitted patch, 33: 2265099-cleanup-session-https-test-33.diff, failed testing.

Status: Needs work » Needs review
znerol’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll
znerol’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
26.09 KB
znerol’s picture

StackMiddleware\Session obviously does not belong into this issue.

mgifford’s picture

Thanks @znerol for all your work on this!

What needs to be tested to mark this RTBC.

I do agree that * An absolute URL. isn't very useful. But I do wonder if this could be shortened to a single line:

+   *   The URL prepared in a way such that the https.php mock front controller
+   *   is used.

Say "Absolute URL prepared for the https.php mock front controller"?

znerol’s picture

Manual testing of the patch should be performed like this:

  • Deploy a fresh D8 instance on a SSL enabled webserver
  • Run the SessionHttpsTest from the simpletest UI (once through http, once through https)
  • Run the SessionHttpsTest from the CLI:
    php core/scripts/run-tests.sh --url http://example.com/ --file core/modules/system/src/Tests/Session/SessionHttpsTest.php
    php core/scripts/run-tests.sh --url https://example.com/ --file core/modules/system/src/Tests/Session/SessionHttpsTest.php
znerol’s picture

Say "Absolute URL prepared for the https.php mock front controller"?

In fact it is not an absolute URL at all but a relative one. The function simply prepends the $url parameter with the path to the mock frontcontroller: return 'core/modules/system/tests/http.php/' . $url;.

mgifford’s picture

I think this is longer and less clear than it needs to be "The URL prepared in a way such that the https.php mock front controller is used."

How about something like:
"URL prepared for the https.php mock front controller."

znerol’s picture

dawehner’s picture

Are we sure this is not a critical issue as it seems to block #2283637: Provide test coverage to prove that an AuthenticationProvider can initiate a session ?

  1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
    @@ -798,7 +798,6 @@ protected function initializeRequestGlobals(Request $request) {
    -    global $base_secure_url, $base_insecure_url;
    
    @@ -839,8 +838,6 @@ protected function initializeRequestGlobals(Request $request) {
         }
    -    $base_secure_url = str_replace('http://', 'https://', $base_url);
    -    $base_insecure_url = str_replace('https://', 'http://', $base_url);
     
    

    Oh, that is great for itself!

  2. +++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
    @@ -28,81 +43,68 @@ class SessionHttpsTest extends WebTestBase {
       protected function setUp() {
         parent::setUp();
    -    $this->request = Request::createFromGlobals();
    -    $this->container->get('request_stack')->push($this->request);
    -  }
     
    -  protected function testHttpsSession() {
    -    if ($this->request->isSecure()) {
    -      $secure_session_name = $this->getSessionName();
    -      $insecure_session_name = substr($this->getSessionName(), 1);
    +    $request = Request::createFromGlobals();
    +    if ($request->isSecure()) {
    +      $this->secureSessionName = $this->getSessionName();
    +      $this->insecureSessionName = substr($this->getSessionName(), 1);
         }
         else {
    -      $secure_session_name = 'S' . $this->getSessionName();
    -      $insecure_session_name = $this->getSessionName();
    +      $this->secureSessionName = 'S' . $this->getSessionName();
    +      $this->insecureSessionName = $this->getSessionName();
         }
    +  }
     
    

    For sure that is part of another issue, but is there a reason we can't create a request object suited for the HTTP and one for the HTTPs usecase?

  3. +++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
    @@ -133,62 +126,57 @@ protected function testMixedModeSslSession() {
         $this->assertNotEqual(substr($form[0]['action'], 0, 6), 'https:', 'Password request form action is not secure');
    -    $form[0]['action'] = $this->httpsUrl('user/login');
     
    ...
         $this->assertEqual(substr($form[0]['action'], 0, 6), 'https:', 'Login form action is secure');
    -    $form[0]['action'] = $this->httpsUrl('user/login');
     
    

    Can we do this changes because we drop mix mode?

  4. +++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
    @@ -269,13 +251,22 @@ protected function testCsrfTokenWithMixedModeSsl() {
    +    $maximum_redirects = $this->maximumRedirects;
    +    $this->maximumRedirects = 0;
    +    $this->drupalPostForm(NULL, $edit, 'Save');
    +    $this->maximumRedirects = $maximum_redirects;
    +
    +    // Follow the redirect header.
    +    $path = $this->getPathFromLocationHeader(TRUE);
    +    $this->drupalGet($this->httpUrl($path));
    +    $this->assertResponse(200);
    

    It would be great to maybe have a short documentation line telling what is going on: We disable the automatic redirect, in order to do what?...

  5. +++ b/core/modules/system/src/Tests/Session/SessionHttpsTest.php
    @@ -284,6 +275,93 @@ protected function testCsrfTokenWithMixedModeSsl() {
    +    // Generate the base_url.
    +    $base_url = $this->container->get('url_generator')->generateFromPath('', array('absolute' => TRUE));
    

    Can't you also use $this->container->get('url_generator')->generateFromRoute('', [], ['absolute' => TRUE]);?

znerol’s picture

Are we sure this is not a critical

See #31, but it should be fixed nevertheless rather sooner than later.

  1. Indeed.
  2. Not interested in the request at all here (it is not an instance variable anymore). The request is only used in order to populate $this->insecureSessionName and $this->$secureSessionName.
  3. This is not about mixed mode sessions per se, but about whether or not the form should be posted to one of the HTTP/HTTPS mock front controllers. This fragments have been extracted to loginHttp and loginHttps methods. Note that even if mixed mode sessions are removed, we still need to test the characteristics of the resulting cookies when logging in via HTTP vs HTTPS.
  4. Comment added.
  5. Fixed.

Status: Needs review » Needs work

The last submitted patch, 46: 2265099-cleanup-session-https-test-46.diff, failed testing.

znerol’s picture

Status: Needs work » Needs review
FileSize
25.57 KB
809 bytes

Re #45.5, empty route name results in route not found, need explicitly supply <front>.

znerol’s picture

Status: Needs review » Needs work

The last submitted patch, 49: 2265099-cleanup-session-https-test-49.diff, failed testing.

mgifford’s picture

What was I doing trying to test a diff???? Anyways, got the patch in #48.

The last submitted patch, 48: 2265099-cleanup-session-https-test-48.diff, failed testing.

neclimdul’s picture

Status: Needs work » Needs review
FileSize
15.28 KB

Just a re-roll

almaudoh queued 56: cleanup-2265099-56.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 56: cleanup-2265099-56.patch, failed testing.

almaudoh’s picture

Issue tags: +Needs reroll
Fabianx’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs beta evaluation, +Needs change record updates

Gonna be bold. It looks great, RTBC!

Needs a beta evaluation (unfrozen: tests-only) and a CR addition to remove $base_url and $base_secure_url. (sniff, I liked those ...)

znerol’s picture

Issue summary: View changes

Just to be clear, it does not remove $base_url, the issue summary is still accurate. I do not think that the removed globals are API because they were only used for tests in core. Which CR needs an update?

znerol’s picture

Issue tags: -Needs beta evaluation
alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -826,7 +826,6 @@ protected function initializeRequestGlobals(Request $request) {
-    global $base_secure_url, $base_insecure_url;

@@ -864,8 +863,6 @@ protected function initializeRequestGlobals(Request $request) {
-    $base_secure_url = str_replace('http://', 'https://', $base_url);
-    $base_insecure_url = str_replace('https://', 'http://', $base_url);

These have been around since d7 - contrib code might be relying on them - we should update a CR or provide a new one. It would be great if we can get rid of DrupalKernel::initializeRequestGlobals() - is there an issue to do that? We could just remove the removals from this issue and do the cleanup there.

znerol’s picture

What about $is_http_mock and $is_https_mock? Do we need a CR update for them?

znerol’s picture

Status: Needs work » Needs review
FileSize
831 bytes
14.32 KB
alexpott’s picture

@znerol for $is_http_mock and $is_https_mock - i don't think so - that is truly just test stuff for the tests you are changing.

znerol’s picture

Issue summary: View changes
Issue tags: -Needs change record updates
Fabianx’s picture

Status: Needs review » Reviewed & tested by the community

Contrib code like securepages / secureforms / ... indeed tries to rely on those super globals, however it should be fine to remove them with a proper CR, etc.

Thanks for restoring for now.

Back to RTBC

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

This is a test only change and therefore permitted in beta. Committed d29700f and pushed to 8.0.x. Thanks!

  • alexpott committed d29700f on 8.0.x
    Issue #2265099 by znerol, neclimdul, rpayanm: Cleanup SessionHttpsTest...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.