I lost 5 hours of my life today, trying to work out why on earth IE users couldn't log into our stage site. I checked session handlers, recorded network traffic, debugged the site to itty bitty little pieces.

Turns out there's a peculiar quirk in IE. Technically, underscores are not permitted in domain/host-names...at least, RFCs 952 and 1123 don't permit them. Most browsers function properly, IE even renders the page...but silently drops any cookies on that domain. Which makes logging in rather an issue.

This patch checks for the presence of underscores in the host-name, and raises a requirements warning if any underscores are present.

Rationale for addressing this issue in core:

  • Underscores in domain (and sub-domain) names are not compliant with the RFCs, but this is not widely-known, even amongst experienced developers.
  • Although IE is technically compliant with the RFCs, its apparent support for domains with underscores (in that it attempts to serve and render the page) serves to obfusticate its inherent lack of support.
  • Drupal already caters for specific IE quirks - form_builder() is one example of a function which caters specifically for IE support.
  • The correlation between the cause (an underscore in the host-name), and the effect (Internet Explorer users are unable to log in) is non-obvious, and the natural problem-research/problem-solving approaches don't lend themselves to discovering the cause easily. The status-report page is an obvious first-call for problem solving, and highlighting the issue there serves as an immediate shortcut to fixing this issue.
  • I and other very experienced developers have been stymied and lost many many hours investigating this problem.
  • The patch has minimal impact on performance and is very simple to understand, so the maintenance cost is very low
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

manarth’s picture

Status: Active » Needs review

Sending the patch to the testbot...

brahmjeet789’s picture

brahmjeet789’s picture

Issue summary: View changes

Fixing typo in spelling.

kevla’s picture

Issue summary: View changes

Just come across this myself. That patch would have been really useful if it had been merged in.

only4kaustav’s picture

IE has problems accepting cookies from subdomain's that dont follow the URI RFC. (http://www.ietf.org/rfc/rfc2396.txt)

manarth’s picture

Version: 7.x-dev » 8.2.x-dev
Status: Needs review » Needs work
Issue tags: +Novice

Patch needs updating for D8.

droplet’s picture

+++ b/modules/system/system.install
@@ -486,6 +486,20 @@ function system_requirements($phase) {
+      'description' => $t('Host names should not include an underscore. Whilst most browsers will still function, Internet Explorer will appear to serve the web site, but will silently drop all cookies for the domain, preventing Internet Explorer users from logging in.'),

Can we shorten the text to say just don't support it in Drupal at all without explain in detail.

manarth’s picture

I'm generally in favour of brevity, but in this case, I think it's useful to highlight the consequences of this requirement failing, especially as (at least with the D7 version of the patch) the requirement is only displayed when it's failing.

The aim of this requirement is to highlight the consequences of an underscore in the host-name, and give relevant feedback to someone trying to debug a problem with IE users logging in. If it simply said "Host names should not include an underscore", the relevance may not be obvious, so it might well be ignored (just as I would ignore other failures, such as file-system not writeable, cron not run recently, etc).

There are also descriptions in core's hook_requirements that are a similar length (e.g. module update-status, upload-progress).

wellme’s picture

Assigned: Unassigned » wellme
wellme’s picture

Status: Needs work » Needs review
FileSize
1.22 KB

Please find the patch.

manarth’s picture

Status: Needs review » Needs work

One very minor niggle: the indentation on the line $requirements is 1 space out.
Everything else looks good to me.

leolandotan’s picture

Hi! I found some minor issues. Others can check the logic thoroughly too.

  1. +++ b/core/modules/system/system.install
    @@ -23,6 +23,20 @@ function system_requirements($phase) {
    +    // underscore character is not permitted.  Whilst most browsers will still
    ...
    +      'title' => t('Host name'),
    +      'value' => t('Fails'),
    +      'severity' => REQUIREMENT_WARNING,
    +      'description' => t('Host names should not include an underscore. Whilst most browsers will still function, Internet Explorer will appear to serve the web site, but will silently drop all cookies for the domain, preventing Internet Explorer users from logging in.'),
    

    Please replace the double spaces before "Whilst".

  2. +++ b/core/modules/system/system.install
    @@ -23,6 +23,20 @@ function system_requirements($phase) {
    +   $requirements['hostname'] = array(
    

    Please fix indentions and its array value declarations.

Thanks!

Ankit Agrawal’s picture

Re-rolled the #10 patch as per #12. Thank you!

Ankit Agrawal’s picture

Assigned: wellme » Unassigned
Status: Needs work » Needs review

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

joelpittet’s picture

Issue tags: +Dublin2016

This could use a review, the patch is straightforward just needs to make sure it is correct english and does what is suggested in the issue summary accurately.

paul_charlet’s picture

Status: Needs review » Reviewed & tested by the community

Reviewed. It does what is suggested.

brahmjeet789’s picture

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 13: 1444718-13.patch, failed testing.

manarth’s picture

Status: Needs work » Reviewed & tested by the community

False negative?

The original patch-report on #13 shows that the patch passed testing, and a manual review doesn't expose any issues. I've resubmitted for another test.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 13: 1444718-13.patch, failed testing.

karlosgliberal’s picture

Status: Needs work » Reviewed & tested by the community

Applied patch is working fine

catch’s picture

Status: Reviewed & tested by the community » Needs work

Does filter_var($url, FILTER_VALIDATE_URL) handle underscores in domain names? If so should we use it here?

Also the message should show the hostname I think. In the middle of a long list of requirement notices you don't necessarily want to look up to the address bar and back.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

drubb’s picture

Updated the patch to use the new core array syntax, added the domain name to the message text.

drubb’s picture

Issue tags: +DevDaysSeville
drubb’s picture

Status: Needs work » Needs review

Not sure about the filter_var() thing. While it returns FALSE for domain names containing underlines, what's the advantage here? It also performs checks not related to this issue.

Haza’s picture

Does filter_var($url, FILTER_VALIDATE_URL) handle underscores in domain names? If so should we use it here?

It does check over underscores. But if we use that here, it might also return false because of an other error in the domain name. That means if we still warn the user about underscores only in the error message when using filter_var(), we might lead the user to a false direction.

Status: Needs review » Needs work

The last submitted patch, 25: 1444718-25.patch, failed testing.

Haza’s picture

+++ b/core/modules/system/system.install
@@ -29,6 +29,20 @@ function system_requirements($phase) {
+    $requirements['hostname'] = array(
+    (...)
+    );

it should be updated to the short array notation [].

drubb’s picture

Just overlooked this. Changed.

drubb’s picture

Status: Needs work » Needs review
darius.restivan’s picture

Seems OK.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

joelpittet’s picture

Status: Needs review » Needs work

$_SERVER['HTTP_HOST'] is not assumed in core, and also could be on $_SERVER['SERVER_NAME'] depending on proxy.

Consider checking both values and at least doing an isset()?

This suggests $_SERVER['SERVER_NAME'] being more reliable, but neither being always available.
https://stackoverflow.com/questions/2297403/what-is-the-difference-betwe...

joelpittet’s picture

Issue tags: +Vienna2017
rakesh_verma’s picture

I'm having a look at this one at DC Vienna.

narnua’s picture

Taking a look with @rakesh_verma at Vienna - edit: can review the patch once rakesh_verma has uploaded it

rakesh_verma’s picture

Considering both $_SERVER['HTTP_HOST'] and $_SERVER['SERVER_NAME'] with an isset. Just as mentioned in comment #35.

narnua’s picture

Patch fails to apply:

1444718-39.patch:12: tab in indent.
	  (isset($_SERVER['SERVER_NAME']) && strpos($_SERVER['SERVER_NAME'], '_') !== FALSE)) {
error: corrupt patch at line 27

Suggestion: change tabs to spaces for indentation

rakesh_verma’s picture

Fixing the test cases.

narnua’s picture

Tried applying patch from #41, error:

error: corrupt patch at line 27

Probably some tab/tabs still left, so rerolling the patch for someone else to review.

Also: this issue should probably be tested by a Windows / IE user to verify the fix works.

narnua’s picture

Patch 43 only aims at removing last remaining tabs, no other changes made.

A bit unsure about the if statement vs Drupal coding standards: as the statement is > 80 characters, it should probably be divided to 2 lines, however might need restructuring to separate statements according to https://www.drupal.org/docs/develop/standards/coding-standards#linelength. I used my best judgement so left it as is for now - hoping at least the patch can now be applied.

narnua’s picture

Status: Needs work » Needs review
brahmjeet789’s picture

mohit1604’s picture

larowlan’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/system/system.install
    @@ -32,6 +32,22 @@ function system_requirements($phase) {
    +  // $_SERVER['HTTP_HOST'] is not assumed in core and could be on $_SERVER['SERVER_NAME']. Henceforth checking both of them.
    

    Comment needs to wrap at 80 chars.

  2. +++ b/core/modules/system/system.install
    @@ -32,6 +32,22 @@ function system_requirements($phase) {
    +  if ((isset($_SERVER['HTTP_HOST']) && strpos($_SERVER['HTTP_HOST'], '_') !== FALSE) || (isset($_SERVER['SERVER_NAME']) && strpos($_SERVER['SERVER_NAME'], '_') !== FALSE)) {
    ...
    +      'description' => t('Host names should not include an underscore, like in %domain. Whilst most browsers will still function, Internet Explorer will appear to serve the web site, but will silently drop all cookies for the domain, preventing Internet Explorer users from logging in.', ['%domain' => $_SERVER['HTTP_HOST']]),
    

    These should use the \Drupal::request()->getHttpHost() which handles all this juggling as well as validation.

piyuesh23’s picture

Status: Needs work » Needs review
FileSize
1.51 KB

I agree around the usage of wrapper function getHttpHost on Request object provided by HttpFoundation rather than relying on $_SERVER. Attaching an updated patch.

Mixologic’s picture

Was making some tweaks to drupalci to use hostnames, and the hostnames I created arbitrarily had underscores in them. (things like http://php_apache_local_41f5fbd85887179e9e38d7234c3ce954).

Anyhow, took me about half a day to figure out what the problem was, so I look forward to something like this patch being louder about invalid configurations.

alexpott’s picture

Status: Needs review » Needs work
Issue tags: +Needs manual testing
  1. +++ b/core/modules/system/system.install
    @@ -32,6 +32,24 @@ function system_requirements($phase) {
    +  $http_host = \Drupal::request()->getHttpHost();
    

    Is there a reason to use getHttpHost() over getHost()? I don't think there is a reason for having the port appended in this instance.

  2. +++ b/core/modules/system/system.install
    @@ -32,6 +32,24 @@ function system_requirements($phase) {
    +  if (strpos($http_host, '_') !== FALSE || (isset($_SERVER['SERVER_NAME']) && strpos($_SERVER['SERVER_NAME'], '_') !== FALSE)) {
    

    There is no need to handle $_SERVER['SERVER_NAME'] here. All you need is: if (strpos($http_host, '_') !== FALSE) {

    See \Symfony\Component\HttpFoundation\Request::getHost().

One thing that is interesting is that the Symfony code allows underscores see...

        // as the host can come from the user (HTTP_HOST and depending on the configuration, SERVER_NAME too can come from the user)
        // check that it does not contain forbidden characters (see RFC 952 and RFC 2181)
        // use preg_replace() instead of preg_match() to prevent DoS attacks with long host names
        if ($host && '' !== preg_replace('/(?:^\[)?[a-zA-Z0-9-:\]_]+\.?/', '', $host)) {
            throw new \UnexpectedValueException(sprintf('Invalid Host "%s"', $host));
        }
Psy Shell v0.8.0 (PHP 7.2.0RC2 — cli) by Justin Hileman
>>> $host = 'blah_ddgfd';
=> "blah_ddgfd"
>>> preg_replace('/(?:^\[)?[a-zA-Z0-9-:\]_]+\.?/', '', $host)
=> ""
alexpott’s picture

There is an interesting debate on https://github.com/symfony/symfony/issues/10467 where it is proposed to allows underscores in URIs... so what's super fun is that whilst getHost() allows underscores... \Symfony\Component\Validator\Constraints\UrlValidator::validate() doesn't. Seems like everyone including M$ are confused on whether underscores should or should not be supported.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs work » Closed (outdated)

Closing this as outdated since the issue seems to be centered around IE which is a no longer supported browser

smustgrave’s picture

Issue tags: +Bug Smash Initiative