Problem/Motivation

The link module only validates http and https, but not other scheme names such as git or ftp.

In Drupal 7, the validation can be turned off, thereby allowing the input of other scheme names. D7 sites could use this to use other scheme names, which will then not be valid in D8 any more.

Even better would be a option in the field configuration to set which scheme names are valid for this field.

Proposed resolution

  • Allow custom protocols.
  • Add more protocols to the list of allowed protocols Let's do this in a follow-up issue.

Remaining tasks

  1. List protocols which will be validated by Drupal. To be handled in the follow-up issue.
  2. Add field configuration for allowing custom protocols. Done.
  3. Ensure field validation included the custom protocols in the whitelist. Done.
  4. Add tests.

User interface changes

A new text field added to the Link field configuration form, empty by default, that accepts a comma-separated list of protocols that will be allowed in addition to the hard-coded ones.

API changes

+++ b/core/lib/Drupal/Component/Utility/UrlHelper.php
@@ -209,11 +209,13 @@ public static function encodePath($path) {
-  public static function isExternal($path) {
+  public static function isExternal($path, $protocol_whitelist = []) {
+++ b/core/lib/Drupal/Component/Utility/UrlHelper.php
@@ -353,8 +357,8 @@ public static function setAllowedProtocols(array $protocols = []) {
-  public static function stripDangerousProtocols($uri) {
+  public static function stripDangerousProtocols($uri, $whitelist = []) {
+++ b/core/lib/Drupal/Core/Utility/UnroutedUrlAssemblerInterface.php
@@ -40,6 +40,8 @@ interface UnroutedUrlAssemblerInterface {
+   *   - 'protocol_whitelist': An explicit whitelist of protocols to be allowed
+   *     in addition to those traditionally allowed. Defaults to an empty array.

This adds a new option to the $options array.

Data model changes

Added to the Link field configuration schema:

+++ b/core/modules/link/config/schema/link.schema.yml
@@ -49,6 +49,9 @@ field.field_settings.link:
+    protocol_whitelist:
+      type: string
+      label: 'Protocol whitelist'
CommentFileSizeAuthor
#52 2031149-48.patch11.92 KBliquidcms
#47 interdiff_2031149_45-47.txt1.29 KBankithashetty
#47 2031149-47.patch11.92 KBankithashetty
#46 interdiff-45_46.txt629 bytesGauravvvv
#46 2031149-46.patch11.96 KBGauravvvv
#45 interdiff_2031149_39-45.txt12.31 KBankithashetty
#45 2031149-45.patch11.92 KBankithashetty
#39 2031149-39.patch12.25 KBSuresh Prabhu Parkala
#38 2031149-36.patch12.22 KBblade_ukraine
#37 2031149-35--on-90x.do-not-test.patch13.17 KBmparker17
#37 2031149-35--on-89x.do-not-test.patch13.22 KBmparker17
#35 interdiff_33-35.txt2.68 KBraman.b
#35 2031149-35.patch12.25 KBraman.b
#33 interdiff_29-33.txt10.68 KBraman.b
#33 2031149-33.patch12.25 KBraman.b
#29 interdiff-2031149-27-29.diff3.53 KBcolan
#29 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-29.patch13.31 KBcolan
#27 interdiff-2031149-25-27.diff436 bytescolan
#27 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-27.patch13.23 KBcolan
#25 interdiff-2031149-23-25.diff3.69 KBcolan
#25 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-25.patch13.23 KBcolan
#23 interdiff-2031149-21-23.diff1.25 KBcolan
#23 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-23.patch9.98 KBcolan
#21 interdiff-2031149-18-21.diff1.17 KBcolan
#21 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-21.patch9.89 KBcolan
#18 interdiff-2031149-16-18.diff369 bytescolan
#18 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-18.patch9.7 KBcolan
#16 drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-16.patch9.17 KBcolan

Issue fork drupal-2031149

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Mac_Weber’s picture

Issue summary: View changes
Status: Active » Postponed

Postponing feature requests as of the allowed beta changes https://www.drupal.org/contribute/core/beta-changes

mgifford’s picture

Status: Postponed » Active

Drupal 8-beta 1 has been released.

Mac_Weber’s picture

Version: 8.0.x-dev » 8.1.x-dev
Status: Active » Postponed

Drupal 8-beta 1 has been released.

Actually we are on Beta 15 now. Please read the policy at https://www.drupal.org/contribute/core/beta-changes

mgifford’s picture

Yup.. I was just going on why this was postponed initially. Totally good to move this to 8.1 and postpone it as something to tackle there.

Mac_Weber’s picture

Title: Support more scheme names » Support more protocols and custom protocols validation
Issue summary: View changes
Related issues: +#2573635: Url::fromUri() should accept protocol-relative URLs, +#2410681: The renderer '#type' => 'link' does not render links to the public:// files
Mac_Weber’s picture

Category: Feature request » Task
Status: Postponed » Active

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

colan’s picture

Assigned: Unassigned » colan

I'm currently working on a solution to allow for a custom protocol whitelist, specified in the field settings. There won't be any custom validation just yet. I'd suggest handling that in a follow-up issue.

colan’s picture

Title: Support more protocols and custom protocols validation » Add support for additional protocols in Link field definition
Status: Active » Needs review
FileSize
9.17 KB

I updated the title as per my previous comment.

Even better would be a option in the field configuration to set which scheme names are valid for this field.

This patch adds a new configuration option that takes a comma-separated list of additional protocols, which are added to the default whitelist.

Status: Needs review » Needs work
colan’s picture

Looks like I forgot to add the new configurable to the schema.

colan’s picture

Issue summary: View changes
Issue tags: +Needs tests

Updated IS (while waiting for test results).

colan’s picture

colan’s picture

The last submitted patch, 23: drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-23.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

The last submitted patch, 25: drupal-add_support_for_additional_protocols_in_link_field_definition-2031149-25.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

colan’s picture

This adds the extra blank line I missed.

I can't fix the build failure, but assuming that clears up on its own, we should now be left with the Drupal\Tests\link\Functional\LinkFieldUITest::testFieldUI error:

The string "Saved link 1 16 0 1 configuration." was not found anywhere in the HTML response of the current page.

Anyone have any ideas?

colan’s picture

Through my own testing, the above patches won't work because the storage is assuming a string, when it should actually be an array.

colan’s picture

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

colan’s picture

Status: Needs review » Needs work
  1. whitelist -> allowlist
  2. May need a reroll for D9 if the retest fails.
raman.b’s picture

Re-rolling for 9.1.x and handling #32.1

raman.b’s picture

Status: Needs work » Needs review

Triggering the tests

raman.b’s picture

Missed a few instances

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

mparker17’s picture

Here is the patch from #35 rebased onto 8.9.x and 9.0.x (no interdiff because no changes).

blade_ukraine’s picture

I have an issue with #35, it does not apply to Drupal 9.1.5. Added fixes to the #35 patch for support Drupal 9.1.5.

Suresh Prabhu Parkala’s picture

A re-rolled patch against 9.2.x-dev.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

wjackson’s picture

The patch in #39 works as expected and allows me to add additional protocols as part of the field configuration.

wjackson’s picture

Status: Needs review » Reviewed & tested by the community

Marking as RTBC, as this successfully applies and works as requested in the original ticket description.

quietone’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs issue summary update

Sorry, this is still work to do here.

The Issue Summary and the tags state that this needs tests and I do not see a test of the new schema property, 'protocol_allowlist'. The patch in #39 is for 9.2.x instead of 9.3.x and it is not passing the commit code check.

The new property name 'protocol_allowlist' is a unusual, all the other instances in core of something that is allowed begins with the word allow. Therefor I think the name should be 'allowed_protocols'. And this change will require an Issue Summary update.

  1. +++ b/core/lib/Drupal/Component/Utility/UrlHelper.php
    @@ -206,11 +206,13 @@ public static function encodePath($path) {
    +   *   A list of extra protocols (strings) that have been explicitly allowed.
    

    s/array/string[]/

  2. +++ b/core/lib/Drupal/Component/Utility/UrlHelper.php
    @@ -206,11 +206,13 @@ public static function encodePath($path) {
    +   * @param array $protocol_allowlist
    

    s/array $protocol_allowlist /string[] $allowed_protocols/

    There are many, many places in the patch where $protocol_allowlist should be changed.

  3. +++ b/core/lib/Drupal/Component/Utility/UrlHelper.php
    @@ -206,11 +206,13 @@ public static function encodePath($path) {
    +   *   A list of extra protocols (strings) that have been explicitly allowed.
    

    I think custom is better than extra and there is no need to add strings, the hint is above.

    'The list of custom allowed protocols'.

  4. +++ b/core/lib/Drupal/Component/Utility/UrlHelper.php
    @@ -344,13 +346,15 @@ public static function setAllowedProtocols(array $protocols = []) {
        * @see \Drupal\Component\Utility\Html::escape()
    

    As above

  5. +++ b/core/modules/link/src/Plugin/Field/FieldType/LinkItem.php
    @@ -112,6 +122,25 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
    +   * Element validate; ensure that the Protocol Allowlist setting is valid.
    

    This feels clumsy. How about, "Validate the format of the allowed_protocol list.'

    It would probably be a good idea to add a sentence that this is not validating the protocols. The user can enter meaningless strings here.

  6. +++ b/core/modules/link/src/Plugin/Field/FieldType/LinkItem.php
    @@ -112,6 +122,25 @@ public function fieldSettingsForm(array $form, FormStateInterface $form_state) {
    +  public static function validateSettingsFormElementProtocolAllowlist($element, FormStateInterface $form_state, $form) {
    

    Not convinced the comment is needed here and the code can be simpler. One option is

        $allowed_protocols = array_map('trim', explode(',', $element['#value']));
        $allowed_protocols = $allowed_protocols == [''] ? [] : $allowed_protocols;
        $form_state->setValueForElement($element, $allowed_protocols);
quietone’s picture

Issue tags: +Needs screenshots

This also changes the user interface for the link field settings so a screenshot before and after is needed. The screenshots needs to be added to the Issue Summary to help reviewers and committers, thanks.

ankithashetty’s picture

Status: Needs work » Needs review
FileSize
11.92 KB
12.31 KB

Updated the patch in #39 to address all the changes provided in #43. Also fixed the custom command failure errors shown in #39. Kindly review the changes made in this patch.

Thank you!

Gauravvvv’s picture

Fixed custom command failed for unknown word. patch added and interdiff attached. Please review

ankithashetty’s picture

The CSpell check failed in core/modules/link/src/Plugin/Validation/Constraint/LinkExternalProtocolsConstraintValidator.php file, so updated the patch accordingly.

Thank you!

Status: Needs review » Needs work

The last submitted patch, 47: 2031149-47.patch, failed testing. View results

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

liquidcms’s picture

FileSize
11.92 KB

Patch from #47 doesn't apply to D9.4 (due to the smallest change in Umani demo config files).

This patch works for 9.4. Haven't tested with 9.5 but suspect it works there as well.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

s_leu made their first commit to this issue’s fork.

s_leu’s picture

Status: Needs work » Needs review
smustgrave’s picture

Status: Needs review » Needs work

Seems to have a test failure

Left some comments on MR.

Previously tagged for issue summary and screenshots.

Leaving tests tag as they will be needed for post_update hook.