Problem/Motivation

We explicitly deny the External plugin to crawl private IP addresses and localhost accessible to the web server. However we should also allow a site administrator (or site host via config overrides) to deny access to crawl URLs by host pattern (hostname and/or port). This should be a deny list because the entire point of the External plugin is to allow humans to link to other web sites.

Steps to reproduce

N/A

Proposed resolution

Add a URL Deny List (host_denylist) option to the External plugin, which allows a text format administrator to enter a newline separated list of host patterns that the External plugin should not crawl. The private URL and localhost options should still exist separate from this.

Remaining tasks

  • Determine if 4.0.x should have this change. Probably?
  • Code
  • Tests
  • Config schema update + config update
  • Change record

User interface changes

Adds a new form element to the configuration form.

API changes

To be determined.

Data model changes

Adds new configuration schema

Issue fork freelinking-3629151

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

mradcliffe created an issue. See original summary.

mradcliffe’s picture

mradcliffe
he/him
Primary language English
commented
Issue summary: View changes

shamir.vs made their first commit to this issue’s fork.

shamir.vs opened merge request !15

shamir.vs’s picture

shamir.vs commented
Status: Active » Needs review

I have implemented the URL Deny List (host_denylist) configuration option for the External freelinking plugin.

What’s included

  • Added the host_denylist setting to the External plugin configuration.
  • Added support for:
    • Exact hostnames, e.g. example.com
    • Wildcard domains, e.g. *.example.com
    • Specific ports, e.g. example.com:8080
    • Wildcard ports, e.g. *:8080
  • Added validation to safely handle protocol prefixes, trailing slashes, comments, and empty lines.
  • Preserved the existing protection against localhost and private/reserved IPv4 and IPv6 addresses.
  • Added the required configuration schema.
  • Added a post-update hook to initialize the new setting for existing configurations.
  • Added automated test coverage for the deny-list functionality and updated the existing default configuration tests.
  • Fixed the PHPStan and PHPCS issues reported by GitLab CI.

The changes are ready for review.

Kindly have a look and let me know if any changes or improvements are needed. Thanks!