Problem/Motivation

Link checker normally checks links during cron and respects the configured check interval. There is no way to immediately re-check a single link, a selection of links, or all links after fixing content, changing configuration, or testing Link checker changes.

#3616064: Provide a way to check specific links immediately implemented the same feature with forms and actions. Its ideas have been merged into MR !114 here, so it can be closed as a duplicate.

Proposed resolution

Add on-demand checks wherever links are managed:

  • "Check" operation on each link in the existing operations column of the Broken links and Own broken links reports. It checks the link immediately (CSRF-protected) and returns to the report with the old and new status code.
  • Bulk action "Check the status of the selected links", with a confirmation form. It checks the selected links in a batch, using check.connections_max concurrent requests like cron does.
  • "Check" button on the link edit form.
  • "Check links" maintenance button on the settings form. It immediately checks all enabled links, regardless of the check interval.
  • Drush command drush linkchecker:check-link 12,34 (alias lcchl).
  • An optional "Check link status" Views field for custom views. It is not added to the shipped report, which uses the operation instead.

Who may check a link is the same everywhere:

  • users with access broken links report;
  • the owner of the content the link was found in, if they have access own broken links report;
  • anyone who may update that content.

Links with checking disabled or without a URL are skipped, the same as cron does.

Two post_updates bring existing sites up to date without manual Views changes:

  • one creates the action;
  • one adds the bulk form column to the broken_links_report view.

The "Check" operation needs no view change, because the report already has an operations column.

Also:

  • getParentEntity() now returns NULL when a stored parent entity type no longer exists, instead of throwing (#21).
  • The "Check links" batch now removes processed queue items, so cron doesn't check the same links again.

Out of scope, for follow-ups:

Remaining tasks

User interface changes

  • "Check" in the operations dropbutton on the Broken links reports.
  • A bulk operations column (checkboxes) as the first column of the reports, with a confirmation form at /admin/reports/linkchecker/check.
  • A "Check" button on the link edit form.
  • A "Check links" button under Maintenance on the Link checker settings form.

API changes

  • New: LinkCheckerService::checkMultiple(), LinkCheckerService::isIgnoredResponseCode(), LinkCheckerBatch::batchSpecific() and LinkCheckerBatch::batchProcessSpecific().
  • Changed signatures:
    • LinkCheckerService::queueLinks() and LinkCheckerBatch::batch() take an optional $ignoreInterval.
    • LinkCheckerBatch::processQueueItem() now returns the number of links processed.
    • batchProcessQueue() and batchFinished() have new signatures.
  • LinkCheckerAdminSettingsForm::__construct() takes an optional LinkCheckerBatch as its last argument; it falls back to the container, so subclasses keep working.
  • New routes linkchecker.check_status and linkchecker.multiple_check_confirm, the linkchecker_check_link action plugin, and the linkchecker_check_status Views field plugin.
  • Minimum Drupal core version is raised to 10.3, for plugin attributes.

Data model changes

New config: system.action.linkchecker_check_link. The broken_links_report view gets a linkcheckerlink_bulk_form field. Both are added to existing sites by post_updates.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

gugalamaciek created an issue. See original summary.

gugalamaciek’s picture

gugalamaciek’s picture

Status: Active » Needs review
gugalamaciek’s picture

StatusFileSize
new5.84 KB
solideogloria’s picture

I think "Check status of the link" is too long of a default label. How about "Retest" or "Check" or something? When editing an entity, the button/link is just "Edit" rather than "Edit the node".

solideogloria’s picture

Status: Needs review » Needs work

joseph.olstad made their first commit to this issue’s fork.

joseph.olstad’s picture

hmm, I tested in my view, Drupal 11.1.x with PHP 8.3.20+ seeing this:

Error: Call to a member function label() on null in Drupal\linkchecker\Controller\LinkcheckerStatusController->checkStatus() (line 56 of modules/contrib/linkchecker/src/Controller/LinkcheckerStatusController.php).
joseph.olstad’s picture

Status: Needs work » Needs review
joseph.olstad’s picture

StatusFileSize
new4.24 KB
joseph.olstad’s picture

joseph.olstad’s picture

joseph.olstad’s picture

joseph.olstad’s picture

Assigned: joseph.olstad » Unassigned
joseph.olstad’s picture

This latest patch adds a "Check links" button on the Maintenance accordion on the linkchecker settings form /admin/config/content/linkchecker

This button is back by popular demand, clients asked , they get.

joseph.olstad’s picture

Credit should go to @jamesyao for the Check links button functionality.

rp7 made their first commit to this issue’s fork.

joseph.olstad’s picture

MR 111 is for 2.0.x
MR 114 is for 2.1.x

I highly recommend that everyone use 2.1.x instead of 2.0.x, 2.0.x has a dependency on der , a very heavy module that should be avoided whenever possible.

imclean’s picture

Using the MR with 2.1.x, I'm seeing the following error when trying to view a report:

Drupal\Component\Plugin\Exception\PluginNotFoundException: The "" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 142 of /web/core/lib/Drupal/Core/Entity/EntityTypeManager.php).

The issue seems to come from here:

/web/modules/contrib/linkchecker/src/LinkCheckerLinkAccessControlHandler.php(68): Drupal\linkchecker\Entity\LinkCheckerLink->getParentEntity()
/web/modules/contrib/linkchecker/src/LinkCheckerLinkAccessControlHandler.php(58): Drupal\linkchecker\LinkCheckerLinkAccessControlHandler->checkParentEntityAccess()

Does the view need to be modified before this feature can be used?

imclean’s picture

Clearing the links and rechecking fixed the problem.

imclean’s picture

There's a problem with this approach.

When there's a broken link on a page, usually the page needs to be updated with the new link or the link is removed altogether.

Using this MR, you can recheck a single link or all links, which it does, but it will continue to return the same error if the remote link isn't valid any more.

It might be more useful to rescan the entity/entities the link was on.

In other words, the broken link itself isn't the problem, the problem is the entity which contains the broken link.

joelpittet’s picture

RE #23 I agree with you @imclean — re-scanning the entity on this manual task would make it significantly more useful. Just to confirm, which MR are you reviewing? Based on Joseph’s comment and yours, I’m guessing it’s MR !114, but want to be sure we’re all looking at the same thing.

imclean’s picture

@joelpittet, yes I'm using MR !114 on 2.1.x.

solideogloria’s picture

Title: Allow to check link status on demand » Allow to rescan entity on demand

I agree, rescanning the entity is 100% more useful.

imclean’s picture

Status: Needs review » Needs work

I guess this would probably be a more appropriate status.

joseph.olstad’s picture

For us we want to do a full scan with button press without waiting for cron.

Sounds like #26 is a new feature request. Should be therefore a new issue.

It's been years already we've used this (as per merge request and patch) for a few years now.

imclean’s picture

@joseph.olstad

For us we want to do a full scan with button press without waiting for cron.

Yes, that's what we want as well.

The current MR allows you to check the remote links again, however usually the problem with dead remote links is that they don't work any more so rescanning the links stored in the database will continually return the same error.

Dead/broken links are changed by updating the entity with new links, or just removing the broken links. After that, the entity needs to be checked again.

There is already a way to do a full scan of all entities. On the linkchecker config screen, under "Maintenance", it's the button "Clear link data and analyze content for links". This could take a while on larger sites.

Being able to rescan individual entities on demand could be useful. It would need to also verify the dead link isn't on any other pages before removing it from the broken link report.

That said, I'm curious as to how you are using this MR. Are you checking internal, external or both types of links? How does the MR help?

joseph.olstad’s picture

That said, I'm curious as to how you are using this MR. Are you checking internal, external or both types of links?

We're checking both types of links.

How does the MR help?

We're clicking the re-analyze button (a real Snoozer) before clicking the "Check links" button (another snoozer).

imclean’s picture

I mean, what's the objective of the MR? For example, say you have some external links on your site which are no longer working. How does this MR help?

imclean’s picture

I'm using the views field to recheck individual links, but the link is no longer valid so always returns an error.

joseph.olstad’s picture

How does this MR help

We don't have to wait a week for cron to re-scan when we're testing out linkchecker releases and or other patches.

imclean’s picture

There already is a button for rescanning on demand: "Clear link data and analyze content for links"

What problem is this issue addressing? (Taking into account my previous comments above.)

joseph.olstad’s picture

@imclean , ya when you're using the above merge request or patch there's a "Check links" button which when pressed will check links. This means you won't have to wait for cron which is often set as weekly on a set interval for instance. No waiting, want it to happen right now then click the button.

This is very helpful for developers like me that occasionally have to do deep dives into this module or look into specific configurations and or folks that for some reason do not want to wait for cron or deal with cron settings or requiring a specific extra cron related module that triggers cron for linkchecker.

joelpittet’s picture

Title: Allow to rescan entity on demand » Allow to check link status on demand

I agree with @joseph.olstad this is starting to feel like scope creep. While adding a manual rescan for entity links (to catch false positives on already corrected links on the content entity) would definitely be useful, it's a larger change. What we have now is already valuable: it allows us to manually verify that a broken link is still broken, without waiting for cron.

@solideogloria @iamclean — I'm resetting the title to reflect the current scope. If we decide to include the rescan functionality, it should be as a follow-up to this patch. My preference is to get this in as-is and open a separate issue for the manual rescan enhancement to this. If you'd strongly prefer to combine them, I'd urge you to fork the current MR and explore that approach on top of what's here in a separate MR... but I find it a bit clearer to make them new issues.

I hope that helps and doesn't muddy the waters — feel free to disagree, I am not the maintainer here.

joseph.olstad’s picture

Version: 2.0.x-dev » 2.1.x-dev
Status: Needs work » Needs review

setting to 2.1.x
(what is 3.0.x)? maybe 3.0.x instead?

imclean’s picture

@joelpittet, that's fine, I'm just struggling to understand the use case for the original issue, which makes it hard to review.

Going back to first principles, can the IS be updated with a detailed explanation beyond just avoiding cron. For example, when a link on an entity is no longer valid - what are the next steps?

And, as I said, we don't need another "rescan all links" button, because there already is one.

imclean’s picture

From the IS: "Let's say, I made changes of link setup and now I want to confirm that it works as I expected."

I missed the significance of this. OK this makes sense.

joseph.olstad’s picture

We're also using this patch, I'm not sure if it's needed for the check button functionality though, might be overkill.
#3212816: Interval of 86400 (one day) is impractical for debugging using GUI

We're also using other linkchecker patches that are unrelated to this.

anybody’s picture

#3471860: [PP-1] Add action to immediately update a 301 URL to the new target is slightly related and contains some more ideas, maybe it could be a nice follow-up.

anybody’s picture

MR 111 is for 2.0.x
MR 114 is for 2.1.x

thanks @joseph.olstad I added a comment to the MRs (while it's also in the header technically) to make this clear, because I was also wondering which is the right MR.

Rebased both and agree this would be totally useful in real-world scenarios ;) LGTM!

anybody’s picture

anybody’s picture

Maybe it would make sense to add an action to allow this as bulk operation? (Or as follow-up?)

joel_osc’s picture

StatusFileSize
new5.17 KB

@anybody if people are interested in a VBO action here is a start on it - we are already testing it on a few sites. The action provides a broken link report as a CSV when complete.

anybody’s picture

Thanks, but I'm not a big fan of such "alternative approaches", let's better get a clean solution finished here.

anybody’s picture

tomefa’s picture

I tried the PR together with many others patchs and got this error:

TypeError: Drupal\linkchecker\Form\LinkCheckerAdminSettingsForm::__construct(): Argument #6 ($checkerBatch) must be of type Drupal\linkchecker\Form\LinkCheckerBatch, Drupal\linkchecker\LinkCheckerBatch given, called in /app/docroot/modules/contrib/linkchecker/src/Form/LinkCheckerAdminSettingsForm.php on line 120 in Drupal\linkchecker\Form\LinkCheckerAdminSettingsForm->__construct() (line 102 of modules/contrib/linkchecker/src/Form/LinkCheckerAdminSettingsForm.php).

solideogloria’s picture

@tomefa You might just need to clear your cache. There's no such thing as Drupal\linkchecker\Form\LinkCheckerBatch

tomefa’s picture

Yes you are right, it was due to some other patchs that were conflicting.
I tried this patch alone and it was applying as expected.

joelpittet’s picture

Issue summary: View changes

Updated the issue summary as requested in #38 (@imclean) to:

  • explain the actual use case;
  • document both parts of the MR: the Views field and "Check links" button;
  • note the getParentEntity() hardening from #21; and
  • mark entity rescanning (#23–#36) and a bulk action (#44) as out of scope for follow-ups.

To keep this focused I am hiding 2.0.x branch and only managing 2.1.x. This will make it into 2.2.0 with any luck. Check out the roadmap to help get that out.
#3624551: [meta] 2.2.0 stable release roadmap

joelpittet changed the visibility of the branch 3426268-allow-to-check to hidden.

joelpittet’s picture

Issue tags: +Needs manual testing

Widened access to the on-demand check route so it isn't limited to the global report. The route now uses a custom access check (CheckLinkStatusAccessCheck) that allows a link to be checked by:

  • users with access broken links report (as before),
  • the owner of the content the link was found in, when they have access own broken links report (so the "Check" field also works on the per-user report from #3334562),
  • anyone with update access to that content, regardless of report permissions, so editors can re-check immediately after fixing a link.

Links whose parent entity no longer exists fall back to the global permission only. Results are cached per permissions and per user with dependencies on the link and its parent. Since the Views field defers to route access, the "Check" column simply stays empty for users who can't check a given link.

I did this with #3504635: Report on Content Edit page in mind.

grevil made their first commit to this issue’s fork.

grevil’s picture

Issue tags: -Needs manual testing

Changes LGTM! just merged some duplicate code to a shared helper method. Also tested it manually and everything works as expected.

One more thing! Is it intentional, that "views.view.broken_links_report.yml" wasn't touched? Currently the "LinkChecker link type: Check link status (Check link status)" field needs to be added manually, which I find quite tedious.

I understand that updating existing views is really annoying, but I think, we should at least update the default "views.view.broken_links_report.yml" and also let the user know about the new field in the update hook? We could also create a post_update hook. What do the maintainers say?

I'll update the "default" view for now, the rest should be decided.

anybody’s picture

Status: Needs review » Needs work

Thank you @grevil, I totally agree and think in the case of linkchecker typically the view will be untouched, so I think that change should be done in a defensive update hook!

Doing that in a test doesn't make much sense to me.

So I'd say that's the last step needed? All other parts also LGTM! :)

@joelpittet do you agree and will you do that?

codebymikey’s picture

Wasn't aware this issue exists, but this functionality was also implemented in #3616064: Provide a way to check specific links immediately, but using forms/actions so it also easily supports bulk operations as well - I think it requires a rebase following the latest changes in dev.

But might be worth reviewing as well, and potentially picking up some of its implementation details here and closing that one as a duplicate.

anybody’s picture

Funny, I wasn't aware of the other one :D

But I agree actions totally make sense.

I'd say we should then

  • rebase the other one
  • ensure that the action is visible in the list (not only bulk) - update the view?
  • ensure that the action is visible in the details page
  • ensure that we have tests for everything

As I think the issues show that is this a very relevant feature.

joelpittet changed the visibility of the branch 2.1.x to hidden.

joelpittet’s picture

Issue summary: View changes
Status: Needs work » Needs review

Rebased #3616064: Provide a way to check specific links immediately and then closed it as a duplicate so we can finish this in one place. Thanks @codebymikey, I've ported the best parts over here and credited on this one.

The biggest change is that "Check" is now an entity operation, so it shows up in the operations dropbutton the Broken links report already has. That's a much better fit than a separate Views column: existing sites get it with no view changes at all, so I reverted adding the "Check link status" column to the default view.

@grevil that makes me wonder whether we need the "Check link status" Views field at all now. The operation covers the shipped reports, and a custom view can add the operations field to get the same link. Dropping it would slim the MR down a bit more: the field plugin, its hook_views_data() and schema, and one test. Any objections, or did you have a use case for the separate column in mind?

Also ported from @codebymikey's MR:

  • LinkCheckerService::checkMultiple() checks links in parallel, respecting check.connections_max the same way cron does.
  • A bulk action "Check the status of the selected links", with a confirmation step. This one does need a post_update, sorry @anybody, but it only adds the bulk form column to broken_links_report and creates the action. Both do nothing if they've already been applied.
  • drush linkchecker:check-link 12,34 (alias lcchl).

A few tweaks on the way in:

  • The operation, bulk action and Check button all use the same access as the check route from #53: report users, owners on their own report, and anyone who can edit the content.
  • Links with checking disabled or no URL are skipped everywhere, the same as cron.

@anybody I also added a "Check" button on the link edit form, which I took as the "details page" you mentioned in #58, since links don't have a separate view page.

Tests cover the operation, bulk check, edit form button, checkMultiple(), the batch and both post_updates. I updated the issue summary to match.

To try it locally, check out the branch, run drush updb, then use "Check" or the bulk action on admin/reports/linkchecker.

An LLM help with some of the tedious porting and collating of this. So a real world test would be appreciated before merging this. I did some very rough manual testings to see if the tests were covering things well enough.

joelpittet’s picture

@grevil since the operation covers the shipped reports and a custom view can add the operations field for the same link, I went ahead and dropped the separate "Check link status" Views field to slim this down. Shout if you had a use case for it in mind and we can discuss.

grevil’s picture

@joelpittet, good stuff!

I prefere the operations integration (looks way cleaner now). Good idea! The operation works as expected! RTCB regarding the new operation.

grevil’s picture

Two things I adjusted:
- The submit handler of the "Check links" button built and deleted the whole query, only to see if there was anything to check and then the actual batch built and executed the query again.
- "checkMultiple" now checks the status of the links via the links hash (to not check the same link twice) and copies their status to each link entity (same behavior as the queue worker).

Otherwise all features implemented here seem to work great! I'd set this to "RTBC", but since I changed stuff myself, someone should take a look at the last two commits.

joelpittet’s picture

Oh thank you for stepping in to fix that multi(I will double check but imagine you’re likely right), I’ll do one last review and likely commit this (tomorrow).

Please 🙏 anyone else on this issue do a manual check on their projects would be appreciated!