Problem/Motivation
Link checker normally checks links during cron and respects the configured check interval. There is no way to immediately re-check a single link, a selection of links, or all links after fixing content, changing configuration, or testing Link checker changes.
#3616064: Provide a way to check specific links immediately implemented the same feature with forms and actions. Its ideas have been merged into MR !114 here, so it can be closed as a duplicate.
Proposed resolution
Add on-demand checks wherever links are managed:
- "Check" operation on each link in the existing operations column of the Broken links and Own broken links reports. It checks the link immediately (CSRF-protected) and returns to the report with the old and new status code.
- Bulk action "Check the status of the selected links", with a confirmation form. It checks the selected links in a batch, using
check.connections_maxconcurrent requests like cron does. - "Check" button on the link edit form.
- "Check links" maintenance button on the settings form. It immediately checks all enabled links, regardless of the check interval.
- Drush command
drush linkchecker:check-link 12,34(aliaslcchl). - An optional "Check link status" Views field for custom views. It is not added to the shipped report, which uses the operation instead.
Who may check a link is the same everywhere:
- users with access broken links report;
- the owner of the content the link was found in, if they have access own broken links report;
- anyone who may update that content.
Links with checking disabled or without a URL are skipped, the same as cron does.
Two post_updates bring existing sites up to date without manual Views changes:
- one creates the action;
- one adds the bulk form column to the
broken_links_reportview.
The "Check" operation needs no view change, because the report already has an operations column.
Also:
getParentEntity()now returnsNULLwhen a stored parent entity type no longer exists, instead of throwing (#21).- The "Check links" batch now removes processed queue items, so cron doesn't check the same links again.
Out of scope, for follow-ups:
- re-scanning the parent entity for links (#23–#36);
- #3471860: [PP-1] Add action to immediately update a 301 URL to the new target, updating 301 links to their new target.
Remaining tasks
- Review MR !114.
Close #3616064: Provide a way to check specific links immediately as a duplicate, with credit carried over.- Forward-port to 3.0.x after commit.
User interface changes
- "Check" in the operations dropbutton on the Broken links reports.
- A bulk operations column (checkboxes) as the first column of the reports, with a confirmation form at
/admin/reports/linkchecker/check. - A "Check" button on the link edit form.
- A "Check links" button under Maintenance on the Link checker settings form.
API changes
- New:
LinkCheckerService::checkMultiple(),LinkCheckerService::isIgnoredResponseCode(),LinkCheckerBatch::batchSpecific()andLinkCheckerBatch::batchProcessSpecific(). - Changed signatures:
LinkCheckerService::queueLinks()andLinkCheckerBatch::batch()take an optional$ignoreInterval.LinkCheckerBatch::processQueueItem()now returns the number of links processed.batchProcessQueue()andbatchFinished()have new signatures.
LinkCheckerAdminSettingsForm::__construct()takes an optionalLinkCheckerBatchas its last argument; it falls back to the container, so subclasses keep working.- New routes
linkchecker.check_statusandlinkchecker.multiple_check_confirm, thelinkchecker_check_linkaction plugin, and thelinkchecker_check_statusViews field plugin. - Minimum Drupal core version is raised to 10.3, for plugin attributes.
Data model changes
New config: system.action.linkchecker_check_link. The broken_links_report view gets a linkcheckerlink_bulk_form field. Both are added to existing sites by post_updates.
| Comment | File | Size | Author |
|---|
Issue fork linkchecker-3426268
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
gugalamaciek commentedComment #3
gugalamaciek commentedComment #4
gugalamaciek commentedComment #5
solideogloria commentedI think "Check status of the link" is too long of a default label. How about "Retest" or "Check" or something? When editing an entity, the button/link is just "Edit" rather than "Edit the node".
Comment #6
solideogloria commentedComment #9
joseph.olstadhmm, I tested in my view, Drupal 11.1.x with PHP 8.3.20+ seeing this:
Comment #10
joseph.olstadComment #11
joseph.olstadComment #12
joseph.olstadComment #13
joseph.olstadSame as https://git.drupalcode.org/project/linkchecker/-/merge_requests/111.diff
Comment #14
joseph.olstadComment #15
joseph.olstadComment #16
joseph.olstadThis latest patch adds a "Check links" button on the Maintenance accordion on the linkchecker settings form
/admin/config/content/linkcheckerThis button is back by popular demand, clients asked , they get.
Comment #17
joseph.olstadCredit should go to @jamesyao for the Check links button functionality.
Comment #20
joseph.olstadMR 111 is for 2.0.x
MR 114 is for 2.1.x
I highly recommend that everyone use 2.1.x instead of 2.0.x, 2.0.x has a dependency on der , a very heavy module that should be avoided whenever possible.
Comment #21
imclean commentedUsing the MR with 2.1.x, I'm seeing the following error when trying to view a report:
The issue seems to come from here:
Does the view need to be modified before this feature can be used?
Comment #22
imclean commentedClearing the links and rechecking fixed the problem.
Comment #23
imclean commentedThere's a problem with this approach.
When there's a broken link on a page, usually the page needs to be updated with the new link or the link is removed altogether.
Using this MR, you can recheck a single link or all links, which it does, but it will continue to return the same error if the remote link isn't valid any more.
It might be more useful to rescan the entity/entities the link was on.
In other words, the broken link itself isn't the problem, the problem is the entity which contains the broken link.
Comment #24
joelpittetRE #23 I agree with you @imclean — re-scanning the entity on this manual task would make it significantly more useful. Just to confirm, which MR are you reviewing? Based on Joseph’s comment and yours, I’m guessing it’s MR !114, but want to be sure we’re all looking at the same thing.
Comment #25
imclean commented@joelpittet, yes I'm using MR !114 on 2.1.x.
Comment #26
solideogloria commentedI agree, rescanning the entity is 100% more useful.
Comment #27
imclean commentedI guess this would probably be a more appropriate status.
Comment #28
joseph.olstadFor us we want to do a full scan with button press without waiting for cron.
Sounds like #26 is a new feature request. Should be therefore a new issue.
It's been years already we've used this (as per merge request and patch) for a few years now.
Comment #29
imclean commented@joseph.olstad
Yes, that's what we want as well.
The current MR allows you to check the remote links again, however usually the problem with dead remote links is that they don't work any more so rescanning the links stored in the database will continually return the same error.
Dead/broken links are changed by updating the entity with new links, or just removing the broken links. After that, the entity needs to be checked again.
There is already a way to do a full scan of all entities. On the linkchecker config screen, under "Maintenance", it's the button "Clear link data and analyze content for links". This could take a while on larger sites.
Being able to rescan individual entities on demand could be useful. It would need to also verify the dead link isn't on any other pages before removing it from the broken link report.
That said, I'm curious as to how you are using this MR. Are you checking internal, external or both types of links? How does the MR help?
Comment #30
joseph.olstadWe're checking both types of links.
We're clicking the re-analyze button (a real Snoozer) before clicking the "Check links" button (another snoozer).
Comment #31
imclean commentedI mean, what's the objective of the MR? For example, say you have some external links on your site which are no longer working. How does this MR help?
Comment #32
imclean commentedI'm using the views field to recheck individual links, but the link is no longer valid so always returns an error.
Comment #33
joseph.olstadWe don't have to wait a week for cron to re-scan when we're testing out linkchecker releases and or other patches.
Comment #34
imclean commentedThere already is a button for rescanning on demand: "Clear link data and analyze content for links"
What problem is this issue addressing? (Taking into account my previous comments above.)
Comment #35
joseph.olstad@imclean , ya when you're using the above merge request or patch there's a "Check links" button which when pressed will check links. This means you won't have to wait for cron which is often set as weekly on a set interval for instance. No waiting, want it to happen right now then click the button.
This is very helpful for developers like me that occasionally have to do deep dives into this module or look into specific configurations and or folks that for some reason do not want to wait for cron or deal with cron settings or requiring a specific extra cron related module that triggers cron for linkchecker.
Comment #36
joelpittetI agree with @joseph.olstad this is starting to feel like scope creep. While adding a manual rescan for entity links (to catch false positives on already corrected links on the content entity) would definitely be useful, it's a larger change. What we have now is already valuable: it allows us to manually verify that a broken link is still broken, without waiting for cron.
@solideogloria @iamclean — I'm resetting the title to reflect the current scope. If we decide to include the rescan functionality, it should be as a follow-up to this patch. My preference is to get this in as-is and open a separate issue for the manual rescan enhancement to this. If you'd strongly prefer to combine them, I'd urge you to fork the current MR and explore that approach on top of what's here in a separate MR... but I find it a bit clearer to make them new issues.
I hope that helps and doesn't muddy the waters — feel free to disagree, I am not the maintainer here.
Comment #37
joseph.olstadsetting to 2.1.x
(what is 3.0.x)? maybe 3.0.x instead?
Comment #38
imclean commented@joelpittet, that's fine, I'm just struggling to understand the use case for the original issue, which makes it hard to review.
Going back to first principles, can the IS be updated with a detailed explanation beyond just avoiding cron. For example, when a link on an entity is no longer valid - what are the next steps?
And, as I said, we don't need another "rescan all links" button, because there already is one.
Comment #39
imclean commentedFrom the IS: "Let's say, I made changes of link setup and now I want to confirm that it works as I expected."
I missed the significance of this. OK this makes sense.
Comment #40
joseph.olstadWe're also using this patch, I'm not sure if it's needed for the check button functionality though, might be overkill.
#3212816: Interval of 86400 (one day) is impractical for debugging using GUI
We're also using other linkchecker patches that are unrelated to this.
Comment #41
anybody#3471860: [PP-1] Add action to immediately update a 301 URL to the new target is slightly related and contains some more ideas, maybe it could be a nice follow-up.
Comment #42
anybodythanks @joseph.olstad I added a comment to the MRs (while it's also in the header technically) to make this clear, because I was also wondering which is the right MR.
Rebased both and agree this would be totally useful in real-world scenarios ;) LGTM!
Comment #43
anybodyComment #44
anybodyMaybe it would make sense to add an action to allow this as bulk operation? (Or as follow-up?)
Comment #45
joel_osc commented@anybody if people are interested in a VBO action here is a start on it - we are already testing it on a few sites. The action provides a broken link report as a CSV when complete.
Comment #46
anybodyThanks, but I'm not a big fan of such "alternative approaches", let's better get a clean solution finished here.
Comment #47
anybodyComment #48
tomefa commentedI tried the PR together with many others patchs and got this error:
TypeError: Drupal\linkchecker\Form\LinkCheckerAdminSettingsForm::__construct(): Argument #6 ($checkerBatch) must be of type Drupal\linkchecker\Form\LinkCheckerBatch, Drupal\linkchecker\LinkCheckerBatch given, called in /app/docroot/modules/contrib/linkchecker/src/Form/LinkCheckerAdminSettingsForm.php on line 120 in Drupal\linkchecker\Form\LinkCheckerAdminSettingsForm->__construct() (line 102 of modules/contrib/linkchecker/src/Form/LinkCheckerAdminSettingsForm.php).
Comment #49
solideogloria commented@tomefa You might just need to clear your cache. There's no such thing as
Drupal\linkchecker\Form\LinkCheckerBatchComment #50
tomefa commentedYes you are right, it was due to some other patchs that were conflicting.
I tried this patch alone and it was applying as expected.
Comment #51
joelpittetUpdated the issue summary as requested in #38 (@imclean) to:
getParentEntity()hardening from #21; andTo keep this focused I am hiding 2.0.x branch and only managing 2.1.x. This will make it into 2.2.0 with any luck. Check out the roadmap to help get that out.
#3624551: [meta] 2.2.0 stable release roadmap
Comment #53
joelpittetWidened access to the on-demand check route so it isn't limited to the global report. The route now uses a custom access check (
CheckLinkStatusAccessCheck) that allows a link to be checked by:Links whose parent entity no longer exists fall back to the global permission only. Results are cached per permissions and per user with dependencies on the link and its parent. Since the Views field defers to route access, the "Check" column simply stays empty for users who can't check a given link.
I did this with #3504635: Report on Content Edit page in mind.
Comment #55
grevil commentedChanges LGTM! just merged some duplicate code to a shared helper method. Also tested it manually and everything works as expected.
One more thing! Is it intentional, that "views.view.broken_links_report.yml" wasn't touched? Currently the "LinkChecker link type: Check link status (Check link status)" field needs to be added manually, which I find quite tedious.
I understand that updating existing views is really annoying, but I think, we should at least update the default "views.view.broken_links_report.yml" and also let the user know about the new field in the update hook? We could also create a post_update hook. What do the maintainers say?
I'll update the "default" view for now, the rest should be decided.
Comment #56
anybodyThank you @grevil, I totally agree and think in the case of linkchecker typically the view will be untouched, so I think that change should be done in a defensive update hook!
Doing that in a test doesn't make much sense to me.
So I'd say that's the last step needed? All other parts also LGTM! :)
@joelpittet do you agree and will you do that?
Comment #57
codebymikey commentedWasn't aware this issue exists, but this functionality was also implemented in #3616064: Provide a way to check specific links immediately, but using forms/actions so it also easily supports bulk operations as well - I think it requires a rebase following the latest changes in dev.
But might be worth reviewing as well, and potentially picking up some of its implementation details here and closing that one as a duplicate.
Comment #58
anybodyFunny, I wasn't aware of the other one :D
But I agree actions totally make sense.
I'd say we should then
As I think the issues show that is this a very relevant feature.
Comment #60
joelpittetRebased #3616064: Provide a way to check specific links immediately and then closed it as a duplicate so we can finish this in one place. Thanks @codebymikey, I've ported the best parts over here and credited on this one.
The biggest change is that "Check" is now an entity operation, so it shows up in the operations dropbutton the Broken links report already has. That's a much better fit than a separate Views column: existing sites get it with no view changes at all, so I reverted adding the "Check link status" column to the default view.
@grevil that makes me wonder whether we need the "Check link status" Views field at all now. The operation covers the shipped reports, and a custom view can add the operations field to get the same link. Dropping it would slim the MR down a bit more: the field plugin, its
hook_views_data()and schema, and one test. Any objections, or did you have a use case for the separate column in mind?Also ported from @codebymikey's MR:
LinkCheckerService::checkMultiple()checks links in parallel, respectingcheck.connections_maxthe same way cron does.broken_links_reportand creates the action. Both do nothing if they've already been applied.drush linkchecker:check-link 12,34(aliaslcchl).A few tweaks on the way in:
@anybody I also added a "Check" button on the link edit form, which I took as the "details page" you mentioned in #58, since links don't have a separate view page.
Tests cover the operation, bulk check, edit form button,
checkMultiple(), the batch and both post_updates. I updated the issue summary to match.To try it locally, check out the branch, run
drush updb, then use "Check" or the bulk action on admin/reports/linkchecker.An LLM help with some of the tedious porting and collating of this. So a real world test would be appreciated before merging this. I did some very rough manual testings to see if the tests were covering things well enough.
Comment #61
joelpittet@grevil since the operation covers the shipped reports and a custom view can add the operations field for the same link, I went ahead and dropped the separate "Check link status" Views field to slim this down. Shout if you had a use case for it in mind and we can discuss.
Comment #62
grevil commented@joelpittet, good stuff!
I prefere the operations integration (looks way cleaner now). Good idea! The operation works as expected! RTCB regarding the new operation.
Comment #63
grevil commentedTwo things I adjusted:
- The submit handler of the "Check links" button built and deleted the whole query, only to see if there was anything to check and then the actual batch built and executed the query again.
- "checkMultiple" now checks the status of the links via the links hash (to not check the same link twice) and copies their status to each link entity (same behavior as the queue worker).
Otherwise all features implemented here seem to work great! I'd set this to "RTBC", but since I changed stuff myself, someone should take a look at the last two commits.
Comment #64
joelpittetOh thank you for stepping in to fix that multi(I will double check but imagine you’re likely right), I’ll do one last review and likely commit this (tomorrow).
Please 🙏 anyone else on this issue do a manual check on their projects would be appreciated!