It would be nice if we don't have to go click that button or run drush to get the existing content on the site into link checker. I'm thinking simply adding a cron job and queue that goes through the existing content and throws the items from _linkchecker_batch_import_nodes (and other scanners, into a queue.

Comments

generalredneck created an issue. See original summary.

generalredneck’s picture

Status: Active » Needs work
StatusFileSize
new3.64 KB

This only has nodes added to the queue. Comments, blocks and others will need to be added. I also added a dependency on Advanced queue out of preference more than anything. You can use drupal's default queue if you wish, but this allowed me to test the queue straight from drush, just making life easier.

Lastly I changed the _linkchecker_batch_import_nodes function to get all content that doesn't already have links seeing as this should only run when content either bypasses linkchecker for some reason or the module is first installed... or someone clears out the queue and doesn't let the batch run all the way.

generalredneck’s picture

May bad... stray dd on the db_select line... here's the fix.

hass’s picture

Category: Feature request » Support request
Status: Needs work » Fixed

Existing content is added without manual interaction. You do not have to press any buttons or run drush.

You may misunderstood how this module works.

generalredneck’s picture

Status: Fixed » Active

Would you be kind enough to go that extra step and enlighten me please? I don't see a node scan anywhere besides when you interact with the maintenance buttons on the admin pages or when you do a node Insert or Node update... You can see those here:

hook_node_insert
http://cgit.drupalcode.org/linkchecker/tree/linkchecker.module#n856

hook_node_update
http://cgit.drupalcode.org/linkchecker/tree/linkchecker.module#n871

and here:
http://take.ms/rINph

and to be further specific... here:
http://take.ms/VtNtF

So from my findings, if you didn't use drush or one of the batch processes only new content from the time you install the module will have the links added to the linkchecker_link table. The batch process runs on any time you change the settings and physically hit the button on that Admin Page, Maintenance Buttons, or Drush. This ignores Features or programmatical configuration which is best practice in the development world. And a batch process can hold up a push on a site that has say... 50,000 nodes.

I put up an example on simplytest.me to prove my point. https://dfs8f.ply.st/node/2

The code behind node/2 is the following: (Man I felt dirty writing this PHP into a node)

variable_set('linkchecker_scan_nodetypes', array('article' => 'article', 'page' => 'page'));
$result = db_select('linkchecker_link', 'll')
->fields('ll')
->execute()
->fetchAll();

var_dump($result);

I have one other piece of content and it looks like this:

This is existing content.
<a href='http://google.com'>Google</a>
<a href='http://doesnotexistinreality.org'>Does Not Exist In Reality</a>

Running Node 2 gives me: array(0) { }
After running cron, I still see array(0) { } (did this just for kicks)
However after running the batch (Specifically reanalyze content), I get:
array(2) { [0]=> object(stdClass)#50 (9) { ["lid"]=> string(1) "1" ["urlhash"]=> string(43) "qiI5wXYJsh66A0xWSvh48-7IzoPtDydoWX0rwv1OTaU" ["url"]=> string(17) "http://google.com" ["method"]=> string(4) "HEAD" ["code"]=> string(2) "-1" ["error"]=> NULL ["fail_count"]=> string(1) "0" ["last_checked"]=> string(1) "0" ["status"]=> string(1) "1" } [1]=> object(stdClass)#81 (9) { ["lid"]=> string(1) "2" ["urlhash"]=> string(43) "vC39WvNyeZ7fhwbDWQEQvffn1vqGzvCCRr0pO3F609k" ["url"]=> string(32) "http://doesnotexistinreality.org" ["method"]=> string(4) "HEAD" ["code"]=> string(2) "-1" ["error"]=> NULL ["fail_count"]=> string(1) "0" ["last_checked"]=> string(1) "0" ["status"]=> string(1) "1" } }

See what I mean? what professional in the right mind configures a production site by clicking a button? And what professional wouldn't want to automate a deployment and not have to wait for a batch process that reanalyzes the existing content?

hass’s picture

Status: Active » Fixed

There is a feature request at #2060243: Split linkchecker_scan_nodetypes as content type specific to support features better.

  1. You install the module
  2. You configure the module
  3. The batch process imports all links from content you configured (one time action).
  4. From now on new links are extracted if you save your content.
  5. Link checks are running in background via cron.

I have no plans to depened on other unppopular modules like advancedqueue. Depending on other modules is always a nightmare as it is not clear how motivated the maintainers of this modules are and if and when they port them to D8/D9/D10 just as one example. We are near end of D7 lifecycle and they only have a 1.0 beta3 that is over 1 year old. You do not like linkchecker module to depend on such modules.

Additionally there is #1882330: Replace Batch API with Queue API. You may search a queue first before posting new issues.

hass’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.