Drupal Association members fund grants that make connections all over the world.
I'm pushing 530 urls in the queue and start running concurrent URL checks with background process calls. Thread limit is ONLY 4. CPU spikes up to 100%, machines becomes unresponsive and memory spikes to... 400+MB and grows very fast up to 1.5GB... after some time cron finishes and only 60 links are checked.
This is not what I expected and it's not usable :-(.
I'm posting latest linkchecker patch in.