Replace error pages in the archive with a notice and list broken links

Problem/Motivation

When a page on the site links to a URL that returns an error, HTTrack saves the error response in the archive and points the link to it. On a Drupal 11 site with about 2,000 published pages, a full archive contained 52 to 72 copies of the site's 404 page.

• HTTrack does not rewrite links in error pages, so the menus and links on these pages point to the root of the file system and lead nowhere when the archive is opened from disk.
• Nothing in the archive tells the reader that the link was already broken when the site was archived. The only trace is the HTTrack cache file hts-cache/new.txt.
• Running HTTrack with -o0 (do not save error pages) is not enough: the links are still rewritten to the local file, which then does not exist.

Proposed resolution

After a successful crawl, before the archive is created, read hts-cache/new.txt and for every URL that returned HTTP 400 or higher:

• If HTTrack saved it as an HTML file, replace the file with a short notice page: the page did not exist when the site was archived, the HTTP status, the original URL and the date. Links to it keep working and lead to the notice.
• If HTTrack saved it as another file type (for example a missing PDF), delete the file, since it contains the HTML error page under a wrong extension.

Write earkiv-broken-links.csv to the root of the archive with one row per broken URL: HTTP status, URL and the page that links to it. This documents the broken links in the archive and gives editors a list to fix.

Remaining tasks

• Implement and test.
• Review.

User interface changes

None. The notice page text is translatable.

API changes

None.

Data model changes

None. Archives get a new file, earkiv-broken-links.csv.

AI-Generated: Yes (Used Claude Code to analyse a full archive, draft this issue and write the merge request.)