earkiv 1.0.0-beta1

The first release of E-Arkiv, which archives a Drupal site as a static copy made with HTTrack, for example to meet the archiving rules for Swedish public authorities. The module requires Drupal 11.2 or 12 and the command line tools httrack, tar and gzip on the server.

Highlights

• Fast and complete crawls with HTTrack: up to 128 simultaneous connections, start URLs from sitemap.xml, extra start URLs and all published content, exclude patterns, and alternative host names such as the production host.
• Error pages are replaced with a notice, and the archive lists broken links, pages that no other page links to and published pages that return an error.
• Every archive has a start page, a manifest with the settings and versions used, SHA-256 checksums of all files, and a checksum of the archive file next to it.
• Drush commands to create, process, retry, cancel and follow jobs, and schedules with recurrence rules that create jobs.
• Robust jobs: resume after interruptions, stop leftover HTTrack processes, fail clearly when a crawl fetches nothing, and clean up all files when a job is deleted.
• Kernel tests and GitLab CI against Drupal 11.2, the current and previous minor versions, and Drupal 12.

Upgrading from 1.0.x-dev

Run drush updatedb. The post update functions set the MIME type of existing archives, process the default plugin settings, let the author fields reference users, remove a storage schema handler that pointed to a missing class, and add the checksum field. Drupal 11.1 is no longer supported.

Issues in this release

• #3636404 Archiving is slow and the archive can be incomplete
• #3636555 Add start URLs and sitemap.xml as crawl entry points
• #3636583 Replace error pages in the archive with a notice and list broken links
• #3636584 Status report never warns when HTTrack, tar or gzip are missing
• #3636586 Improve feedback during the archiving run
• #3636593 Add Drush commands to create and cancel archive jobs
• #3636718 Treat alternative host names as the same site
• #3636813 Archive published pages that no other page links to
• #3636834 Archive links in the archive list open a new tab instead of downloading the file
• #3636840 Routes require a permission that does not exist
• #3636844 Allow up to 128 connections and recommend a value for production servers
• #3636849 Fix five bugs that can give a broken or incomplete archive
• #3636905 List published pages that return an error in earkiv-published-errors.csv
• #3636912 Validate input and make failed jobs recoverable
• #3636927 Fix the entity definitions and validate schedule rules
• #3636922 Create archive jobs from active schedules
• #3636938 Add a manifest, checksums and a start page to the archive
• #3636948 Fix the coding standards and check merge requests with GitLab CI
• #3637005 Make the GitLab CI jobs pass
• #3637010 Make the module compatible with Drupal 12
• #3637050 Stop the running time when the archive has been created