Validate input and make failed jobs recoverable

Problem/Motivation

A review of the module found input that is accepted but breaks the job, and edge cases where the status of a job ends up wrong or a job cannot be recovered.

Input

1. The URL of the site accepts file:// and ftp://, while the additional start URLs require http or https.
2. HTTrack reads an exclude pattern without * or / as a command line option. The pattern Ox makes the job fail with exit code 255.

Status and resume attempts

3. An error before a new job has started, or before a waiting archive is being stored, sets the job to Error at once, without the resume attempts.
4. The crawl and the storage share one count of resume attempts, which is never reset. Two failed crawl attempts leave one attempt for storing the archive.
5. A job with the status Error cannot be run again. The only way is to create a new job.
6. A job that is cancelled while the error pages are handled or the archive is compressed can be set back to "Waiting" by the last save of the running process.
7. The storage step does not extend the processing lock. Copying a large archive to another file system can take longer than the lock timeout, and the job is then treated as abandoned.
8. The archive that waits to be stored is in the temporary directory. If it is gone when the job is stored, for example after a restart, the job fails after its resume attempts and cannot be recovered.

Proposed resolution

1. Require an http or https URL for the site.
2. Require exclude patterns to contain * or /, with an error message that shows how to write the pattern.
3. Treat an error in a new or waiting job as an error in its step, so that it gets the resume attempts.
4. Reset the count of resume attempts when the archive has been created, so that storing has its own attempts.
5. Add drush earkiv:retry ID and a Retry operation in the archive list for jobs with the status Error. A job whose archive is still waiting to be stored is stored again; other jobs start a new crawl.
6. Check whether the job was cancelled before the error pages are handled and before the last save, and stop if it was.
7. Move the archive with a rename when possible, and otherwise copy it in parts and extend the lock while it is copied.
8. If the waiting archive is missing when the job is stored, start a new crawl instead of failing.

Each fix is a separate commit in the merge request.

Remaining tasks

• Implement and test.
• Review.

User interface changes

A Retry operation for failed jobs in the archive list. Validation messages for the URL and the exclude patterns.

API changes

New Drush command earkiv:retry.

Data model changes

None.

AI-Generated: Yes (Used Claude Code to review the module and draft this issue.)