Problem

On Drupal 7 every connector with its cron setting on scanned for new releases on cron and then parsed a set number of releases per run, the "Number of releases to look at once" setting of the connector. Only releases of enabled projects (status = 1) with a package were considered, oldest first.

On 3.0.x cron does not parse at all: l10n_server_cron() fills the l10n_server_parser queue and scans connectors, but the queue worker declares no cron key, so core cron never processes the queue. Parsing only happens through drush queue:run l10n_server_parser, drush l10n_server:parse or the admin batch. Around that, the Drupal 7 rules are lost too:

  • The queue selection ignores the project status and the download link, so releases of disabled projects and releases without a package are queued on every cron run, and a release queued on creation is parsed even when its project is disabled.
  • The connector's limit setting lost its meaning: it only makes the admin scan batch repeat a full scan that many times (which downloads the release list that many times for the drupal.org connector).
  • The admin "Parse" batch calls the parse handler without a release, which is a PHP error for the drupal.org connector, and stores no counts or parse time for the local connector.
  • drush l10n_server:parse uses its own copy of the parse loop with its own limit, re-parses already parsed releases by default and does not check the project status.

Proposed resolution

Cron parses like the Drupal 7 connectors: every connector with cron enabled scans, then parses up to its limit of unparsed releases of its enabled projects, oldest first. Every other way to parse does the same thing.

  • New ReleaseParser service with the parse-and-record step (project and connector enabled, parse, store counts and parse time), shared by cron, the queue worker, the parse batch and drush.
  • L10nServerReleaseStorage::getIdsToParse() selects the unparsed releases with a download link of enabled projects for a connector, with optional project and version filters; getIdsToQueue() shares the query, so the drush queue command applies the same rules.
  • l10n_server_cron() scans and parses per connector instead of filling the queue. The queue stays for drush queue:run, fed by release creation, and its worker also skips disabled projects.
  • The limit setting is labelled "Number of releases to parse at once" again, as on Drupal 7; the scan batch scans once; the parse batch parses the releases cron would take next.
  • drush l10n_server:parse parses through the service, takes the connector's limit unless --limit is given, and skips parsed releases like cron and the UI do (start over with a release to parse it again). The project argument and --release filter stay, --only-unparsed and --only-unqueued go.

Tests

Drupal 7 first: L10nDrupalFilesTestCase::testCronParse() runs cron against a package directory: nothing happens until the connector's cron setting is on, then cron scans and parses one release per run, and a disabled project's release is left alone until the project is enabled again. Ported as DrupalCronParseTest (kernel), which runs the cron service against the potx test package, checks the limit with two releases per run, the selection filters behind the drush command, and that a release without a package is left alone.

LLM disclosure

LLM was used to find, diagnose explain and fix this issue. With human review.

Comments

gábor hojtsy created an issue. See original summary.

  • be08936a committed on 7.x-1.x
    fix #3621583: Parse releases on cron like Drupal 7: a set number per...

  • efca07b2 committed on 3.0.x
    fix #3621583: Parse releases on cron like Drupal 7: a set number per...
gábor hojtsy’s picture

Status: Active » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.