Problem/Motivation
The batch from LocaleFetch::buildUpdateBatch() is used in the installer, when installing modules and theme, when adding a new language, cron automatic translation updates (only when configured, not enabled by default) and when we have stale data about translation updates but need to update translations. So most of the time with locale :)
This adds 4 operations per project and language: version check, status check, download and import. The status check does a HEAD request to the translation server, and if the file was newer the download operation right after that GETs the same file. This separation is for operation reuse I believe, as there is a report page which checks status without download and the update form fetches without re-checking when the status is fresh — but in buildUpdateBatch() (used by the installer, optionally cron and the stale-status update path) the two always happen right after each other which is unfortunate. Network latency could make this really slow.
A Drupal CMS 2.x-dev install in German takes 47 projects, results in 188 batch operations out of which 98 do requests to ftp.drupal.org (50 HEAD + 48 GET). A GET with If-Modified-Since would do both in one request: ftp.drupal.org supports the conditional request.
While looking at this, it rudned out that English on the default server pattern is only flagged as unusable for the GET request but but the HEAD is still sent — that request should also be skipped. Included here for simplicity for now.
Steps to reproduce
Install Drupal CMS in German, check how many HTTP requests it takes.
Proposed resolution
- Refactor existing duplicated try/catch and redirect handling in
checkRemoteFileStatus()anddownloadTranslationSource()to a shared private helper. - Add a new
LocaleFileManager::fetchTranslationSource()which does a conditional GET returning aRemoteFileInfo. Added a newNotModified = 304enum case, plus included the saved file on success. - New
LocaleFetch::batchConditionalFetch()replacing the status check (HEAD) + download (GET) pair inbuildUpdateBatch()./li> - The status-only and fetch-only flows keep their existing operations, unchanged.
- English + default server pattern no longer makes any request, in both the new operation and
batchStatusCheck(). Custom server patterns for English are still checked.
Changes to batch and HTTP requests (Drupal CMS 2.x, German install, 47 projects)
| before | after | |
|---|---|---|
| Translation batch operations | 188 | 141 |
| Requests to ftp.drupal.org | 98 | 52 |
Note that the amount of data transferred does not change much, since the eliminated HEAD exchanges carry only headers, plus the same file bodies are downloaded either way. How much wall time this saves depends entirely on network latency: each removed request is a full round trip plus a TLS connection setup (every batch operation opens a fresh connection!). The halved request and connection load on the translation server applies everywhere, for every use of translation updates though.
Remaining tasks
Review.
User interface changes
None.
Introduced terminology
None.
API changes
Existing methods keep their signatures and behavior.
New public method on two services, one new enum case, one new property on RemoteFileInfo.
Data model changes
None.
Release notes snippet
Localization translation downloads in installation and updates do not run two separate requests to localize.drupal.org anymore (one HEAD, one GET). The new combined GET request makes translated installs and later translation updates faster in case of noticable network latency.
LLM disclosure
LLM was used to diagnose this problem, iterate on solutions and develop results. The results were reviewed and adjusted as a human though.
| Comment | File | Size | Author |
|---|
Issue fork drupal-3619842
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
gábor hojtsyComment #4
gábor hojtsyComment #5
drummEverything that would be good for updates.drupal.org in the related issues would also be good for translation downloads.
It would be good to add
'decode_content' => 'br'to the HTTP request while we are here.Comment #6
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.