Localization update feature logs 404 errors too aggressively

Maybe it's because the module has no translatable strings.

This module has nothing to do with language updates.

Moving to Core issue queue.

I confirm that this error occurs when there are no translation for a module available. There are (currently) no translations for Twig Tweaks in Korean. This will typically happen with languages that only have a small amount of translations.

Localization Update module has a facility to exclude modules from being translated. Perhaps this should be made available for Drupal 8 too.

The problem we have with this is that it spams up our watchdog for each custom and contrib module without translations, because it prints the error message (notice) on every cron run, regardless of update check frequency.

I found this little gem in the locale_translation_batch_status_check function that runs before the update files are downloaded:

// @todo What to do with when the file is not found (404)? To prevent
//   re-checking within the TTL (1day, 1week) we can set a last_checked
//   timestamp or cache the result.

This causes a LOT of noise and makes monitoring the site's health unnecessarily difficult.

Here is a patch that adds an option to completly prevent the notice in logs. While it is not the optimal version where the notice is omitted for some period of time, it is a solution to the problem. To know what files couldn't be loaded you can enable the option and run the cron.

sorry forgot to change status

Thank you very much, I think this is a good option and the current situation is messing up logs. RTBC +1! Let's please have some more feedback.

I'll now RTBC this because it is a working solution and better than the current situation. If someone from the core developers has a different way to go, please reset the status. Otherwise let's get this forward. It's spamming our logs...

This seems to be a very specific feature for a very specific problem you are having and looks like a random option on the UI. Are other logs of the module not so noisy?

Why is the module checking the nonexistent file on all crons even if the frequency of checking is not set up to happen that often? The solution proposed here sounds like would mask that larger question that may result in other wastes of resources.

Also retitling for accuracy about the problem.

Are other logs of the module not so noisy?

Other log messages of this module rarely ever show up. In fact the only other log that is somewhat similar noisy is cron with the message "Cron run completed.". And for X custom modules or untranslated modules, you get X-times more messages. Sscores of pages we host have over 90% of the log messages only this message.
In the module is this comment:

File not found occurs when a translation file is not yet available at the translation server. But also if a custom module or custom theme does not define the location of a translation file. By default the file is checked at the translation server, but it will not be found there.

As I understand it, it is not realy seen as an error which makes it even more unnecessary of a flood of logs.

This seems to be a very specific feature for a very specific problem you are having and looks like a random option on the UI.

You are right, except I don't think only a few people having this problem. It is managable to ignore this, but the more custom modules are used the noisier the module. Unfortunally I didn't found a better place for this.

Why is the module checking the nonexistent file on all crons even if the frequency of checking is not set up to happen that often?

Currently it only saves the time of the last check if no error ocurred. As the module says:

// @todo What to do with when the file is not found (404)? To prevent
//   re-checking within the TTL (1day, 1week) we can set a last_checked
//   timestamp or cache the result.

I'll look into using the set time limit on 404 errors, also to cut down on request while cron run.
And maybe an option to aggregate the messages into a single message, stating what modules got a 404-error.

Do you think that would be a more fitting solution?

I think it's worth metioning that, if you have 10 custom modules and the crun runs every 10 minutes (and we have also systems where it runs every 5) this alone produces 1,440 log messages per day.

Any progress on this? I'd like to confirm my +1 on RTBC...

Looks like there are various possibilities that themselves would cut down on logging noise without adding a one-off configuration option, eg. don't even try to find translation files for custom modules (if we can identify custom modules vs. contrib modules) or update the checking date even if the file was not found, so it is not checked too often.

Typo in title

Adding tags

I've added a patch that addresses the "What to do with when the file is not found (404)?" comment and prevents repeated checking of translations until the update timer expires. I don't think this is really the final solution we're looking for, but it helps reduce the log noise.

And maybe an option to aggregate the messages into a single message, stating what modules got a 404-error.

I quite like this suggestion by @ben-buske in #16, a single message per cron run would help a lot and keep the information visible. Something listing projects that completely failed with 404 and projects for which only some languages failed, e.g. "Translation file not found for myproject, myotherproject, someproject (de, fr).".
A starting point might be to set $failure = TRUE; in case of 404 as well, and somehow collectively handle that in Drupal\locale\Plugin\QueueWorker\LocaleTranslation.

Edit: You should be able to use both #9 (offers configuration to suppress messages) and #25 (prevents requesting files every cron, regardless of TTL) together.

Besides log messages, this also makes the cron longer because of the frequent HTTP requests, so caching the 404 helps with that as well.

As logs are still being filled up with this, I'd vote for a commit of #25 by ckaotik to mitigate the problem. I personally like #9, but that's not my choice. So after using #25 and #9 for over two years now, I'll mark #25 RTBC and hope we can proceed to discuss #9 to get this whole issue solved soon.

Thanks a lot, all!

This needs an Issue Summary update to outline the various solutions discussed, and patches made for (#9), and why #25 what chosen. In #19 @Gábor Hojtsy mentions other changes that could be made - should followups be made to discuss those options?

The 8.9.x patch is being tested every two days - that needs to be changed to 9.4.x

This is also tagged for tests but there are none in the latest patch.

Therefor setting to NW.

Moving to PNMI based on #31

I don't understand why this is postponed based on my comment in #31. In that comment I think I outlined the work that needs to be done here next. I am setting back to needs works.

I like where this is going. But why are we so worried about removing these noisy log messages? I've got a 40+ language website that we have a full install profile test suite setup. For me, the issue is deeper. Because we do a foreach on each module in each language, testing and installing the site from configuration takes a _really_ long time. In addition to logging to drush or watchdog, it would be nice to use async guzzle and call each of the URLs in a shorter time frame. It would speed up downloads. I think we'd need to concern ourselves with some type of poor mans throttle so we don't send off 1000 URL requests, or even 100. That higher volume of traffic might cause performance issues on the localization servers.

Sadly this needs another reroll for Drupal 10.2. Doesn't apply anymore.

Created MR for 11.x

#31 mentions an issue summary update which is still needed.

Also test coverage.