Strings from test cases are parsed, they should not be

The localization server does not have a concept of prioritized strings vs. other strings, so to solve this that way would be quite a stretch.

Why do tests translate strings that do not otherwise show up on the Drupal UI as translated to begin with? Things like getInfo(), assert messages, etc. are not supposed to be translated. The coding standards at https://www.drupal.org/node/325974 for some reason do not explain the assert messages convention but it does the getInfo().

Can you list representative examples of those 500 - 2000 strings (that is a pretty wide range?).

Test cases should not have translatable strings which are not showing up in Drupal code itself (outside of tests). Implementing this feature on the localization server side is not the solution. Why are tests using such strings, care to answer my question?

This is a great example. Whatever it asserts would ALSO be t()-ed in Drupal itself, that is what is being tested after all. The assert message 'Translations found' however is not translated. Not seeing anything in this example or the other example you posted that is not t()-ing things that should already be t()-ed in Drupal's code.

$this->assertText(t('Updates for: @project', array('@project' => t('Drupal core'))), 'Translations found');

Looking at the concrete strings, these do seem to be not testing what Drupal actually generates:

$ git grep "Updates for:"
core/modules/locale/src/Tests/LocaleUpdateInterfaceTest.php:    $this->assertText(t('Updates for: @modules', array('@modules' => 'Locale test translate')), 'Translations available');
core/modules/locale/src/Tests/LocaleUpdateInterfaceTest.php:    $this->assertText(t('Updates for: @project', array('@project' => t('Drupal core'))), 'Translations found');
core/modules/locale/src/Tests/LocaleUpdateTest.php:    $this->assertText('Updates for: Contributed module one, Contributed module two, Custom module one, Locale test', 'Updates found');
core/modules/locale/templates/locale-translation-update-info.html.twig:    <span class="locale-translation-update__message">{% trans %}Updates for: {{ module_list }}{% endtrans %}</span>

The right source string to use in the test would be t('Updates for @module_list') NOT t('Updates for: @modules') or t('Updates for: @project'). The next one:

$ git grep "%title has been created"
core/modules/filter/src/Tests/FilterAdminTest.php:    $this->assertRaw(t('Basic page %title has been created.', array('%title' => $edit['title[0][value]'])), 'Filtered node created.');
core/modules/forum/src/Tests/ForumTest.php:      $this->assertNoRaw(t('@type %title has been created.', array('@type' => $type, '%title' => $title)), 'Forum topic was not created');
core/modules/forum/src/Tests/ForumTest.php:      $this->assertRaw(t('@type %title has been created.', array('@type' => $type, '%title' => $title)), 'Forum topic was created');
core/modules/node/src/NodeForm.php:      drupal_set_message(t('@type %title has been created.', $t_args));
core/modules/node/src/Tests/NodeCreationTest.php:    $this->assertRaw(t('@post %title has been created.', array('@post' => 'Basic page', '%title' => $edit['title[0][value]'])), 'Basic page created.');
core/modules/node/src/Tests/NodeCreationTest.php:    $this->assertRaw(t('@post %title has been created.', array('@post' => 'Basic page', '%title' => $edit['title[0][value]'])));
core/modules/system/src/Tests/Entity/EntityTranslationFormTest.php:    $this->assertRaw(t('Basic page %title has been created.', array('%title' => $edit['title[0][value]'])), 'Basic page created.');
core/modules/taxonomy/src/Tests/TermTest.php:    $this->assertRaw(t('@type %title has been created.', array('@type' => t('Article'), '%title' => $edit['title[0][value]'])), 'The node was created successfully.');

Where it tests t('@post %title has been created.') or t('Basic page %title has been created.') all of those are wrong and should be t('@type %title has been created.') based on the similar string that is actually output.

The only reason the wrong assertions pass is we don't run the tests translated :) Given the source strings are different, they would not pass with translations that may be different.

These need to be resolved in core, these are testing bugs.

For the RSS examples, they indeed come from test modules, I would argue the test module should not t() the source either, unless it needs to test the translation. For the last one, that is as your snippet shows too, comes from Drupal's node type list builder, so not test only either:

$ git grep "No content types available."
core/modules/node/src/NodeTypeListBuilder.php:    $build['table']['#empty'] = $this->t('No content types available. <a href=":link">Add content type</a>.', [
core/modules/node/src/Tests/NodeTypeTest.php:    $this->assertRaw(t('No content types available. <a href=":link">Add content type</a>.', [

@droplet: it would be a great meta issue to crowd-source :)

So I made up my mind because I heard this from several more people. Also apparently we already try to avoid parsing tests. We exclude the ..../tests/... directories, so test modules are not included. See https://localize.drupal.org/translate/projects/drupal/releases/454190 there are no such files parsed. However, there are 1550 files that are ..../Tests/.... (note case difference). Those are the actual tests. We should exclude those too probably. Not sure of the overall impact of them, but I made up my mind about making people work on those. Why would they. You are right. I was wrong.

Ok committed. Now deploying to drupal.org.

Ok this got rid of 496 strings. Not a small feat to avoid people working on things they will not need. Ideally tests would only ever use strings that exist on the frontend only, but its clear that tests also use test modules and strings from there, as well as several bogus strings.