Problem/Motivation

Working on demo_umami (Out of the box demo core profile) many strings (from fields, views, etc) are missing from https://localize.drupal.org
These fields are available for translation in the UI of the Drupal site which that profile was installed on.

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

shaal created an issue. See original summary.

shaal’s picture

@gabor said:

The schema used is based on the dependencies identified and those don't seem to be identified here well.
_potx parse yaml file is where the info dependencies are stored for parsing them for config schema
it takes the dependencies array but not the install array which install profiles use.

and suggested creating this patch.

This patch bring many more strings from demo_umami, but there are many still missing.
For an example, the field label "Ingredients" in the Recipe content type.

When I hacked the module, by adding to /modules/contrib/potx/yaml_translation_patterns.yml:

  - matches: 'config/install/*.yml'
    translatable_keys:
      - label

running drush potx --folder=/var/www/html/core/profiles/demo_umami/config/install DID extract the "Ingredients" field label and many more.

shaal’s picture

I added the hack to this patch as well. It seems to be adding exactly the missing fields from demo_umami core profile when running `drush potx`.

It now finds these missing strings that were not there before:
> msgid "Administrator"
> msgid "Banner image"
> msgid "Block the selected user(s)"
> msgid "Cancel the selected user account(s)"
> msgid "Content Link"
> msgid "Cooking time"
> msgid "Copyright"
> msgid "Difficulty"
> msgid "Disclaimer"
> msgid "Easy"
> msgid "Editor"
> msgid "Find out more link"
> msgid "German"
> msgid "Hard"
> msgid "Ingredients"
> msgid "Make content sticky"
> msgid "Make content unsticky"
> msgid "Media Library thumbnail (220x220)"
> msgid "Name (A-Z)"
> msgid "Name (Z-A)"
> msgid "Number of servings"
> msgid "Overridden block the selected user(s)"
> msgid "Preparation time"
> msgid "Promo image"
> msgid "Promo text"
> msgid "Promo title"
> msgid "Promote content to front page"
> msgid "Publish content"
> msgid "Recipe instruction"
> msgid "Remove content from front page"
> msgid "Unblock the selected user(s)"
> msgid "Unpublish content"
> msgid "above"
> msgid "inline"
> msgid "visually_hidden"

Gábor Hojtsy’s picture

We need to ignore the hack, that is not going to help. Here are the config files found with #2 that are otherwise not found:

core/profiles/demo_umami/config/install/views.view.articles_aside.yml
core/profiles/demo_umami/config/install/views.view.featured_articles.yml
core/profiles/demo_umami/config/install/views.view.frontpage.yml
core/profiles/demo_umami/config/install/views.view.promoted_items.yml
core/profiles/demo_umami/config/install/views.view.recipe_collections.yml
core/profiles/demo_umami/config/install/views.view.recipes.yml
core/profiles/demo_umami/config/install/views.view.taxonomy_term.yml
core/profiles/demo_umami/config/install/workflows.workflow.editorial.yml
core/profiles/demo_umami/config/install/block.block.seven_page_title.yml
core/profiles/demo_umami/config/install/block.block.umami_page_title.yml
core/profiles/demo_umami/config/install/block.block.seven_local_actions.yml
core/profiles/demo_umami/config/install/block.block.umami_local_tasks.yml
core/profiles/demo_umami/config/install/media.type.image.yml
core/profiles/demo_umami/config/install/block.block.umami_search.yml
core/profiles/demo_umami/config/install/media.type.file.yml
core/profiles/demo_umami/config/install/media.type.audio.yml
core/profiles/demo_umami/config/install/media.type.video.yml
core/profiles/demo_umami/config/install/block.block.help.yml
core/profiles/demo_umami/config/install/block.block.seven_help.yml
core/profiles/demo_umami/config/install/media.type.remote_video.yml
core/profiles/demo_umami/config/install/block.block.umami_branding.yml
core/profiles/demo_umami/config/install/block.block.breadcrumbs.yml
core/profiles/demo_umami/config/install/block.block.seven_breadcrumbs.yml
core/profiles/demo_umami/config/install/block.block.seven_content.yml
core/profiles/demo_umami/config/install/block.block.umami_content.yml
core/profiles/demo_umami/config/install/block.block.umami_main_menu.yml
core/profiles/demo_umami/config/install/block.block.seven_messages.yml
core/profiles/demo_umami/config/install/block.block.umami_messages.yml
core/profiles/demo_umami/config/install/block_content.type.basic.yml
core/profiles/demo_umami/config/install/node.type.article.yml
core/profiles/demo_umami/config/install/node.type.page.yml
core/profiles/demo_umami/config/install/block.block.seven_primary_local_tasks.yml
core/profiles/demo_umami/config/install/block.block.seven_secondary_local_tasks.yml
core/profiles/demo_umami/config/install/block.block.seven_login.yml
core/profiles/demo_umami/config/install/taxonomy.vocabulary.tags.yml
core/profiles/demo_umami/config/install/contact.form.feedback.yml
core/profiles/demo_umami/config/install/block.block.umami_account_menu.yml

core/profiles/minimal/config/install/block.block.stark_page_title.yml
core/profiles/minimal/config/install/block.block.stark_local_actions.yml
core/profiles/minimal/config/install/block.block.stark_local_tasks.yml
core/profiles/minimal/config/install/block.block.stark_tools.yml
core/profiles/minimal/config/install/block.block.stark_admin.yml
core/profiles/minimal/config/install/block.block.stark_branding.yml
core/profiles/minimal/config/install/block.block.stark_login.yml
core/profiles/minimal/config/install/block.block.stark_messages.yml

core/profiles/standard/config/install/block.block.bartik_page_title.yml
core/profiles/standard/config/install/block.block.seven_page_title.yml
core/profiles/standard/config/install/block.block.bartik_local_actions.yml
core/profiles/standard/config/install/block.block.seven_local_actions.yml
core/profiles/standard/config/install/block.block.bartik_local_tasks.yml
core/profiles/standard/config/install/block.block.bartik_tools.yml
core/profiles/standard/config/install/comment.type.comment.yml
core/profiles/standard/config/install/node.type.article.yml
core/profiles/standard/config/install/node.type.page.yml
core/profiles/standard/config/install/block.block.bartik_messages.yml
core/profiles/standard/config/install/block.block.seven_messages.yml
core/profiles/standard/config/install/block_content.type.basic.yml
core/profiles/standard/config/install/block.block.bartik_powered.yml
core/profiles/standard/config/install/block.block.seven_primary_local_tasks.yml
core/profiles/standard/config/install/block.block.seven_secondary_local_tasks.yml
core/profiles/standard/config/install/block.block.seven_login.yml
core/profiles/standard/config/install/taxonomy.vocabulary.tags.yml
core/profiles/standard/config/install/contact.form.feedback.yml
core/profiles/standard/config/install/block.block.bartik_search.yml
core/profiles/standard/config/install/block.block.bartik_help.yml
core/profiles/standard/config/install/block.block.seven_help.yml
core/profiles/standard/config/install/block.block.bartik_branding.yml
core/profiles/standard/config/install/block.block.bartik_breadcrumbs.yml
core/profiles/standard/config/install/block.block.seven_breadcrumbs.yml
core/profiles/standard/config/install/block.block.bartik_content.yml
core/profiles/standard/config/install/block.block.seven_content.yml
core/profiles/standard/config/install/block.block.bartik_account_menu.yml
core/profiles/standard/config/install/block.block.bartik_main_menu.yml

None of the field files seem to be found in install profiles still. That said, other modules, eg. book and forum ship with optional fields and those are found.

core/modules/book/config/optional/field.field.node.book.body.yml
core/modules/forum/config/optional/field.field.node.forum.body.yml

etc.

Gábor Hojtsy’s picture

Here is a debugging patch. It shows the extent of the problem is much wider :(

When run on core I get this:

Missing schemas:
core/modules/block_content/config/install/field.storage.block_content.body.yml
Missing schemas:
core/modules/comment/config/install/field.storage.comment.comment_body.yml
core/modules/comment/config/install/system.action.comment_delete_action.yml
core/modules/comment/config/install/system.action.comment_publish_action.yml
core/modules/comment/config/install/system.action.comment_save_action.yml
core/modules/comment/config/install/system.action.comment_unpublish_action.yml
Missing schemas:
core/modules/media_library/config/install/views.view.media_library.yml
Missing schemas:
core/modules/node/config/install/field.storage.node.body.yml
core/modules/node/config/install/system.action.node_delete_action.yml
core/modules/node/config/install/system.action.node_make_sticky_action.yml
core/modules/node/config/install/system.action.node_make_unsticky_action.yml
core/modules/node/config/install/system.action.node_promote_action.yml
core/modules/node/config/install/system.action.node_publish_action.yml
core/modules/node/config/install/system.action.node_save_action.yml
core/modules/node/config/install/system.action.node_unpromote_action.yml
core/modules/node/config/install/system.action.node_unpublish_action.yml
Missing schemas:
core/modules/user/config/install/system.action.user_block_user_action.yml
core/modules/user/config/install/system.action.user_cancel_user_action.yml
core/modules/user/config/install/system.action.user_unblock_user_action.yml
Missing schemas:
core/profiles/demo_umami/config/install/field.field.block_content.banner_block.field_banner_image.yml
core/profiles/demo_umami/config/install/field.field.block_content.banner_block.field_content_link.yml
core/profiles/demo_umami/config/install/field.field.block_content.banner_block.field_summary.yml
core/profiles/demo_umami/config/install/field.field.block_content.banner_block.field_title.yml
core/profiles/demo_umami/config/install/field.field.block_content.basic.body.yml
core/profiles/demo_umami/config/install/field.field.block_content.disclaimer_block.field_copyright.yml
core/profiles/demo_umami/config/install/field.field.block_content.disclaimer_block.field_disclaimer.yml
core/profiles/demo_umami/config/install/field.field.block_content.footer_promo_block.field_content_link.yml
core/profiles/demo_umami/config/install/field.field.block_content.footer_promo_block.field_promo_image.yml
core/profiles/demo_umami/config/install/field.field.block_content.footer_promo_block.field_summary.yml
core/profiles/demo_umami/config/install/field.field.block_content.footer_promo_block.field_title.yml
core/profiles/demo_umami/config/install/field.field.media.audio.field_media_audio_file.yml
core/profiles/demo_umami/config/install/field.field.media.file.field_media_file.yml
core/profiles/demo_umami/config/install/field.field.media.image.field_media_image.yml
core/profiles/demo_umami/config/install/field.field.media.remote_video.field_media_oembed_video.yml
core/profiles/demo_umami/config/install/field.field.media.video.field_media_video_file.yml
core/profiles/demo_umami/config/install/field.field.node.article.body.yml
core/profiles/demo_umami/config/install/field.field.node.article.field_image.yml
core/profiles/demo_umami/config/install/field.field.node.article.field_tags.yml
core/profiles/demo_umami/config/install/field.field.node.page.body.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_cooking_time.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_difficulty.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_image.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_ingredients.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_number_of_servings.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_preparation_time.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_recipe_category.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_recipe_instruction.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_summary.yml
core/profiles/demo_umami/config/install/field.field.node.recipe.field_tags.yml
core/profiles/demo_umami/config/install/field.field.user.user.user_picture.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_banner_image.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_content_link.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_copyright.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_disclaimer.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_promo_image.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_summary.yml
core/profiles/demo_umami/config/install/field.storage.block_content.field_title.yml
core/profiles/demo_umami/config/install/field.storage.media.field_media_audio_file.yml
core/profiles/demo_umami/config/install/field.storage.media.field_media_file.yml
core/profiles/demo_umami/config/install/field.storage.media.field_media_image.yml
core/profiles/demo_umami/config/install/field.storage.media.field_media_oembed_video.yml
core/profiles/demo_umami/config/install/field.storage.media.field_media_video_file.yml
core/profiles/demo_umami/config/install/field.storage.node.field_cooking_time.yml
core/profiles/demo_umami/config/install/field.storage.node.field_difficulty.yml
core/profiles/demo_umami/config/install/field.storage.node.field_image.yml
core/profiles/demo_umami/config/install/field.storage.node.field_ingredients.yml
core/profiles/demo_umami/config/install/field.storage.node.field_number_of_servings.yml
core/profiles/demo_umami/config/install/field.storage.node.field_preparation_time.yml
core/profiles/demo_umami/config/install/field.storage.node.field_recipe_category.yml
core/profiles/demo_umami/config/install/field.storage.node.field_recipe_instruction.yml
core/profiles/demo_umami/config/install/field.storage.node.field_summary.yml
core/profiles/demo_umami/config/install/field.storage.node.field_tags.yml
core/profiles/demo_umami/config/install/field.storage.user.user_picture.yml
core/profiles/demo_umami/config/install/filter.format.basic_html.yml
core/profiles/demo_umami/config/install/filter.format.full_html.yml
core/profiles/demo_umami/config/install/filter.format.restricted_html.yml
core/profiles/demo_umami/config/install/system.site.yml
core/profiles/demo_umami/config/install/system.theme.yml
core/profiles/demo_umami/config/install/user.role.administrator.yml
core/profiles/demo_umami/config/install/user.role.anonymous.yml
core/profiles/demo_umami/config/install/user.role.authenticated.yml
core/profiles/demo_umami/config/install/user.role.author.yml
core/profiles/demo_umami/config/install/user.role.editor.yml
core/profiles/demo_umami/config/install/user.settings.yml
Missing schemas:
core/profiles/minimal/config/install/system.theme.global.yml
core/profiles/minimal/config/install/user.settings.yml
Missing schemas:
core/profiles/standard/config/install/field.field.block_content.basic.body.yml
core/profiles/standard/config/install/field.field.comment.comment.comment_body.yml
core/profiles/standard/config/install/field.field.node.article.body.yml
core/profiles/standard/config/install/field.field.node.article.comment.yml
core/profiles/standard/config/install/field.field.node.article.field_image.yml
core/profiles/standard/config/install/field.field.node.article.field_tags.yml
core/profiles/standard/config/install/field.field.node.page.body.yml
core/profiles/standard/config/install/field.field.user.user.user_picture.yml
core/profiles/standard/config/install/field.storage.node.comment.yml
core/profiles/standard/config/install/field.storage.node.field_image.yml
core/profiles/standard/config/install/field.storage.node.field_tags.yml
core/profiles/standard/config/install/field.storage.user.user_picture.yml
core/profiles/standard/config/install/filter.format.basic_html.yml
core/profiles/standard/config/install/filter.format.full_html.yml
core/profiles/standard/config/install/filter.format.restricted_html.yml
core/profiles/standard/config/install/system.cron.yml
core/profiles/standard/config/install/system.theme.yml
core/profiles/standard/config/install/user.role.administrator.yml
Missing schemas:
core/profiles/testing/config/install/system.theme.yml
Missing schemas:
core/profiles/testing_config_overrides/config/install/system.action.user_block_user_action.yml
core/profiles/testing_config_overrides/config/install/system.cron.yml
Missing schemas:
core/profiles/testing_multilingual/config/install/language.entity.de.yml
core/profiles/testing_multilingual/config/install/language.entity.es.yml
Missing schemas:
core/profiles/testing_multilingual_with_english/config/install/language.entity.de.yml
core/profiles/testing_multilingual_with_english/config/install/language.entity.es.yml

My suspicion is this is an issue with dependency management in how config schema is read since core finds these along the same core schemas, so it must be how we read the config schemas.

Gábor Hojtsy’s picture

FileSize
1.89 KB

Above results produced with this patch.

Gábor Hojtsy’s picture

FileSize
1.08 KB

Yeah the drupal: namespace prefix for dependencies was what broke potx's dependency parsing. This founds schema for all config files, except

Missing schemas:
core/profiles/testing/config/install/system.theme.yml
Missing schemas:
core/profiles/testing_config_overrides/config/install/system.cron.yml

But these two from tests I would not expect to have schema.

Ideally this would come with some test coverage expansion but the fix we have at least.

shaal’s picture

Status: Needs review » Reviewed & tested by the community

All missing strings of demo_umami that are listed in #3, and many many more (from views, etc.) are now available in general.pot file, after applying patch #7.

Gábor Hojtsy’s picture

Title: Many strings from demo_umami core profile are not being extracted » Profile dependencies and composer style dependencies are not found in config parsing

Retitle for actual problem.

Gábor Hojtsy’s picture

FileSize
1.07 KB

Looked into providing test coverage but the dependency resolution of config and modules for config does not seem to be actually tested AT ALL. So should have some general test coverage for that first.

Also looking at https://www.drupal.org/docs/8/creating-custom-modules/let-drupal-8-know-... and the apparently existing code at #2983584: Comprehensive solution for potx config project:module format dependencies and #3014336: Handle the project:module format for dependencies to extract translatable strings from config: the l10n_server part., there are more types of qualified project names that would need to be parsed so slightly adjusted. Still runs the same on core.

  • Gábor Hojtsy committed 45fad79 on 8.x-1.x
    Issue #3042842 by Gábor Hojtsy, shaal: Profile dependencies and composer...
Gábor Hojtsy’s picture

Version: 8.x-1.x-dev » 7.x-3.x-dev
Status: Reviewed & tested by the community » Needs review
FileSize
1.08 KB

Rerolled it for 7.x-3.x too. Let's run tests on this as well, so we can roll it out on l.d.o.

  • Gábor Hojtsy committed 4282c2f on 7.x-3.x
    Issue #3042842 by Gábor Hojtsy, shaal: Profile dependencies and composer...
Gábor Hojtsy’s picture

Status: Needs review » Fixed
Gábor Hojtsy’s picture

@drumm rolled this out on l.d.o just now.

jhodgdon’s picture

I think 8.6.x needs to be reparsed for this, or maybe it is still in progress?

Gábor Hojtsy’s picture

I sent the latest 8.6 release for reparsing, not going to send all of them as I don't think that is helpful.

Gábor Hojtsy’s picture

Was 9080 strings before reparsing, came back with 9178 strings. Looks good to me :)

jhodgdon’s picture

Woot! This has been bothering me since I first made translation screenshots of the User Guide, back in the 8.2 days. Sorry it took me so long to file an issue... or maybe I did make one... let's see. Oh yeah:
#2806065: Theme region names are not making it into PO database
I'll mark that as a duplicate of this issue. Doh!

jhodgdon’s picture

Oops that is a duplciate of a different issue, sorry.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.