Problem/Motivation

Symfony 3.4 provides a shortcut to inject all services tagged with a specific tag, which is a common need in some applications, so you don't have to write a compiler pass just for that.

However, when you use a tagged argument the YAML decoder needs to support 'custom tags'.
Which at the moment is not supported in Drupal.

There is more that can be done with custom tags since Symfony 3.3. to mention a few, here are some of the custom tags that are already supported by Symfony YAML component.

!!binary for storing binary data
!php/const: for referring to PHP constants.
It already supports also defining your own custom YAML tags (Custom tags are arbitrary strings that start with the ! character, such as the !my_tag value).
The above can only be possible if the Yaml::PARSE_CUSTOM_TAGS flag is supported in Drupal.

Proposed resolution

Update Drupal\Component\Serialization\YamlSymfony::decode and pass in the flag 'SymfonyYaml::PARSE_CUSTOM_TAGS'

Remaining tasks

User interface changes

None

API changes

No change.

Data model changes

None

Issue fork drupal-3108309

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

bonrita created an issue. See original summary.

larowlan’s picture

Status: Active » Postponed (maintainer needs more info)

Given we already support the 'service_collector' tag to avoid needing a compiler pass (which I think may have inspired the symfony feature) - do we need this?

Sample from core:

the collector:

cache_tags.invalidator:
    parent: container.trait
    class: Drupal\Core\Cache\CacheTagsInvalidator
    calls:
      - [setContainer, ['@service_container']]
    tags:
      - { name: service_collector, call: addInvalidator, tag: cache_tags_invalidator }

and a tagged service:

cache_tags.invalidator.checksum:
    class: Drupal\Core\Cache\DatabaseCacheTagsChecksum
    arguments: ['@database']
    tags:
      - { name: cache_tags_invalidator}

In this scenario, addInvalidator is called on Drupal\Core\Cache\CacheTagsInvalidator for every service tagged with cache_tags_invalidator

See \Drupal\Core\DependencyInjection\Compiler\TaggedHandlersPass

bonrita’s picture

Thanks, @larowlan for the info.
Indeed the above-mentioned example can already be done by Drupal using the service_collector tag. However, that is just part of the problem.
There is more that can be done with custom tags since Symfony 3.3. to mention a few, here are some of the custom tags that are already supported by Symfony YAML component.

  • !!binary for storing binary data
  • !php/const: for referring to PHP constants.

It already supports also defining your own custom YAML tags (Custom tags are arbitrary strings that start with the ! character, such as the !my_tag value).
The above can only be possible if the Yaml::PARSE_CUSTOM_TAGS flag is supported in Drupal.

bonrita’s picture

StatusFileSize
new2.24 KB
new3.05 KB
bonrita’s picture

Status: Postponed (maintainer needs more info) » Needs review
bonrita’s picture

Issue summary: View changes

The last submitted patch, 4: 3108309-4-tests.patch, failed testing. View results

bonrita’s picture

StatusFileSize
new3.07 KB
bonrita’s picture

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new1.67 KB

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

longwave’s picture

godotislate’s picture

if we implemented this then we could also use !tagged_iterator and friends from Symfony.

Along those lines, we could also use !service_closure to define services following this: #3108020: Support ServiceClosureArgument in \Drupal\Component\DependencyInjection\Dumper\OptimizedPhpArrayDumper::dumpValue. Though I have found that autowiring closures (using annotations) solves that since the arguments don't even need to be defined in YAML.

kim.pepper made their first commit to this issue’s fork.

kim.pepper’s picture

Version: 9.5.x-dev » 11.x-dev

kim.pepper’s picture

Status: Needs work » Needs review

Created a MR from the patch in #8

kim.pepper changed the visibility of the branch 3108309-support-yamlparsecustomtags-in to hidden.

longwave’s picture

Title: Support Yaml::PARSE_CUSTOM_TAGS in \Drupal\Component\Serial::decodeization\YamlSymfony:: » Support Yaml::PARSE_CUSTOM_TAGS in \Drupal\Component\Serialization\YamlSymfony::decode

Fixing title.

longwave’s picture

Status: Needs review » Needs work

Given we added constant parsing as well, can we add a test for that too?

What do we need to add to the PECL parser to support this?

longwave’s picture

Also, is there any additional overhead in adding this parsing? Is there a case where we might not want this flag to be enabled?

alexpott’s picture

@longwave I don't really think there is much overhead - given we would already be parsing the tag in Symfony and throwing an error. The question is really can we duplicate this functionality in PECL. And the problem is we cannot. Looking at the underlying PECL code (https://github.com/php/pecl-file_formats-yaml/blob/php7/parse.c#L641) there is no way to subscribe a callback to all tags. We would need to file an issue and get it into PECL yaml ... not fun. Maybe we could make it make it really simple to register new tags to a PECL yaml callback that produces TaggedValue objects.

longwave’s picture

@alexpott should we just force the Symfony parser in the DI YamlFileLoader, as that's currently the only place (I think) that we need this?

longwave’s picture

Further to #24 we already have #2951046: Allow parsing and writing PHP class constants and enums in YAML files to enable PARSE_CONSTANT which also works on PECL and has tests, so let's only work on custom tags here.

alexpott’s picture

Re #27 I was tempted to suggest that. I was reticent because service.yml files are one of the thing parsed on cold cache. But the more I think about it the more I think that this is a good approach for now. The capabilities of the YAML parser and container configuration are quite closely related so this does not feel like a bad decision - also we use FileCache here so potentially this has no impact on cold cache as that can persist across a cache rebuild.

kim.pepper’s picture

longwave’s picture

Status: Needs review » Needs work
kim.pepper’s picture

Status: Needs work » Needs review
longwave’s picture

Status: Needs review » Reviewed & tested by the community

Does what it says, has test coverage. Not even sure we should bother with catching the exception in the test, just let it fail if the exception is thrown, but the code is already written so let's leave it as-is.

alexpott’s picture

I don't think we should do this until we've removed support for PECL yaml in #3205480: Drop PECL YAML library support in favor of only Symfony YAML - this intentionally makes the pecl and symfony parsers different and I don't think we should do that.

longwave’s picture

Status: Reviewed & tested by the community » Postponed

Let's use this status, then.

wim leers’s picture

Title: Support Yaml::PARSE_CUSTOM_TAGS in \Drupal\Component\Serialization\YamlSymfony::decode » [PP-1] Support Yaml::PARSE_CUSTOM_TAGS in \Drupal\Component\Serialization\YamlSymfony::decode
wim leers’s picture

Title: [PP-1] Support Yaml::PARSE_CUSTOM_TAGS in \Drupal\Component\Serialization\YamlSymfony::decode » Support Yaml::PARSE_CUSTOM_TAGS in \Drupal\Component\Serialization\YamlSymfony::decode
Status: Postponed » Active
Issue tags: +Needs reroll

andypost made their first commit to this issue’s fork.

andypost’s picture

Status: Active » Needs review
Issue tags: -Needs reroll

rebased

longwave’s picture

Status: Needs review » Reviewed & tested by the community

This is just adding the flag and some tests, the dependent issue landed, back to RTBC as per #33.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed a91f4f4e5b to 11.x and c6a3077802 to 10.3.x. Thanks!

  • alexpott committed c6a30778 on 10.3.x
    Issue #3108309 by kim.pepper, bonrita, alexpott, andypost, longwave:...

  • alexpott committed a91f4f4e on 11.x
    Issue #3108309 by kim.pepper, bonrita, alexpott, andypost, longwave:...
kim.pepper’s picture

Status: Fixed » Needs work

Unfortunately, this commit came after #3205480: Drop PECL YAML library support in favor of only Symfony YAML which means we added the flag to the deprecated \Drupal\Component\Serialization\YamlSymfony::decode() method.

We need to make the change to \Drupal\Component\Serialization\Yaml::decode() now.

#3414208: Add support for tagged_iterator to YamlFileLoader is doing that already, so I don't know if we need a separate dedicated issue to do that.

alexpott’s picture

@kim.pepper whoops nice spot and yeah sure let's do it in that issue.

wim leers’s picture

Status: Needs work » Fixed

Looks like @kim.pepper did that as of ~14 hours ago: https://git.drupalcode.org/project/drupal/-/merge_requests/6134/diffs?co...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

kim.pepper’s picture