Replace field tokens by empty strings when there is no field value

I would say that it should be replaced by an empty string. That is what I would expect if I am using this token.

Makes sense to me too. I'll give Dave a chance to give his OK on this one if has a minute as it's related to #1614456: [current-page:query:?] should fall back to empty string, which I'm not sure if we should do.

patch #7 is not running for me: Drupal 8.1.1 and Token 8.x-1.x-dev and PHP 7.x

some patches using (e.g. in token.tokens.inc around line 1265)
($entity->get($field_name)
(e.g. "Chained tokens for field tokens")

and some others
($entity->get($name)
(e.g. this one and "Allow field tokens also for base fields")

i'am not sure what's right or wrong...

Since Issue#2621598 patch#7 does not apply anymore. Let's reroll (co-work with boogiewoogieman).

I've tried to add a test case for empty field value but failed. Could anyone help me, please? I'm fairly new to this.

You defined a non-existing property, that's not the same as an empty field at all.

Testing an empty field requires that you create an entity which has no value for the field and pass that in as data and test for a valid token.

Here's a patch that includes the code change and updates tests. Feedback welcome!

This patched test only validates the text field type as an empty string. Should we consider validating other field types?

I'm surprised this issue hasn't gained more attention. I'll speculate that most field token usage are for required field, and as a result, this issue isn't common.

I do wonder if this change gets committed; how it would impact users. I imagine there could be custom code out there that is stripping un-replaced tokens. I'm guessing the impact would vary by implementation. For example, unsetting an array key or removing an object property if detecting an un-replaced token would no longer work.

Changing back to needs review

After some discussion with teammates, I was pointed to Drupal core Token::replace() method's options parameter includes a boolean option, clear.

See https://git.drupalcode.org/project/drupal/-/blob/10.1.x/core/lib/Drupal/...

* - clear: A boolean flag indicating that tokens should be removed from the
* final text if no replacement value can be generated.

It looks like the Token module is using this approach in this UrlTest, https://git.drupalcode.org/project/token/-/blob/8.x-1.x/tests/src/Kernel..., but I don't think any where else.

I've come across a scenario where the token for an image media's file token is not being replaced when the optional (not required) media entity reference field does not have a value.

Token: [node:field_media:entity:field_image:entity:url]

I've also come across a few other related issues

• #3167731: Undocumented :clear functionality - Substitute token with empty value if value is null
• #1123266: Field token is not replaced if field value is empty

I tried to create a tests for the media entity file token replacement to demonstrate what I think would expected behavior, replacing the a token with empty string if it can not be evaluated/replaced.

Hello, from the previous MR it sometimes worked for me. However, when having URL fields it would fail. So instead of just letting it continue, I'm shorting it so that it replaces and then continues to next token.

I didn't do anything with the test, but maybe this will help other people. On my end it work with the general tests that the previous one was passing and fixed the ones it was failing.

verbatim as MR 66

I had to reroll the latest patch in order to get it to apply cleanly on Drupal 11.3.8.