Provide array of primitive property values on multivalue field

Hey das-peter!

Open to add this, but considering the complexity of the field token code, I think this should have test coverage, would expect it shouldn't be too hard to add that to existing test coverage and would also serve as a usage example.

Hi @das-peter and @Berdir,

thank you very much for your work on this. I'd like to discuss the expected results here for some examples because once shipped this will never be easily changed again and it's a HUGE further step forwards for token to allow joining multi-value entity reference values!

So please let's add all examples with a behaviour description to the issue summary so we can then write tests against them. Would you be interested to help?

Writing these examples and comparing / validating them against existing logic for example for menu path etc. is also possible for non developers!

Here are my suggestions for multi-value fields:

General

Use * as delta iterator

My basic suggestion is to add "*" as delta iterator for multi value fields. Only multi value items have a delta so that would be safe and short to define * as "0-infinite" delta in ascending order.

[node:multivalued_field:*:{Everything like with delta}]
The difference to an explicit delta number is, that we have to join the result values. Because we iterate over the delta, we hopefully shouldn't have to care much about the concrete results but simply join each delta result.

As a fist step I'll add a discussion results part to the issue summary. Feel free to add examples and improve it. Also the issue summary should be updated according to #2 where @Berdir wrote he's interested.

I'm very excited about this imporant improvement!

PS: If the issue author thinks this is too far away from his original issue, I can of course create a new issue.
Otherwise we should align the title to something like "Provide iteration on multi value field deltas". But I guess a native speaker will be able to improve that ;)

PPS: If we form a small group together with @Berdir to push things forward here, I guess we can have fast and well tested results and prevent this from being an issue for years ;) Interested?

I altered the patch to work with release 8.x-1.8.

Wrong file naming, new file added.

Updated patch (just one line of diff context) to work with latest commit.

+++ b/token.tokens.inc
@@ -1679,6 +1702,29 @@ function field_tokens($type, $tokens, array $data, array $options, BubbleableMet
+                $replacements[$original] = token_render_array($field_data_arr, $options + ['join' => '']);

This part doesn't make too much sense because of the 'join' => '' part. This renders the values without any separator, and there is no chance to explode it. Or did I miss something?

Yes, it makes sense to have values comma-separated at least.

Updated join with comma

Patch makes some CS clean-up

Needs work for tests

+++ b/token.tokens.inc
@@ -1778,7 +1803,30 @@ function field_tokens($type, $tokens, array $data, array $options, BubbleableMet
+                $replacements[$original] = token_render_array($field_data_arr, $options + ['join' => ',']);

#16 Updated join with comma

From the OT summary TODO list I see the idea to provide options for a choice of common separators, which makes perfectly sense. So, hard coded join with comma should be rather extended by a join with "var of whatever is given behind JOIN:_" in the token.

Other question: Is this project https://www.drupal.org/project/aet possibly interesting to look at in any way to check for overlapping functionality? But maybe I am wrong at a glimpse.

Tried the patch from #17, but is not giving me the results I expected.
I am testing it out on a webform_submission where I have a webform field that references nodes. Trying to get all email fields from those nodes.

I have tried:
[webform_submission:values:autocaristen:*:entity:field_email]
Results in "webform_stored_value1entitywebform_stored_value1entitywebform_stored_value3"

[webform_submission:values:autocaristen:entity:*:field_email]
Results in nothing, empty string

[webform_submission:values:autocaristen:entity:field_email:*]
Results in nothing, empty string

[webform_submission:values:autocaristen:array:entity:field_email:join:+]
Results in "webform_stored_value1entitywebform_stored_value1entitywebform_stored_value3"

[webform_submission:values:autocaristen:array:entity:field_email:join:,]
Results in error, invalid token

[webform_submission:values:autocaristen:entity:array:field_email:join:+]
Results in nothing, empty string

Is what I am trying to do supposed to work with this patch? If so, how? If not, how can I help get it working?

I also noticed a possible error in the patch code:

  // Handle array tokens if there are any.
  if ($array_token) {
    $property_array_tokens = $token_service->findWithPrefix($field_tokens, 'array');
    foreach ($property_array_tokens as $token => $original) {

The variable $original is already defined in an outer foreach and overriden here, that could be an issue.

This patch works for me.

@weseze
This should work but only return the string separated by a comma.
[webform_submission:values:autocaristen:entity:array:field_email]

I've attached an updated patch to handle [entity:field_name:array:entity:property_name] type tokens (for accessing properties of referenced entities on an entity reference field).

Interdiff gave me an error so I couldn't generate one - the patch just adds a code block lines 1714-1732 (after application).

NB it doesn't handle the ':join' suffix functionality.

EDIT: it doesn't seem to add the token in hook_token_info.

This patch applies to the latest DEV version.

Code looks good and it works great. Taxonomy term reference field example (buried all the way in a capability to get information for the current page being viewed, courtesy the Context Stack module):

[current:view:node:field_topics:array:entity:name]

Produces for example: "Politics,Money,Crime"

Hello. The available token options of the array are not shown in the list of tokens. The only token that is being displayed in the list is array:target_id. At least the array:entity:name would be useful to add it in the list since some modules (ex. contact_emails) check the existence and the validity of the token.

Reroll of #23 against latest version.

This still needs works because of the tests. I needed this too, but I actually needed being able to customize the join ¿separator?

With this patch
[node:multivalued_field_image:array:target_id:join:+] would return 1+2
[node:multivalued_field_image:array:target_id] would return 1,2

This definitely needs a ton of tests though. I'm marking it as needs review just to see if current tests are still passing with this.

At least I didn't broke anything. Back to needs work.

@Berdir @Anybody Can we work on defining the minimum acceptable test cases that would need to be for this to be accepted?
I'm willing to work on those.

I also needed the ability to customize the join separator, but #28 didn't work for me. I modified it just a little bit.

Marked as NR to check if tests still pass.
@lisagodare@gmail.com Could you describe your scenario? If last patch wasn't working for you, maybe is something to consider when we write the tests.

Sure thing!
It's sort of complex, but in the end I'm using tokens + metatag + pivot (multivalue in metatag), and needed to not use the default comma as separator, because that breaks on text that has commas in it.
Using the patch in this issue for a custom delimiter in Metatag, which I have set to `||`:
https://www.drupal.org/node/3067803
And using patch in #28, my field token looks something like:
[node:multivalued_entity_reference_field:array:entity:field_text:join:||]

However, this resulted in always using a comma for the separator, until the changes I made in #30.

Thanks for the answer! Yeah, my scenario was similar with schema_metatag, i had a similar token pattern that you do and in my case I needed #2976935: Improve support for multi-value schema using Metatag's custom separator option, so I might overlooked yours. Thanks for documenting it!

This patch does not appear to work with webforms. I'm attempting to get a multivalue term reference field to output the term names in a list with this [webform_submission:values:multivalue_term_reference:array:entity:name] and it's being output as "Term 1 (1)entityTerm 2 (2)".

I am doing token replacement via custom code with a custom WebformHandler that's calling on the webform submission handler's token service.

Any thoughts?

Edit: I wanted to note that I'm using Drupal 10.1 and using Tokens 1.12

For me, patch #30 doesn't work for image-style tokens like [node:field_image:array:image_style] or [node:field_image:array:entity:image_style]
Patch #36 fixes works for [node:field_image:array:image_style]

Since 1.17 converted the hooks to OOP hooks I rerolled the patch in #36 for that

I created a merge request with the patch in #38.

For our situation, we have a multi-value taxonomy term entity reference field (let's call it field_tags) where some of the term names contain commas. This is incompatible with producing useful data for use in the for example the "about" property with schema_metatag.

I tested the MR and was able to produce a comma-separated list of taxonomy term IDs (versus term names) using:

[node:field_tags:array:target_id]

I was less successful referencing the term name with custom join character and then using that character as the "Separator used with multiple values"

I tried:

[node:field_tags:array:entity:name:join:+]

The issue description doesn't clearly state what syntax the MR is supposed to add/support. If someone knows, it would be good to document that so folks can accurately review.