Problem/Motivation

For entity reference fields (and also timestamp fields), there exists more possibilities for tokens than the one configured using the token view mode.

For instance, given a field_sample_entity_reference that points to another node, all fields, and node properties might be wanted as tokens (eg, [node:field_sample_entity_reference:title], node:field_sample_entity_reference:field_another_field).

In Drupal 7, this was accomplished by the entity_token sub-module of the Entity API module.

Proposed resolution

Support this here, or discuss and move back into the 8.x branch of the Entity API module.

Remaining tasks

  • Integrate into the token browser
  • Re-roll patch so it applies
  • Tests

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#63 chained_tokens_for-2493559-63-interdiff.txt4.46 KBBerdir
#63 chained_tokens_for-2493559-63.patch13.27 KBBerdir
#62 interdiff-61-62.txt596 bytesBambell
#62 chained_tokens_for-2493559-62.patch11.89 KBBambell
#61 interdiff-58-61.3.txt8.71 KBBambell
#61 chained_tokens_for-2493559-61.3.patch11.82 KBBambell
#61 entity-reference-field-tokens-referenced-label-for-description.png73.58 KBBambell
#58 interdiff-57-58.txt2.64 KBBambell
#58 chained_tokens_for-2493559-58.patch12.49 KBBambell
#57 interdiff-55-57.txt5.82 KBBambell
#57 chained_tokens_for-2493559-57.patch12.97 KBBambell
#55 chained-2493559.50.patch14.49 KBlarowlan
#51 chained-2493559.50.patch14.37 KBlarowlan
#51 interdiff.txt1.63 KBlarowlan
#45 interdiff-45-42.2.txt10.35 KBBambell
#45 chained_tokens_for-2493559-45.2.patch12.96 KBBambell
#42 interdiff-38-42.txt5.32 KBBambell
#42 chained_tokens_for-2493559-42.patch11.39 KBBambell
#41 chained-tokens-entity.png125.45 KBBambell
#38 token-chained-tokens-browser-after-node-term.png274.21 KBBambell
#38 chained_tokens_for-2493559-38.patch12.31 KBBambell
#32 2493559-32.patch19.39 KBjhedstrom
#29 token-2493559-29.patch19.44 KBgapple
#26 2493559-26.patch19.58 KBjhedstrom
#26 interdiff.txt5.52 KBjhedstrom
#22 2493559-22.patch17.38 KBjhedstrom
#22 interdiff.txt3.82 KBjhedstrom
#20 2493559-20.patch14.64 KBjhedstrom
#20 interdiff.txt3 KBjhedstrom
#19 2493559-19.patch12.27 KBjhedstrom
#19 interdiff.txt940 bytesjhedstrom
#17 2493559-17.patch12.18 KBjhedstrom
#17 interdiff.txt1.07 KBjhedstrom
#10 token-Added_chained_tokens_for_field_tokens-2493559-10.patch11.96 KBMaouna
#10 9-10-interdiff.txt4.7 KBMaouna
#9 token-Added_chained_tokens_for_field_tokens-2493559-9.patch12.39 KBMaouna
#7 token-Added_chained_tokens_for_field_tokens-2493559-7.patch12.07 KBMaouna
#5 token-Added_chained_tokens_for_field_tokens-2493559-5.patch7.33 KBMaouna
#2 token-Added_chained_tokens_for_field_tokens-2493559-2.patch3.58 KBMaouna
#1 token-Added_chained_tokens_for_field_tokens-2493559-1.patch3.24 KBMaouna
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Maouna’s picture

Maouna’s picture

Berdir’s picture

Status: Active » Needs review
Berdir’s picture

Status: Needs review » Needs work
  1. +++ b/token.tokens.inc
    @@ -1257,6 +1258,8 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
     
    +    $chained_field_token_candidates= array();
    

    missing space before the =

  2. +++ b/token.tokens.inc
    @@ -1272,6 +1275,51 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
    +      else {
    +        // Check for chained token.
    +        $chained_token_names = explode(':', $name);
    +        if ($entity->hasField($chained_token_names[0]) ) {
    

    This should be an elseif () and checking if there is a : on the token, or it might do weird stuff if it's another token.

    You can also use something like this to avoid using the exploded value as an array:

    list($field_name) = explode(':', $name);

  3. +++ b/token.tokens.inc
    @@ -1272,6 +1275,51 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
    +          // Get field property definition.
    +          $field = $entity->$chained_token_names[0];
    +          $field_definition = $field->getFieldDefinition();
    +          if ($field_definition instanceof FieldConfigInterface) {
    +            $field_property_definition = $field_definition->getFieldStorageDefinition()->getPropertyDefinitions();
    +          }
    +          elseif ($field_definition instanceof BaseFieldDefinition) { //only possible with https://www.drupal.org/node/2493567
    +            $field_property_definition = $field_definition->getPropertyDefinitions();
    +          }
    +          else {
    +            continue;
    +          }
    

    You can simplify this a bit, make sure the field really exists with $entity->hasField() (or it will throw an exception if it doesn't) and then just do $entity->getFieldDefinition($field_name)->getFieldStorageDefinition(). That method works for base field definitions too, they are field definitions and field storage definitions and the method just returns themself again.

Maouna’s picture

Status: Needs work » Needs review
FileSize
7.33 KB

I updated my patch. Thank you, Berdir, for your feedback which I tried to integrate.

Apart from that, I included the possibility to get the values from multiple fields, too. For example, if field_article_reference was a multiple field on node, referencing articles: [node:field_article_reference:1:title]

Also, the entities are now used in the translation selected in $options['langcode'].

Dave Reid’s picture

So now the hard part, is how do you expose all these additional tokens in the UI? Without that part, this cannot move forward.

Maouna’s picture

Yes, providing the tokens in the token tree was a bit tricky. The updated patch is doing that now. Please have a look whether my approach is a good one or should be modified.

Status: Needs review » Needs work

The last submitted patch, 7: token-Added_chained_tokens_for_field_tokens-2493559-7.patch, failed testing.

Maouna’s picture

Status: Needs work » Needs review
FileSize
12.39 KB
Maouna’s picture

I updated the patch. It includes now the bubbleable metadate of core.

Status: Needs review » Needs work

The last submitted patch, 10: token-Added_chained_tokens_for_field_tokens-2493559-10.patch, failed testing.

The last submitted patch, 10: token-Added_chained_tokens_for_field_tokens-2493559-10.patch, failed testing.

juampynr’s picture

jhedstrom’s picture

Priority: Normal » Major

This is what would provide support for things like [node:field_my_entity_reference:title], yes?

Unless I'm misunderstanding, this is at least a major.

Berdir’s picture

This is one related issue, yes. See also #2621598: Add support for field properties, which I think is more progressed and has info of the many hard problems here.

Also, for the record, token in 7.x never did this either, this was the entity tokens module, actually.

Not sure if we should close this as a duplicate, but it actually doesn't do the entity part yet, so maybe we could do that here then.

jhedstrom’s picture

Updating the IS to reflect Berdir's feedback above.

jhedstrom’s picture

Status: Needs work » Needs review
FileSize
1.07 KB
12.18 KB

Re-roll of #10 to get this going again.

Status: Needs review » Needs work

The last submitted patch, 17: 2493559-17.patch, failed testing.

jhedstrom’s picture

Assigned: Unassigned » jhedstrom
Status: Needs work » Needs review
FileSize
940 bytes
12.27 KB

This fixes the failing tests. Working on adding some tests for the chained bit.

jhedstrom’s picture

FileSize
3 KB
14.64 KB

This adds tests for entity reference chaining.

scoff’s picture

It doesn't seem to work with taxonomy term fields. Works with Media (media_entity). Somewhat works with Paragraphs.
Am I missing something?

jhedstrom’s picture

This adds a failing test for the taxonomy issue mentioned in #21. Not sure what the fix is just yet.

Status: Needs review » Needs work

The last submitted patch, 22: 2493559-22.patch, failed testing.

The last submitted patch, 22: 2493559-22.patch, failed testing.

jhedstrom’s picture

I made a bit of progress in figuring out why certain types (taxonomy_term) aren't working.

It comes down to this bit of code in token_tokens():

  // Entity tokens.
  if (!empty($data[$type]) && $entity_type = \Drupal::service('token.entity_mapper')->getEntityTypeForTokenType($type)) {
    /* @var \Drupal\Core\Entity\EntityInterface $entity */
    $entity = $data[$type];

Since taxonomy_token_info() doesn't use the actual entity id, but term instead, a mapping is needed. However, in this case, the $type variable is set to the actual entity id (taxonomy_term), so the reverse mapping would be needed. I don't think we can just check for both token-to-entity-type and entity-type-to-token here, so some other refactoring of this logic will be needed I think.

jhedstrom’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
5.52 KB
19.58 KB

The issue mentioned in #25 was actually related to this patch already, so the fix was to run entity types through the entity mapper service to convert them to token types prior to generating tokens. This also adds a test for cardinality > 1 chained tokens.

kevin.dutra’s picture

I've looked over #26 and things are looking pretty good. I only had time to do a small amount of manual testing, so if someone was able to do a bit more on that front, it would be good.

Berdir’s picture

+++ b/tests/src/Kernel/FieldTest.php
@@ -201,4 +270,80 @@ class FieldTest extends KernelTestBase {
+
+    $this->assertTokens('node', ['node' => $entity], [
+      'test_reference:title' => Markup::create('Test node to reference'),
+      'test_reference:test_field' => Markup::create('foo'),
+      'test_term_reference:term_field' => Html::escape($term_reference_field_value),
+    ]);

I'm not sure how this will work when combined with the field property patch.

That exposes things like test_reference:target_id and :entity.

And we can't do both at the same time.

So I still think we have to do that issue first, and then do something like test_reference:entity:title.

gapple’s picture

Tried this patch for use in a pathauto pattern, but validating the pattern fails because the numeric index is not understood in Token::getInvalidTokens().
I was able to get validation to pass by marking the 'multiple_field_{entity_type}' tokens as dynamic, but this caused the available tokens list to show the delta token as the child of a '?' placeholder instead of the field itself.

Status: Needs review » Needs work

The last submitted patch, 29: token-2493559-29.patch, failed testing.

The last submitted patch, 29: token-2493559-29.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
FileSize
19.39 KB

Thanks for the review @Berdir. I haven't had time to incorporate these changes, but just so this patch doesn't grow stale, here is a reroll against the latest 8.x. The tests are failing (as expected with the latest base field patch landing).

Status: Needs review » Needs work

The last submitted patch, 32: 2493559-32.patch, failed testing.

The last submitted patch, 32: 2493559-32.patch, failed testing.

scoff’s picture

I get an error "Field field_section:url:path is unknown." trying to save a node or running bulk update in Pathauto.

Using Tokens latest dev + 2493559-32.patch on Drupal 8.1.1
field_section is an Entity reference (Taxonomy term)
Pattern is /[node:field_section:url:path]/[node:nid]

The token is in the available tokens list under node:field_section as expected but none of node:field_section:* work (well, I haven't tried all of them) .

Berdir’s picture

#2621598: Add support for field properties is in, so this will need a big reroll now.

With all the work that the other issue did, I would expect that this can now be done by building on top of that and just making sure the reference tokens are defined and passed along + test coverage.

Bambell’s picture

I'll work on this today.
Edit : And tomorrow*.

Bambell’s picture

Round 1.

I tried to re-use a maximum of code, but it was somewhat difficult to read and mostly no longer relevant with #2621598: Add support for field properties in. The logic is pretty straightforward. For single valued entity reference fields, if there is a token type defined for the referenced entity, field's token type is changed to that of the referenced entity. For multivalued fields, the list tokens are generated and it's the type of those that is changed. For fields token replacement, I add a check to verify if the field is an entity reference field. If so, I build the parameters needed for \Drupal::token()->generate() dependently of the field's cardinality, making sure only 1 token is passed to generate(). Some of that code could be taken form the previous patch.

Bambell’s picture

Status: Needs work » Needs review
Berdir’s picture

Status: Needs review » Needs work
+++ b/tests/src/Kernel/FieldTest.php
@@ -316,4 +388,96 @@ class FieldTest extends KernelTestBase {
+
+    $this->assertTokens('node', ['node' => $entity], [
+      'test_reference:title' => Markup::create('Test node to reference'),
+      'test_reference:test_field' => Markup::create('foo'),
+      'test_term_reference:term_field' => Html::escape($term_reference_field_value),

I would expect that this now works on the field property level, just like entity field api.

Meaning, on an entity refrence field, I'd expect to see this:

[node:some_field:target_id] (already exists)
[node:some_field:entity] (added by this patch).

:entity is then the reference and of type term, for example.

Basically very similar to the image style patch

Bambell’s picture

FileSize
125.45 KB

We are loosing entity reference field property tokens with these patches (:target-id, :entity, etc.). The field's token type is changed to that of the referenced entity. As far as I can tell, there's 3 things we can do :

1- Add the field property tokens to the referenced entity's tokens (very bad).
2- Add the referenced entity's tokens to the entity reference field tokens (sounds like a bad idea).
3- Add instead an intermediary token to the entity reference field tokens that is of the referenced entity's type, so that we have :

[node:{entity_reference_field}:target_id]
[node:{entity_reference_field}:entity]
[node:{entity_reference_field}:referenced_entity:{referenced_entitys_tokens}]

I quickly started to implement the third option, but the problem comes with mutlivalued fields. Too many nested tokens, we cannot see the referenced entity's tokens in the tokens browser (see screenshot).

Not too sure which direction to go with this at this point.

Bambell’s picture

Status: Needs work » Needs review
FileSize
11.39 KB
5.32 KB

Work in progress patch for 3., tests will fail.

Status: Needs review » Needs work

The last submitted patch, 42: chained_tokens_for-2493559-42.patch, failed testing.

The last submitted patch, 42: chained_tokens_for-2493559-42.patch, failed testing.

Bambell’s picture

Here we go, we now have those tokens :

[node:{entity_reference_field}:target_id]
[node:{entity_reference_field}:entity]
[node:{entity_reference_field}:entity:{referenced_entitys_tokens}]

The token replacement code isn't too elegant, though. Tried to keep it intelligible, but covering all cases is a bit complex.

Also added a basic test for :entity tokens. Interestingly, the replacement entity provided isn't identical to the referenced entity, it's missing fields and some values..

jhedstrom’s picture

This is looking pretty good! Thanks @Bambell for picking this up!

jhedstrom’s picture

Interestingly, the replacement entity provided isn't identical to the referenced entity, it's missing fields and some values..

Can the test be updated to demonstrate this issue?

jibran’s picture

+++ b/tests/src/Kernel/FieldTest.php
@@ -107,6 +122,64 @@ class FieldTest extends KernelTestBase {
+    // Add a node reference field.
+    $reference_storage = FieldStorageConfig::create([
+      'field_name' => 'test_reference',
+      'entity_type' => 'node',
+      'type' => 'entity_reference',
+    ]);
+    $reference_storage->save();
...
+    $reference = FieldConfig::create([
+      'field_name' => 'test_reference',
+      'entity_type' => 'node',
+      'bundle' => 'article',
+      'label' => 'Test reference',
+    ]);
+    $reference->save();
...
+    $reference_storage = FieldStorageConfig::create([
+      'field_name' => 'test_term_reference',
+      'entity_type' => 'node',
+      'type' => 'entity_reference',
+      'settings' => [
+        'target_type' => 'taxonomy_term',
+      ],
+    ]);
+    $reference_storage->save();
...
+    $reference = FieldConfig::create([
+      'field_name' => 'test_term_reference',
+      'entity_type' => 'node',
+      'bundle' => 'article',
+      'label' => 'Test term reference',
+      'settings' => [
+        'handler' => 'default:taxonomy_term',
+        'handler_settings' => [
+          'target_bundles' => [
+            $this->vocabulary->id() => $this->vocabulary->id(),
+          ],
+        ]
+      ]
+    ]);
+    $reference->save();

There is a trait for this EntityReferenceTestTrait and all this can be replaced by createEntityReferenceField

larowlan’s picture

+++ b/token.tokens.inc
@@ -1395,12 +1407,67 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
+          elseif (count($parts) == 3 && ($parts[1] === 'entity')) {
+            // Token is [entity:field_name:entity:value].
+            $literal_text = $parts[2];

this breaks those of format

[entity:field_name:entity:url:path]

I think the == should be >=

larowlan’s picture

Working on fix

larowlan’s picture

Status: Needs review » Needs work

The last submitted patch, 51: chained-2493559.50.patch, failed testing.

The last submitted patch, 51: chained-2493559.50.patch, failed testing.

Berdir’s picture

index d418088..b9a7d88 100644
--- a/modules/contrib/token/tests/src/Kernel/FieldTest.php

--- a/modules/contrib/token/tests/src/Kernel/FieldTest.php
+++ b/modules/contrib/token/tests/src/Kernel/FieldTest.php

You forgot a --relative :)

larowlan’s picture

Status: Needs work » Needs review
FileSize
14.49 KB

I ran --relative.
But I forgot to cd.

jhedstrom’s picture

This doesn't incorporate the test cleanup @jibran mentions in #48, but is looking good functionally.

Bambell’s picture

Bambell’s picture

Berdir’s picture

Status: Needs review » Needs work

Nice, getting close :)

Feedback on the test and another idea to simplify it a bit more:

  1. +++ b/tests/src/Kernel/FieldTest.php
    @@ -107,6 +124,40 @@ class FieldTest extends KernelTestBase {
    +    // Add a field to the vocabulary.
    +    $storage = FieldStorageConfig::create([
    +      'field_name' => 'term_field',
    +      'entity_type' => 'taxonomy_term',
    +      'type' => 'text',
    +    ]);
    +    $storage->save();
    +    $field = FieldConfig::create([
    +      'field_name' => 'term_field',
    +      'entity_type' => 'taxonomy_term',
    +      'bundle' => $this->vocabulary->id(),
    

    The field is added to terms, not to the vocabulary. "to terms of the created vocabulary" maybe?

  2. +++ b/tests/src/Kernel/FieldTest.php
    @@ -316,4 +367,118 @@ class FieldTest extends KernelTestBase {
    +      'test_reference:target_id' => $reference->id(),
    +      'test_term_reference:target_id' => $term_reference->id(),
    +      'test_term_reference:entity:url:path' => '/' . $term_reference->toUrl('canonical')->getInternalPath(),
    

    Lets add some bogus examples as well here, for fields/properties that don't exist.

    so non_existing_field:entity:title should be empty, test_reference:foo:bar should be empty, and test_reference:entity:foo should be empty, all without errors.

  3. +++ b/tests/src/Kernel/FieldTest.php
    @@ -316,4 +367,118 @@ class FieldTest extends KernelTestBase {
    +    $this->assertTokens('node', ['node' => $entity], [
    +      'test_term_reference:0:entity:term_field' => Html::escape($terms[1]->term_field->value),
    +      'test_term_reference:1:entity:term_field' => Html::escape($terms[2]->term_field->value),
    +      'test_term_reference:2:entity:term_field' => Html::escape($terms[3]->term_field->value),
    

    Also mix in one example without an explicit delta as well, to make sure those can be mixed, even in the same token replace call.

  4. +++ b/token.tokens.inc
    @@ -24,6 +27,7 @@ use Drupal\Core\TypedData\PrimitiveInterface;
     
    +
     /**
      * Implements hook_token_info_alter().
    

    unrelated change.

  5. +++ b/token.tokens.inc
    @@ -1276,6 +1280,16 @@ function field_token_info_alter(&$info) {
    +            'name' => 'Referenced entity',
    +            'description' => t('The entity referenced by this entity reference field.'),
    

    based on the target definition, we could actually display the label of the referenced entity here, check how \Drupal\Core\Field\Plugin\Field\FieldType\EntityReferenceItem::propertyDefinitions() does that.

  6. +++ b/token.tokens.inc
    @@ -1395,12 +1409,14 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
           // tokens.
    -      else if ($field_tokens = \Drupal::token()->findWithPrefix($tokens, $field_name)) {
    +      else {
    +        $field_tokens = \Drupal::token()->findWithPrefix($tokens, $field_name);
             $property_token_data = [
               'field_property' => TRUE,
               $data['entity_type'] . '-' . $field_name => $data['entity']->$field_name,
               'field_name' => $data['entity_type'] . '-' . $field_name,
             ];
    +
             $replacements += \Drupal::token()->generate($field_name, $field_tokens, $property_token_data, $options, $bubbleable_metadata);
    

    revert those changes

  7. +++ b/token.tokens.inc
    @@ -1411,16 +1427,43 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
    +          $filtered_tokens = \Drupal::token()->findWithPrefix($tokens, $delta);
    ...
    +        else {
    +          $delta = 0;
    

    Add a comment here, that the first part is not actually a delta, and we need to reset it to 0.

    Actually, that makes me wonder if we can't improve this to only do a single explode.

    Basically, that would look like this:

    $delta = 0;
    $parts = explode(':', $name); (maybe rename $name to $token in the foreach as well)
    if (count($parts) == 1) {
    // invalid token, ignore.
    continue;
    }
    if (is_numeric($parts[0]) && count($parts) > 1) {
    $delta = $parts[0];
    $property_name = $parts[1];
    $filtered_tokens = \Drupal::token()->findWithPrefix($tokens, $delta);
    }
    else {
    $property_name = $parts[0];
    }

    // check if delta exists
    ...

    if (property exists) {
    $property_value = ...;
    if ($property_value instanceof FieldableEntityInterface) {
    // do entity stuff here
    }
    else {
    // plain replacement here.
    }
    }

    I think that would be more readable and easier to extend with e.g. the image style stuff as well.

Bambell’s picture

Regarding 7., this won't work with :entity tokens (fieldname:delta:entity or fieldname:entity). It will try to generate tokens (it will get into if ... instanceof FieldableEntityInterface) while what we want is to directly provide a replacement.

Bambell’s picture

Here we go, all points have been addressed. :entity tokens are now being replaced by the referenced entity's label.

Bambell’s picture

Adding a one line comment to clarify something.

Berdir’s picture

Some final cleanups and also adding test coverage for the token info.

jhedstrom’s picture

Status: Needs review » Reviewed & tested by the community

This looks good to go!

I manually tested too and verified this is behaving as expected in the UI.

  • Berdir committed e595673 on 8.x-1.x authored by Bambell
    Issue #2493559 by Bambell, jhedstrom, Maouna, larowlan, Berdir, gapple:...
Berdir’s picture

Status: Reviewed & tested by the community » Fixed

Thanks everyone!

Committed.

jibran’s picture

Nice work everyone. Great to see this patch is committed. @Berdir I hope this patch also works for DER field.

Berdir’s picture

@jibran: I fear not, although it should support ERR. For DER, you'd need to have fake properties, one for each enabled target type to be able to expose that in a token compatible format in the token info. That said, replacements might actually work.

jibran’s picture

@Berdir would you like to create an issue in DER issue queue with instructions to fix it?

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

nghai’s picture

Hi,

I have a query. Will this "Chained tokens" work only for Content entities reference fields which extended "\Drupal\Core\Entity\ContentEntityInterface" class not for other entities reference fields like Config entities which extended "\Drupal\Core\Config\Entity\ConfigEntityType")?

Why I am asking this because the node fields token replacement is not working for Domain entity reference fields?
If I am using this token [node:field_domain_access:0:entity:id] it results in a fatal error which states "Object of class Drupal\domain\Entity\Domain could not be converted to string in Drupal\Component\Render\HtmlEscapedText->__construct()" which is because this "field_token_info_alter" hook does not return any matching token info.