[meta] Improve field support and add test coverage

It's great that you're working on this. I would suggest that you open separate issues for new features, then we can discuss them separately.

Then we'll convert this issue into a meta.

Adding support base fields makes sense, as this should essentially give you free token integration for custom entity types. It wasn't done so far for historical reasons and because there are a lot of custom tokens already. We need to make sure that we're not creating too many duplicating tokens (for example in case core doesn't use the field names for the token names..). Also, if you do this, then you can just remove those checks completely and we can support all field definitions.

1. +++ b/token.tokens.inc
   @@ -1274,6 +1274,38 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
   +    // Chained token relationships.
   +    $fields = $entity->getFieldDefinitions();
   +    $token_service = \Drupal::token();
   +    foreach (array_keys($fields) as $field_name) {
   

   Looping over field definitions isn't a bad as looping over instantiated fields, so that's not as bad as it was before the referenced commit.

   However, that could still result in quit a few findWithPrefix() calls. Maybe we can optimize it to first find all unique "field:*" just once, and then loop over them and check if they are fields?

2. +++ b/token.tokens.inc
   @@ -1274,6 +1274,38 @@ function field_tokens($type, $tokens, array $data = array(), array $options = ar
   +      $entity_type2 = FALSE;
   +      $entity2 = FALSE;
   +      if (isset($field_property_definition['value']) && ($field_property_definition['value']->getDataType() == 'timestamp' )) {
   +        $entity_type2 = 'date';
   +        $entity2 = $field->value;
   +      }
   +      elseif (isset($field_property_definition['entity'])) {
   +        $entity_type2 = $field_property_definition['entity']->getTargetDefinition()->getEntityTypeId();
   +        $entity2 = $field->entity;
   +      }
   

   Using $entity for something that's not an entity at all is pretty confusing. I think we should just move the findWithPrefix() call into thw two if's and then use variable names that make sense (or just hardcode it)

#2772673: Provide tokens for referenced entities in the active language. is now fixed as well.

#2497247: Field tokens in correct language is still open, but I think that is actually a duplicate of the previous issue, so marking this as fixed!