'unify' hook_token_info()

+++ b/core/modules/system/system.api.phpundefined
@@ -3326,81 +3326,64 @@ function hook_tokens_alter(array &$replacements, array $context) {
-    'needs-data' => 'node',

This is critical information that tells us if the token is global, or requires input.

A concern that was just voiced on IRC was that the suggested solution does not allow developers to add tokens to existing types. I want to make token_info() use NestedArray, so this functionality is still available.

That's precisely the reason that the two separate pools of hook data were initially defined. The set of "Tokenable things" and the set of "Discrete tokens" can be managed independently without explicitly digging in and altering someone else's data.

I don't have really strong feelings one way or another, as long as it doesn't prevent a dev from hucking arbitrary tokens into a given token space.

We should have a separate issue for renaming 'needs-data' to a 'needs-data' => TRUE/FALSE flag instead.

This is looking really awesome. The only thing I can think of is that I actually prefer the tokens to be listed like they were before, but all in one array:

  // Core tokens for nodes.
  $tokens['node'] = array(
    'name' => t('Nodes'),
    'description' => t('Tokens related to individual content items, or "nodes".'),
    'needs-data' => 'node',
  );
  $tokens['node']['tokens']['nid'] = array(
    'name' => t("Content ID"),
    'description' => t('The unique ID of the content item, or "node".'),
  );
  $tokens['node']['tokens']['vid'] = array(
    'name' => t("Revision ID"),
    'description' => t("The unique ID of the node's latest revision."),
  );
  $tokens['node']['tokens']['tnid'] = array(
    'name' => t("Translation set ID"),
    'description' => t("The unique ID of the original-language version of this node, if one exists."),
  );
  ...

As compared to what #9 currently has in the hooks:

  // Core tokens for nodes.
  $tokens['node'] = array(
    'name' => t('Nodes'),
    'description' => t('Tokens related to individual content items, or "nodes".'),
    'needs-data' => 'node',
    'tokens' => array(
      'nid' => array(
        'name' => t('Content ID'),
        'description' => t('The unique ID of the content item, or "node".'),
      ),
      'vid' => array(
        'name' => t('Revision ID'),
        'description' => t("The unique ID of the node's latest revision."),
      ),
      'tnid' => array(
        'name' => t('Translation set ID'),
        'description' => t('The unique ID of the original-language version of this node, if one exists.'),
      ),
  );

The former is a little more verbose, but I find it easier to read and scan, especially for people that want to look at the info hooks to figure out which tokens to alter.

If we agree to go with the first syntax, I would prefer to have it addressed not in a followup issue but here.

+++ b/core/includes/token.incundefined
@@ -235,31 +237,24 @@ function token_find_with_prefix(array $tokens, $prefix, $delimiter = ':') {
 function token_info() {
-  $data = &drupal_static(__FUNCTION__);
-  if (!isset($data)) {
-    $data = module_invoke_all('token_info');
-    drupal_alter('token_info', $data);
+  $token_info = &drupal_static(__FUNCTION__);
+
+  if (is_null($token_info)) {
+    $token_info = module_invoke_all('token_info');
+    drupal_alter('token_info', $token_info);
+    // Expand data types that extend another type.
+    foreach ($token_info as &$type) {
+      if (isset($type['type']) && isset($token_info[$type['type']])) {
+        $type = NestedArray::MergeDeep($token_info[$type['type']], $type);
+      }
+    }
   }
-  return $data;
+
+  return $token_info;
 }

This is a functional change, and should be removed for now. This will also conflict with #1969540: Convert token.inc to a service

+++ b/core/modules/system/system.api.phpundefined
@@ -3303,83 +3303,97 @@ function hook_tokens_alter(array &$replacements, array $context) {
+ *     - type (optional): Another data type that this type extends.
+ *   - type (optional): Another data type that this type extends.

This seems potentially confusing that 'type' is back to back. We should move the latter 'type' up above with the other first-level key/values before 'tokens'.

+++ b/core/modules/system/system.api.phpundefined
@@ -3303,83 +3303,97 @@ function hook_tokens_alter(array &$replacements, array $context) {
+ * For example:
+ * @code
+ *   $tokens['node'] = array(
+ *     'needs-data' => 'node,
+ *     'name' => t('Nodes'),
+ *     'tokens' => array(
+ *       'title' => array(
+ *         'label' => t('Title'),
+ *       ),
+ *       'author' => array(
+ *         'name' => t('Author'),
+ *         'description' => t('The author of the current node.'),
+ *         'type' => 'user',
+ *       ),
+ *     ),
+ *   );
+ * @endcode

Isn't this what the example code in hook_token_info() is for? This seems to be unnecessary. Also it's incorrect. The 'title' token uses 'label' instead of 'title' and the 'description' key/value is not optional yet.

We should probably just remove this chunk.

+++ b/core/modules/system/system.api.phpundefined
@@ -3303,83 +3303,97 @@ function hook_tokens_alter(array &$replacements, array $context) {
+ * @see token_info()

Function no longer exists, since it's the service now?

Fixing tags

This doesn't look impossible to implement with a bc layer - even if we had to do two info hooks for the different formats, so moving back to 8.x for now.

I think this should probably be closed in favour of #2233353: Convert Token API into plugins?

#33: I agree this should be closed in favour of #2233353: Convert Token API into plugins.

I'll leave it for 3 months for further discussion. If there is none I will close it after those three months have passed.