[current-page:query:?] should fall back to empty string

The responsibility is on the module calling token_replace() to use $options['clear'] = TRUE to replace any tokens that do not have values with empty strings.

While I understand the position in #1, I think the current-page tokens should take responsibility for setting optional tokens to blank. These are valid tokens which just happen to be not present I the current request.

Here's a compelling use case. A module, webform in this case, allows tokens in conjunction with input text formats. The tokens are substituted and check_markup() is called on the result. If the tokens are cleared, then valid text (such as media module markup) is mistaken as missing tokens and removed.

But when tokens aren't removed, optional current-page tokens are left unreal aced. Catch 22.

If page-request accepts that any of its tokens not present in the current page are actually empty, the problem is resolved.

I give up on D7, but for D8 it would be great if the token module would dramatically reduce it's namespace. This would make it possible to use the 'clear' option without fear of removing desirable markup. In D7, Webform resorted to replacing the tokens, then running the text through check_markup to apply the input filters, then process it one more time to remove any uncleared tokens.

With the official release of D8 on November 18, 2015, it is imperative that the Tokens module be brought up to speed with an official release. It is the #4 most downloaded module and is a blocker for Pathauto and other modules. I recommend moving the issue to D9 if you still feel it is important.

Please do not go through marking issues as won't fix. They are still valid, and only maintainers should be marking issues as won't fix as it's ultimately their decision to make.

Entity Embed/Embed in 8.x are no longer using the token syntax but HTML5 placeholders. I think that's a thing that others will do too.

If not, then maybe the clean option should be improved to not remove tokens that are not actually tokens. I don't think this specific token needs to special case anything for that.

The use case for this token is in metatag canonical naming. According to the current Google docs on pagination, if you don't have an "All" pages, each page should have it's own canonical URL with the proper query paramenters. E.g. /blog?page=2. Currently you can produce this using:

[current-page:url:absolute]?page=[current-page:query:page]

The disadvantage is that the main page now has a URL of /blog?page= which works but doesn't look great.

This can be compounded by the need for tag grouping landing pages like /blog?page=tag=

It would be nice if there was a [current-page:query-parameter:?] that would return key=value or and empty string if the key was not there.

@Refineo I adding a patch which will replace token with url query and will not show [current-page:query:?]

If the current page URL contains multiple query parameters, the & is escaped. E.g.:

Page URL: https://foo.bar/blog?tag=security&page=4
Token: [current-page:url:absolute]
Replacement: https://foo.bar/blog?tag=security&page=4

I think the URL replacement value needs to be created as a MarkupInterface instance so that it does not get escaped (~line 204 in Token.php).

@cgmonroe thanks for your update but in your previous comment page url andReplacement are same so, i am not able to get your point.

thank's.

Sorry, the replacement URL originally just had '&' and that got rendered as' &'. I've edited my comment so the the displayed text reads correctly.

Basically the plain & query parameter separator gets encoded to & which causes problems with things like canonical links.

You would use a link like this to meet Google's current recommendations on paginated content.

@cgmonroe hi, I try to reproduce the same issue on my local drupal version of 8.3.7 and i used the [current-page:url:absolute] token and it is replacing the multiple query token by & not by &. added images for reference.

absolute url is not accepting multi query so, adding a patch for the same.
isssue:-
URL: https://localhost/test/add/page?tag=security&page=4
Token: [current-page:url:absolute]
Replacement: https://localhost/test/add/page

Attached is an image the kint output I grabbed from add debug statements to the metatags module. (Was quickest way to do this :) )

The code that generated it is below. (Placed in the MetatagManager.generateRawElements method at about line 514.

Note that the debug code is a straight use of the tokenService replace function. Uses a standard url tag and a node for input. Nothing tweaked by metatag.

if ( $tag->type() == 'uri') {
   $results = [];
   $results['Tag value'] = $tag->value();
   $results["Token replacements"] = $token_replacements;
   $result = $this->tokenService->replace($tag->value(), $token_replacements, ['langcode' => $langcode]);
   $results["Token service results"] = $result;
   $results["Decoded result"] = htmlspecialchars_decode($result);
   ksm($results);
}

@cgmonroe i am able to reproduce the issue for Token: [current-page:url:absolute] , but i believe the issue is related to metatag module and will have a separate issue create for the same.

Metatag has used drupal token services but implemented it differently which must be a reason why tokens are not decoded accordingly.

thank's.

@vakulrai, the patch you wrote in #8 sounds exactly like the solution I require.

However, when I added it to my composer.json I got:

  - Applying patches for drupal/token
    https://www.drupal.org/files/issues/2018-04-26/queryemptystring-1614456-7.patch (1614456-7)
   Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2018-04-26/queryemptystring-1614456-7.patch

I'm running;
Token: 8.x-1.3
Drupal core: 8.5.3
PHP 5.6.34

Can you think of any reason the patch wouldn't apply?

- edit -
For completion's sake -

I also tried to apply currentabsolute-1614456-12.patch - which resulted in the same composer errror.
- end edit -

@vakulrai and @Adamation

The problem is still unresolved, even to this day.

Token: 8.x-1.5
Drupal code: 8.6.4
PHP 7.0.16

Cannot apply patch https://www.drupal.org/files/issues/2018-04-26/queryemptystring-1614456-7.patch
Cannot apply patch https://www.drupal.org/files/issues/2018-05-02/currentabsolute-1614456-12.patch

I had no other choice but to edit it via webform_element_alter

function MODULE_webform_element_alter(array &$element, \Drupal\Core\Form\FormStateInterface $form_state, array $context) {
  if (key_exists('#default_value', $element)
    && is_string($element['#default_value'])
    && preg_match("/^\[current\-page\:query\:.*\]$/", $element['#default_value'])) {
    $element['#default_value'] = '';
  }
}

@Dark-Aii - I resorted to my own fix too.

I wrote a small module to clear them.

<?php

function MODULENAME_tokens_alter(array &$replacements, array $context, \Drupal\Core\Render\BubbleableMetadata $bubbleable_metadata) {
  if ($context['type'] == "current-page") {
      foreach ($context['tokens'] as $token) {
        if (!isset($replacements[$token])) {
          $replacements[$token] = '';
        }
      }
    }
}

If on D8 would the token_default module help? https://www.drupal.org/project/token_default

If your use-case is token_filter module see #2331901: Allow admin configuration to remove tokens if no replacement value can be generated

Rerolled and updated the patch.

Code typo.

Patch in #21 applies but doesn't seem to work (Token 8.x-1.6).

I confirm patch #21 applies but does nothing

As said above the patch on #21 applies cleanly, however it does nothing.

Upon further investigation the code where the value of the token is defined

 if ($query_tokens = \Drupal::token()->findWithPrefix($tokens, 'query')) {
      $bubbleable_metadata->addCacheContexts(['url.query_args']);
      foreach ($query_tokens as $name => $original) {
        if (\Drupal::request()->query->has($name)) {
          $value = \Drupal::request()->query->get($name);
          $replacements[$original] = $value;
        }
      }
    }

is unchanged. So I am not certain what the patch is for.

I am on 8.8.8 and Using this token in the metatag [current-page:url:absolute] returns all queries correctly or no queries if none are present.
<link rel="canonical" href="http://test.dev.localhost/resource-library?page=12" />
It only presents problems if I set tokens like so:
[current-page:url:absolute]?page=[current-page:query:page] then ?page is output no matter what but it has to since it's not part of a token but a hardcoded part.

I don't think I can reproduce the issue then unless I am doing something wrong. Also comparing current module code and patch from 21 - current code already has a lot from 21 patch so not sure patch 21 is needed.

I have the same issue on fresh D9 using default values in a webform with the token [current-page:query:Y].

Webform does not pass clear by default, but implemented a :clear suffix to fix this issue. See #2970785: Allow any webform token to be cleared using the :clear suffix

So [current-page:query:Y:clear] will change the token-string to an empty as default value.

For all others make sure that

$options = ['clear' => TRUE];

is set to default to an empty string.

Sorry for closing ..

Here a patch that worked for me, plain a simple. Hopefully works for you.

I had to apply #21 and #28 to make [current-page:url:absolute] work. Thank you

Code from #28 pushed to the MR and also fixed the test.

I can't reproduce the original issue on the 8.x-1.x branch, so this change may no longer be necessary.

On the 8.x-1.x branch, [current-page:query:key] resolves to an empty string rather than remaining as the literal string "[current-page:query:key]", as originally reported.

Also, there is merge conflict.