Without reading the 522 long field tokens thread where this may already be covered, what are your thoughts about adding this option?

I've hit this requirement a lot recently, two examples are:

Webform custom email component
By using tokens and webform 7.x-4.x it is possible to add an email field on the node and embed this into the webform to send to that email address on submission. Currently, if you use the email field for this, you need to define this field as plain text in the token display mode. More of a pain than anything else having to define this custom display mode and configure this field not to use the default formatter.

Read More Control integration
This module allows you to pull in entity tokens for the link, but the setting allows HTML so users have full control of the "Read more<span class="element-invisible"> about @title</span>", so you can specify a field here, "[node:field_read_more]<span class="element-invisible"> about [node:title]</span>". Since the token uses the safe value, this encodes HTML entities, et al, even though the token_replace() has the 'sanitize' setting set to FALSE.

Both these would be fine using the raw values. In the second use-case, the raw value is required.

One possible solution is to define a new hook that defines a raw field value. If missing, you could drop back to the $item['value'] key if set. A possible hook for a file field could be (throw a coin if to decide if this one should use the fid or file path, file path seems more intuitive to me)

function file_field_token_raw($items, $context) {
  $values = array();
  foreach ($items as $delta => $item) {
    if (isset($item['file'])) {
      $values[$delta] = $item['file']->filename;
    }
  }

  return $values;
}
CommentFileSizeAuthor
#3 field_tokens-sanitize-1713164-3.patch2.14 KBTess Bakker
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dww’s picture

dww
we/he/they
CreditAttribution: dww commented
Category: feature » task
Priority: Normal » Major

Yes, we need access to raw field values via tokens for modules like auto_nodetitle to work. See #1071596: HTML Special characters in node title for lots of confusion on the topic.

For text fields, it seems you can configure the 'tokens' view mode, and configure them to use 'plain text' and it basically works. But then that token is *always* unsanitized. It'd be nice if you could just conditionally express this when you use the token, instead of having to choose site-wide if a given field's tokens will be safe or raw...

I tried to follow most of #691078: Field tokens but it wasn't obvious to me why the raw tokens were removed. Seems like a separate issue here for this request is valuable.

Also, this *seems* like a regression from D6 token.module, so I'm at least going to call it a "task", not a "feature". Seems more like "bug" to me, but I don't want to piss off Dave Reid. ;)

Thanks!
-Derek

Media Crumb’s picture

Media Crumb CreditAttribution: Media Crumb commented

Did anything ever happen with this. I'm running into an issue with Meta tag module where I need to get access to the RAW token value. Pulling my hair out. Not sure why these values show up in views just fine.

Tess Bakker’s picture

Tess Bakker
they/them
Primary language Dutch
CreditAttribution: Tess Bakker at ezCompany commented
Title: Raw field tokens » field_tokens() isn't respecting sanitize option
Category: Task » Bug report
Status: Active » Needs review
FileSize
field_tokens-sanitize-1713164-3.patch2.14 KB

The module auto_nodetitle uses the option sanitize = FALSE, but this isn't respected by field_tokens().
Therefore the module does some tinkering to fix this.

It would be great if this bug can be fixed in tokens, so other modules can relay on field_tokens() again.

A possible solution is attached as a patch, but it depends on Entity.

Status: Needs review » Needs work

The last submitted patch, 3: field_tokens-sanitize-1713164-3.patch, failed testing.

Dave Reid’s picture

Dave Reid
he/him
Primary language English
Location Nebraska USA
CreditAttribution: Dave Reid commented
Category: Bug report » Feature request

We cannot just add a late dependency on entity module. It would have to be completely optional please.

Given that Field API itself provides no options for unsanitized vs sanitized, reclassifying as a feature request.

Alan D.’s picture

Alan D. CreditAttribution: Alan D. commented

So if it currently renders HTML version with sanitized content... maybe a post-processor to strip the tags and to un-sanitize?

i.e. something like this totally untested code:

$field_output['#post_render'][] = 'token_post_render_field_token';

function token_post_render_field_token($markup, $elements) {
  // Strip tags and de-sanitize if required.
  // using  empty($elements['#token_options']['sanitize']) as it is more obvious than !$elements['#token_options']['sanitize']
  if (isset($elements['#token_options']['sanitize']) && empty($elements['#token_options']['sanitize'])) {  
    $markup = strip_tags($markup);
    // This is PHP 5.1.0 + function. DRUPAL_MINIMUM_PHP = 5.2.4 in 7.35
    $markup = htmlspecialchars_decode($markup);
  }
    
  return $markup;
}
Tess Bakker’s picture

Tess Bakker
they/them
Primary language Dutch
CreditAttribution: Tess Bakker at ezCompany commented

Nice idea, but you can't un-sanitize a sanitized string.
In the end I like to see every html tag or any other great xss code in my raw field result.

Would a sub-module be oké?

Let say the sub-module 'tokens_raw' has a dependency on Entity and the field_tokens() handles the raw function if the sub-module is active. In this way the 'documentation' part is fixed for beginners and the functionality works as expected.

Reading code in the morning isn't a good idea. :)
@Alan D. Will try your post-processor solution this week!

Alan D.’s picture

Alan D. CreditAttribution: Alan D. commented

Let say the sub-module 'tokens_raw' has a dependency on Entity and the field_tokens() handles the raw function if the sub-module is active.

hehe, I have used this approach regularly... a custom token that does what I want

g33kg1rl’s picture

g33kg1rl CreditAttribution: g33kg1rl commented

Is there any progress on this? If someone provides a patch, I can test it. :)

ciss’s picture

ciss CreditAttribution: ciss at yousign GmbH commented

For those that need a quick solution, here's a workaround that decodes HTML entities if sanitize = false is specified in the token options:

/**
 * Implements hook_field_attach_view_alter().
 */
function MODULE_field_attach_view_alter(&$output, $context) {
  if ($context['display'] === 'token') {
    foreach (element_children($output) as $name) {
      $output[$name]['#post_render'][] = '_MODULE_field_token_unsanitize';
    }
  }
}

function _MODULE_field_token_unsanitize($output, $element) {
  if (isset($element['#token_options']['sanitize']) && !$element['#token_options']['sanitize']) {
    return html_entity_decode($output);
  }
  return $output;
}