Provide option to allow removing slashes (/) from tokens

Thanks for the work, committed.

It still doesn't work due to this in filefield_paths.module:383:

  // Process string tokens.
  $value = token_replace($value, $data, array('clear' => TRUE));

  $paths = explode('/', $value);
  foreach ($paths as $i => &$path) {
    …

Whenever a token e.g. [node:title] contains a '/', filefield_paths will create new folders according to node title, so if you have a node title "very/long/subdirectory", pathauto won't get a chance to clean this up and you end up with a very deep folder structure. I suspect you could do all kinds of wrong if you start using this for node title: "../../index.php" or worse. token_replace() needs to be run on each individual $path within the for-loop to make sure it's the folder structure the site admin has configured that will rule, rather than what an unknowing user inputs.

Folder separation should happen before token replacement, like this:

  $paths = explode('/', $value);
  foreach ($paths as $i => &$path) {

    // Process string tokens.
    $path = token_replace($path, $data, array('clear' => TRUE));
    …

Patch attached.

Nope. That wasn't enough. There is also this:

        $pathinfo = pathinfo($path);
        $path = str_replace($pathinfo['filename'], pathauto_cleanstring($pathinfo['filename']), $path);

So if [node:title] is "You Cant See Me/This Will Be Processed.ButNotThis", only "This Will Be Processed" will be sent to the cleaners. The replaced path will still have unprocessed parts of the file name with potential directory delimiters.

This new patch rebuilds the file name based on everything returned from pathinfo() and it also keeps #2047835: "Cleanup using Pathauto" Not Working Properly happy.

The question is whether users want to use tokens with an existing directory structure, but that would require additional configuration in the field form in the line of [ ] Keep directory structure from tokens. But I imagine this is a followup issue.

Aaand, pathinfo() sets 'dirname' to '.' when only file name is present which resulted in files starting with '.' (hidden). Added test for that too.

Steps to reproduce after clean install with 'standard' profile:

1. Enable the filefield_path and pathauto modules
2. Configure the 'field_image' field in the 'article' content type: admin/structure/types/manage/article/fields/field_image
3. Under 'File (field) Path Settings' set 'File Name' to [node:title].[file:ffp-extension-original] and enable 'Cleanup using Pathauto' under 'File Name Options'.
4. Create an article with title a/b/c/d and upload a random image.
5. Observe that the image path now is .../files/images/a/b/c/d.jpg
6. Apply patch in #6, repeat step 4 and observe the new path to be .../files/images/abcd.jpg, as expected.

This is kind of critical, so it would be nice if someone could confirm and/or review it.

@kaare,

Please note that this original issue is that the user wanted slashes in tokens not to be removed, and your issue is that you do want slashes in tokens to be removed. So what you see as a critical issue, others see as a feature, and therefore I am in a very difficult situation, to fix your issue I have to break things for others.

So what is the solution? I'm open to suggestions.

In the meantime, you could always make custom tokens that sanitize the node title as you see fit. Not ideal, but this issue does need further thought before it can proceed.

Also, this issue, and my last reply, inspired a new module; Rule Tokens.

It allows you to use a Rule component to process the value of a token creating a new token. So you could theoretically create a new token called something like [node:title:rule_tokens:rules_remove_slashes] that removed all slashes from the [node:title] token just using Rules for the logic.

Oh, I'm so sorry. Totally misinterpreted that. I totally see you being torn by this. '/' is the essence of this module, and how to interpret that when part of tokens is tough.

Though, my expectation on this is:

• Tokens used in the file_name part should not be given a chance to change the directory structure. It's unexpected. You configure the file name and expect it to reside in the directory configured in file_path.
• When ...['file_name']['options']['pathauto'] (Clean up using pathauto) is set, pathauto should orchestrate manipulation of the individual characters, including '/'. At least for file_name

So my suggestion would be to let tokens values orchestrate the directory structure in file_path, but not in file_name.

It is a very tricky situation indeed, I entirely see the logic in what you're saying, but I don't like making decisions that break things for others.

The correct way of tackling this issue is to make that logic configurable by the user, allow them to make that decision. That can be done with the Rule Token module I provided, at least for those willing to delve into Rules, and for the others, I will keep thinking on exactly what I want to do.

However, the plan for sometime now has actually been to rebuild FFP with Rules in a new module, and then make FFP a sub-module of the new module that provided the same existing functionality of FFP, the same interface, but used the new module as the engine, so maybe the Rule Token module may just be the way it's going to be.... I can't yet say.

@Deciphered
kaare's solution actually fixes the module to behave exactly how it is expected.

Pathauto should be cleaning up each tokenised value and then the rest of slashes should be used as path delimiters. I'm really struggling to understand how one would use this functionality otherwise (i.e. currently, if a token contains slash, the file is created in a folder).

Bumping this to critical since files get misplaced.

Provided patch addresses the issue in another way by processing each token separately in token replacement callback.

@alex.designworks,

Please refer to https://drupal.org/comment/7965831#comment-7965831, implementing this path will break things for other users. Until a solution can be determined that allows both usecases to be catered for I can not implement a change.

Also, that patch does not appear to be rolled from HEAD and reintroduces code that has been removed and is unrelated to this issue.

And lastly, this issue is not critical as currently this is works as designed and a feature request to support an alternate usecase, and there are workarounds currently available.

The initial patch https://drupal.org/files/1942720.patch works with replaced values instead of working with array of tokens=>values.

My patch uses callback that works with tokens=>values array directly. Although it does not have the logic to skip tokens that have ':join-path', it allows to add such logic to cover both cases _automatically_. This means that all occurrences of tokens with ':join-path' will continue having them replaced using native token module replacement mechanism; other tokens will have their values replaced using this module mechanism. In other words, the module will behave as expected, but will respect slashes in token values.

Also, the patch was rolled from 7.x. I'm not sure what other branch I should use.

Ok so about 6 month after last post ...
Is there any working solution to avoid slash owner token create subdirectories ?
eg : "My famous post 24 / 24 is all the day" (files/my-famous-post-24/24.jpg)

Thanks

This patch adds an option to remove slashes (/) from tokens. It then chooses when to processes string tokens based on that option.

Works for me. Review and see if this solves the above issues for everyone else.

@BWPanda: Excellent idea to have an option for this. This will solve the dilemma between existing and expected behavior. Though this patch needs some work:

• It doesn't work for any combination of Remove slashes … and Cleanup using Pathauto for both File path options and File name options. Using [node:title] as part of file name token and creating an article named "Sub/dir" still puts the image in .../sub/dir-<filename>.
• Using pathauto (which is the only way this patch will ever clean up the file path), this code will only clean up the file name part of the path, as I pointed out in #5, meaning this setting will never bite.

@Deciphered: Please respect the severity of this issue. Messing with subdirs in a file name is a bug and a potential security issue.

There are some questions still:

• Using pathauto to clean up, should Remove slashes actively remove them, or let pathauto take care of it? Or rather, let pathauto get the first attempt, then remove what's left?
• The complexity in order to be backwards compatible in filefield_paths_process_string() is doubled. This should probably be split up.

This is a working edition with backwards compatibility. It works by removing slashes after pathauto has called it a day, thereby letting the site admin use pathauto to select what character '/' should be replaced with.

The last patch is a combination of @BWPanda's work in #18 and the patch in #6.

Thank you kaare! This solution in #23 worked for me and was exactly what I needed.

#23 worked for me, thanks.
The bulk rename proces is also faster!

Thx to kaare! Works like a charm.
Any way to hook cleanup rules for tokens?

Anyone can update patch #23 for latest dev?

@UksusoFF,

No need for a re-roll, patch still applies to dev.

@kaare,

Thanks for the work, this meets all the requirements I had. I will be committing this at some point in the future (soon), but before I do so there does need to be some tests written. I will likely write them myself, but if you do feel so inclined, feel free to add them to the patch.

@Deciphered
I've unsuccessful tried apply this patch on previous dev version. On current version from 2015-Sep-08 it's work great.

@UksusoFF,

'patch -p1 < [patch]', works perfectly for me, but if it doesn't work for you then I'd suggest waiting till it's committed.

@Deciphered

On current version from 2015-Sep-08 it's work great.

Ah, misread. Good to hear.

Thanks for the patience and patches. Committed and pushed.