Add support for non-numeric wildcards during validation

Here's a patch that performs this validation. Realising that running preg_match() across all the tokens is inefficient, the code is only invoked if there are invalid tokens after normal validation.

Update patch for flattened token structure.

Can add tests if approach is approved by maintainer.

Go, bot, go.

Change to require characters in the wildcard part of the token, not just the right prefix. Added test cases for both validation, and token-value

The current code loops across each of the tokens found in the input string (small number), and checks if they're in the list of available tokens (big number) - using isset() or a single preg_match() for the _N wildcard. This is very efficient.

However, in order to check an input-token against a wildcard available-token, we need to

• find the wildcard tokens in the list of available tokens
• match these against the list of tokens from the input string

So, the extra loop is actually around the *available* tokens, not the list of tokens being validated (probably not clear from the patch). By putting it below the existing loop:

• we don't do ANY extra work unless there are actually tokens not found by the current code
• only the input-tokens that are previously unmatched are checked

Here's just the token.module patch with a bit more context.

Updated patch to use new assertTokens() functionality.

OK, works for me. Once this is applied, I'll submit a patch for Date module.

PS: the first line of token_test_exit() generates a warning for me. What's that function?

function token_test_exit() {
  debug($debug);
  if ($debug = variable_get('token_page_tokens', array())) {

My only thought about -? vs -* is that '?' kindof implies a single letter/number, whereas * makes it clear that there could be quite a bit of stuff after the * (such as provided by date module). This may just be my geek/regex interpretation of that, of course.

Actually, the regex in your patch

elseif (preg_match('/^(.*[_-])([^-_])+$/', $token, $matches)) {

only supports having ONE word-part (as delimited by _-) after the wildcard, which really limits what a module can use the wildcard for. Are you sure?

Your call, of course - but I disagree with "Anything that really wants to do this right should be using D7". Of course D7 has better token support, but most sites are still D6, and have no realistic chance of upgrading or being implemented in D7 (so many contrib modules not updated, in particular).

Regarding your comment on performance - that's exactly why I put the code in a second loop, that is only applied if there are tokens that are still invalid after primary checking.

It's fine, but doesn't solve my original problem: "proper" wildcard support.

Thanks. I've updated the patch for Date module to not create multi-word elements in the wildcard part of the tokens.

http://drupal.org/node/593840#comment-4457364