Invisimail Filter does not work and causes body to disappear when email domain is longer than 8 characters

sub

Dear All,

I can confirm the above bug, having just made the switch from spamspan to invisimail, as the latter supports mail icon links, whereas spamspan always renders the e-mail address as text. I tried all the different Invisimal options, as well as different filter arragements to no avail.

As this is a critical issue (I have to disable invisimail for now) it would be wonderful to find a solution. Thank you so much for all your great help.

Chris

I can confirm the same bug.

Same here... subscribing.

Ditto - subscribing

As far as I have been able to tell this module doesn't work at all. I have tried every permutation of filter arrangement and all the config options to no avail. The closest I've been able to get is by enabling javascript which adds a snippet to the page that appears to be encoded however it is NOT visible to the browser because it is commented out! I can only see it upon viewing the source of the page. Even worse is the plain unencoded email is displayed directly after this invisible javascript snippet.

This was working on my drupal install until my host updated to PHP 5. Now I get the blank pages but it seems to randomly happen across the site. It has also happened with much shorter domain names. In one example, abc.co.uk would be ok, abcd.co.uk wouldn't.

I think I may have discovered the issue here, as I've been experiencing the same issue, but not consistently on domain or eMail address length, but rather on post length. Can anyone else do a bit of testing to confirm that?

I think the underlying issue is that the backtrack limit is hit with preg_replace_callback in the invisimail_filter function. And unfortunately, when that happens, preg_replace_callback returns NULL. However, if you inspect the results of a call to preg_last_error() right after the call to preg_replace_callback, the return value is 2, which is "backtrack limit hit". See http://bugs.php.net/bug.php?id=40846 for more on this.

So, given making changes to php.ini and PCRE library bits and pieces in a shared hosting environment will be a bit tricky, we need a work around to 'chunk' strings into smaller bits if we're going to still use preg_replace_callback (and even then, the bit's might be a bit too big), or we need an alternative to that function.

One way of doing it may be to, in the invisimail_filter function, parse $string into relatively short strings, using a space as a delimiter, operating on the shorter strings, and then reassembling the string. What really matters for this filter is that eMail addresses remain whole in the shorter strings.

I'll work on a patch for this, and report back.

Shawn

Update: so the bit of HTML that is causing me issues and returning preg_last_error() of 2 is <a href="mailto:info@ClearlyByDesign.com" rel="nofollow">Contact us</a>. And interestingly, now that I look at it, there is no "mailto" or anchor text on pages that are working, just regular eMail addresses. And, much longer text is passing through whereas this short bit is not... so more investigation underway...

Shawn

OK, may be narrowing this down, at least for me. Question for all who are having issues: the pages / nodes where the filtered output is blank... do they have constructs something like <a href="mailto:some-address@some-domain>somthing that is not some-address@some-domain</a>?

The call to preg_replace_callback seems to be failing on that for me. And just to make things a bit weirder, while that seems to be failing, a construct like <a href="mailto:info@ClearlyByDesign.com" rel="nofollow">Contact us</a> at info@ClearlyByDesign.com... works, with both the "Contact us" and "info@ClearlyByDesign.com" being obfuscated. What it seems to have picked up on was the <a href="mailto:info@ClearlyByDesign.com" rel="nofollow">info@ClearlyByDesign.com in the string, and then just did the string replace on the whole matched string.

So, it seems like preg_replace_callback is failing when $pattern looks like something like the above. Unfortunately I'm not a regex expert, so I'm struggling a bit with why $pattern is so complex. Why would it not be simply $pattern = "@$mail@$modifiers";, and then in the callback function invisimail_callback, just return invisimail_ascii_encode($matches[0], $format['js'], $format['link']);? That would pick up any eMail address in a 'mailto' link as well as regular ones on the page, wrapped in anchor tags or not. Or am I missing something?

Shawn

I feel like I'm overposting... so I've narrowed the 'sypmtoms' down to

1. the link text is not the same as the eMail addy (had to patch the pattern to actually make this work... see below), and
2. the domain of the eMail addy is over 13 characters long (13?? don't ask me!)

Does this match anyone else's experience (please patch the pattern and try it.

To patch the pattern, substitute the $pattern assignment with

  $pattern = "@(?:(<a [^>]*href=['\"](mailto:$mail)['\"][^>]*>)?((?>(?<!mailto:))$mail))|(<a [^>]*href=['\"](mailto:$mail)['\"][^>]*>(.*?)</a>)@$modifiers";

Explanation:

1. The first part of the patten (the bit that starts with '(?:' and ends at the '|') is the original pattern that didn't allow for link text to be different from the eMail addy.
2. The second part of the pattern allows for link text that is something other than the eMail addy.

In addition, you'll need to modify the function invisimail_callback as follows:

function invisimail_callback($matches) {
  $format = $GLOBALS['invisimail_format'];

  // If $matches[1] is empty, no link portion was matched for this item, so
  // it's a simple replace operation, unless there was an <a> tag but the
  // link text and eMail did not match, in which case we need to replace the
  // eMail address in the <a> tag.
  
print 'invisimail_callback: $matches=';var_dump($matches);
  if (empty($matches[1])) {
    if ( empty($matches[4]) ) {
      return invisimail_ascii_encode($matches[3], $format['js'], $format['link']);
    }
    else {
      return str_replace($matches[5], _invisimail_encode_string($matches[5], FALSE),
          $matches[0]);
    }
  }
  // We DO have an existing link portion. Do our best to preserve it; that means
  // ignoring the js setting, since it makes for heinous string transformations.
  return str_replace(array($matches[2], $matches[3]),
    array(_invisimail_encode_string($matches[2], FALSE), _invisimail_encode_string($matches[3], FALSE)),
    $matches[0]);
}

which will take care of the additional part of $matches. The var_dump call is there so I can see (a) that the function is actually called, and (b) what's in $matches.

After 13 characters of a domain where the link text is not the same as the eMail addy, the call-back isn't called... no results from var_dump... and the return code from preg_last_error() is 2.

So... any ideas anyone?

OK, after doing a fair bit of investigation, I've come up with the attached patch that modifies the invisimail.module as follows:

1. Split the pattern into two bits, and do two replacements. The first bit is geared toward the case where the link text is different from the mailto: eMail address. The second pass uses the original pattern, but modified to be much less greedy.
2. Split the text contained in $string into chunks to permit better monitoring (this is configurable with a setting in the filter settings).
3. Add watchdog entries if the call to preg_replace_callback fails so we can at least see what failed where.
4. Modified the function invisimail_callback to handle the case where the link text is different from the mailto: eMail address.

This now seems to be working for me in all cases, with long domain names, etc. Would really appreciate it if anyone could take a look / test this and see if you get similar results.

Shawn

Subscribing

First, this should have been marked "needs review" since there is a patch.

Second, the patch didn't apply properly because it wasn't rolled relative to the module root.

Third, holy crap that's an epic patch! :-) I don't understand everything it's doing (especially the regex, which is still black magic to me), but I just tested it and it works in every situation I can find, including some from other issues. It also vastly improves the documentation, too.

I cleaned up the coding style a slight bit (a typo or two and whitespace usage, etc.) and committed #12. Thanks a bunch, sdsheridan!

I guess I'll reopen this one. Hope the title change is OK - at least it is visible on the module main page now.

preg_last_error() is unavailable in PHP 5.1. So people shouldn't upgrade unless they're sure of their PHP version.

Sorry, no patch included. I'ma deal with resulting issues on my client's site now ;)
May have a look later.

PHP < 5.2 hasn't been supported in a long time. Drupal 7 requires it, and a lot of contribs as well. I don't even have access to a pre-5.2 server, and 5.3 is becoming increasingly common. I didn't realize preg_last_error() was only available in 5.2 so it's not specified in the module, but I'll add a note to the project page.

Restoring issue information as this is the wrong thread for such discussion.

My pleasure :-)