Can't filter out certain characters like ¡! ¿?, they lead to error in line 116 of module [#1914098]

As title says, I can't configure natural sort to filter out certain characters like ¡ ! ¿ ?. Trying to do it, I'm getting this error:

There was an HTTP AJAX error. Resulting code HTTP: 500 More detailed information about the issue. Paht: /batch?id=540&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: in _views_natural_sort_store_node() (line 116 of /home/content/97/9512397/html/sites/all/modules/views_natural_sort/views_natural_sort.module).

Regards.

Comment	File	Size	Author
#6	views_natural_sort-utf8_encode_transformed_data-1914098-6.patch	364 bytes	generalredneck
#6

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

generalredneck

him/his

English

Texas, USA 🇺🇸

CreditAttribution: generalredneck commented 11 February 2013 at 15:19

Status:

Active

» Postponed (maintainer needs more info)

If you wouldn't mind providing me with the logged error from your server's log I would be grateful. If you don't have access to that, then maybe if you create a piece of content, most likely with the characters in question, you may see the exact full error.

I've tried special characters such as these in my setup as well and I don't have any problems. You can actually see that here.

Let me know what you turn up!

Thanks

Comment #2

generalredneck

him/his

English

Texas, USA 🇺🇸

CreditAttribution: generalredneck commented 11 February 2013 at 15:22

I thought about it for a second, your second line for more detailed information may be from the server logs. With that said, for some reason there is an issue storing to the database. Does it work with out those characters? Can you store those characters as titles?

Trying to figure out what is different between your setup and mine.

Comment #3

NoRandom CreditAttribution: NoRandom commented 11 February 2013 at 23:34

I'm having problems with the error logs, they are empty. I'm going to investigate futher.

About that special characters in titles, yes, I can use them. Actually I already had some nodes with them before the installation of Natural Sort.

Regards.

PS.- Extra information: Problematic symbols are "¡" and "¿" wich are the symbols we use in spain to start exclamations and interrogative sentences. On the other hand, "!" and "?" work perfectly.

Comment #4

NoRandom CreditAttribution: NoRandom commented 12 February 2013 at 00:35

It seems the problem is in line 184 of the module. Original content:

$regex[] = '/[' . preg_quote($symbols) . ']/';

Even this simple regular expression doesn't work:

$regex[] = '/[' . preg_quote('¿') . ']/';

I've tried with /[?]/, /[\?]/, /[\x{00BF}]/ and they neither work.

The simplest workaround I found is to simply put in this line "non word" characters or opening punctuation character:

$regex[] = '/\W/';

$regex{} = '/\p{Ps}/';

Comment #5

generalredneck

him/his

English

Texas, USA 🇺🇸

CreditAttribution: generalredneck commented 12 February 2013 at 21:41

Ok I was able to reproduce it by hardcoding $symbols = '¿'; I will have to take a look at this and see what I can figure out.

Comment #6

generalredneck

him/his

English

Texas, USA 🇺🇸

CreditAttribution: generalredneck commented 22 February 2013 at 22:21

Status:

Postponed (maintainer needs more info)

» Needs review

File	Size
views_natural_sort-utf8_encode_transformed_data-1914098-6.patch	364 bytes

I'm sorry it took so long for me to get back around to this problem... Here's what I found out... first, make sure your tables are set to use Collation utf8_general_ci. Next, utf8 is pretty much the standard when using drupal, other people have had simular problems when they build tables with multiligual data. might checkout #1140194: SQLSTATE[HY000]: General error: 1366 Incorrect string value for a field with accents.

Lastly, I'm making a patch that will utf-8 encode data put into my table. Lets see if this helps the problem along... it seems to have done so on my end.

Comment #7

NoRandom CreditAttribution: NoRandom commented 23 February 2013 at 14:21

You don't have say sorry for anything ;)

It works perfectly and I have my database configured in utf8_unicode_ci. I don't know why it was configured this way and if it could cause any kind of problem.

Thanks a lot for all your efforts and support. I've found another minor problem, going to open a new post.

Regards.