Patch for handling unicode in freetagging in drupal.js [#44939]

The current version of drupal.js's autocomplete feature doesn't handle unicode(somebody can probably give a better explanation). All I know is that freetagging's autocomplete feature does not support Korean and I presume other unicode languages.

As thox (#drupal) suggested, I changed the escape() to encodeURIComponent() in drupal.js and now autocomplete works now. I'm submitting a patch for review as thox asked.

Comment	File	Size	Author
	drupal.js_4.patch	459 bytes	John Hwang

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

Thox CreditAttribution: Thox commented 16 January 2006 at 12:01

It may also be worthwhile to wrap encodeURIComponent() around the i variable on the same line.

The patch should also fix accented characters, which is essentially the same problem with the escape() function. Can a JS/unicode wizard give an opinion?

Comment #2

Steven CreditAttribution: Steven commented 16 January 2006 at 16:20

What browser are you using? Arbitrary Unicode characters work fine for me in both Safari and Firefox.

Comment #3

John Hwang CreditAttribution: John Hwang commented 17 January 2006 at 15:42

Without the encodeURIComponent(), it doesn't work for Korean (presumming Chinese and Japanese as well). Even with the new function swap, it still doesn't work in IE. Korean Freetagging autosearch does work in Firefox.

Comment #4

Steven CreditAttribution: Steven commented 22 January 2006 at 17:36

Status:

Needs review

» Fixed

This patch only affects HTTPPost, and autocomplete only uses HTTPGet so it's logical that it doesn't fix it ;).

But, testing shows that escape() is indeed the wrong function to use. It uses %NN for unicode characters < 255 and %uNNNN with utf-16 words. encodeURIComponent() on the other hand uses %NN-escaped UTF-8 bytes. So HTTPPost needs to be changed anyway.

The issue with autocomplete is that IE doesn't deal properly with non-ASCII characters in XMLHttpRequest URIs. They get converted to "\xNN". So we need to escape the autocomplete query with encodeURIComponent() as well.

I tested this on IE6, Firefox, Opera and Safari and now arbitrary unicode works everywhere.