Strings being served through REST server in JSON format have UTF-8 special chars being escaped.

For instance, in hook_services_request_postprocess_alter() we have:

$result = array(6 => 'Español');

And the response sent through the service is:

{"6": "Espa\u00f1ol"}

Possibly related issues:
#2072923: Unicode characters appearing in JSON messages with PHP version 5.4 and Services module
#2198731: Russian JSON output is escaped

CommentFileSizeAuthor
#12 2285191-12.patch1.11 KBtyler.frankenstein
#7 special_chars_escape-2285191-6.patch668 bytesAlbert.Liu
#3 Fix-warning_array_diff_key_with_conditional_fields_2279491-3.patch3.07 KBmarcoscano
#1 SERVICES-utf8-chars-escaped-in-JSON-2285191-1.patch613 bytesmarcoscano

Comments

marcoscano’s picture

marcoscano
Primary language Portuguese, Brazil
Location Barcelona, Spain
commented
Status: Active » Needs review
StatusFileSize
new SERVICES-utf8-chars-escaped-in-JSON-2285191-1.patch613 bytes

The problem apparently is in json_encode() inside the ServicesJSONFormatter->render().

Not sure if this is the good approach, but for simple data arrays, this conversion seems to work.

Status: Needs review » Needs work

The last submitted patch, 1: SERVICES-utf8-chars-escaped-in-JSON-2285191-1.patch, failed testing.

marcoscano’s picture

marcoscano
Primary language Portuguese, Brazil
Location Barcelona, Spain
commented
Status: Needs work » Needs review
StatusFileSize
new Fix-warning_array_diff_key_with_conditional_fields_2279491-3.patch3.07 KB

sorry, too fast...
we need to check if the data being returned is an object or multi-dimensional arrays, before performing array_map().

still not sure if this is the right approach, but well, for my use case, where the responses will be always a single value or a one-dimensional array, it seems to work

Status: Needs review » Needs work

The last submitted patch, 3: Fix-warning_array_diff_key_with_conditional_fields_2279491-3.patch, failed testing.

heine’s picture

heine commented

The escape sequence is allowed per the JSON specification. Why is it a problem? Is your consumer not conformant?

marcoscano’s picture

marcoscano
Primary language Portuguese, Brazil
Location Barcelona, Spain
commented
Category: Bug report » Support request
Status: Needs work » Closed (works as designed)

@Heine, you are right!
sorry, I'm not very familiar with the json specification, and instead of verifying it first, I just assumed that if my consumer requested non escaped strings I should provide this.

My bad, apologies for that.

Albert.Liu’s picture

Albert.Liu commented
Version: 7.x-3.7 » 7.x-3.5
Assigned: Unassigned » Albert.Liu
Status: Closed (works as designed) » Needs review
Issue tags: +REST API, +json, +services
StatusFileSize
new special_chars_escape-2285191-6.patch668 bytes

Hi @marcoscano, I just face this problem, and I fixed that with JSON_UNESCAPED_UNICODE parameter in json_encode.

Please see my patch file and give a try.

Status: Needs review » Needs work

The last submitted patch, 7: special_chars_escape-2285191-6.patch, failed testing.

heine’s picture

heine commented

Albert.Liu, the escape sequence is allowed per the JSON specification. Can you explain why it is a problem?

ka3ax’s picture

ka3ax commented

got the same problem with german umlaut. After setting the Header to "Content-Type text/plain; charset=UTF-8" no worries about that...

aamir@pwdigitas.com’s picture

aamir@pwdigitas.com commented

I am calling api of third party service that returns simple JSON which is valid json.
The json decode in drupal throws error due to special characters. What could be the issue?

tyler.frankenstein’s picture

tyler.frankenstein commented
Status: Needs work » Needs review
StatusFileSize
new 2285191-12.patch1.11 KB

I've combined some of the efforts of the latest patch here with a patch from #3056883: PHP 5.3 incompatibility: JSON_PARTIAL_OUTPUT_ON_ERROR into the following patch. This may also be a solution for #3103614: Form errors return corrupted characters..

Does this patch fix it for you?