When using this module for trimming text on a specified amount of words, it works only on texts with latin characters, because it uses php's str_word_count function to determine the number of words. This function does not truly support utf-8 characters.

While it's not a perfect solution for all alphabets maybe it is better to split words by a regex looking for space characters.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

drupov’s picture

I am adding a patch that tries to solve this. Please review. Thanks!

drupov’s picture

FileSize
952 bytes

Just saw that the module is using preg_split to put the words into an array on order to trim it, so why not reuse this logic for the count of words too.

I am updating the patch to use that logic.

judapriest’s picture

Hello,

Could you provide a example of what you mean by "latin characters" and "utf-8 characters" ? Because your patch seems to be fine, but without it, it seems to work fine too (I do prefer your version with preg_split :p). But I think with preg_split you should also take into account returns (\n\r, \n).

Here how I did my test , http://generator.lorem-ipsum.info/ :
- Cyrillic
- Hindi
- Japanese

I tested smart trim with those 3 languages, on a fresh drupal 7.30 and simplytest.me, with lastest dev version (Note that japanese don't have much space when written). Cannot reproduce your probleme for now.

Jieyyal’s picture

Do not work with Chinese, number, symbol mixed string!

Bensbury’s picture

Doesn't word with Japanese - it won't obey the trim length for character or word.
Fresh Drupal 7.38

-- Patch worked by the way.

egontinno’s picture

Patch #2 also needed for Russian text.

RogerRogers’s picture

This patch doesn't apply cleanly any longer. However, I've manually applied the lines and can confirm this works. It would be great if this can make it into the release cycle because it is required for non latin languages, in this case Persian.

Chris Matthews’s picture

Assigned: drupov » Unassigned
Status: Needs review » Needs work
Issue tags: +Needs reroll

The 5 year old patch in #2 to smart_trim.module does not apply to the latest smart_trim 7.x-1.x-dev and needs to be re-rolled.

jigish.addweb’s picture

Status: Needs work » Needs review
FileSize
935 bytes

@Chris Matthews, Here's the patch on the same kindly review it.

Chris Matthews’s picture

Priority: Major » Normal
Issue tags: -Needs reroll
Parent issue: » #3017949: Plan for Smart Trim 7.x-1.6 release