HTML filter leaves whitespaces reloaded [#3081180]

There is an issue #2179755: HTML filter leaves whitespaces about whitespace and HTML filter but this unfortunately was not solved properly.

If for example you have this text in body field:

<h2>Introduction&nbsp;</h2>
<p>Introduction</p>

and the "Correct faulty and chopped off HTML" text processor is enabled in text filter, the   will be replaced by some special char that trim(search_api/includes/processor_html_filter.inc:116) can't remove when "HTML filter" processor is enabled in Search api filters.

The output from above text with "Correct faulty and chopped off HTML" text processor(checking the $text variable in search_api/includes/processor_html_filter.inc:111):

 <h2> introduction  </h2> 
  introduction

This untrimmed space leads us to the following error:

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '85213-body:value-introduction' for key 'PRIMARY': INSERT INTO

I created a patch to fix this.

Comment	File	Size	Author
#5	3081180-5--html_filter_breaking_spaces.patch	2.36 KB	drunken monkey
#5	7.x-1.x: PHP 5.3 & MySQL 5.5, D7 24 pass
#5	3081180-5--html_filter_breaking_spaces--interdiff.txt	2.76 KB	drunken monkey
#2	search_api-html_filter_whitespace-3081180-2.patch	2.18 KB	mibfire
#2

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

13 September 2019 at 15:11

mibfire created an issue. See original summary.

Comment #2

mibfire CreditAttribution: mibfire commented 13 September 2019 at 15:15

File	Size
search_api-html_filter_whitespace-3081180-2.patch	2.18 KB

The patch that solves this issue.

Comment #3

donquixote CreditAttribution: donquixote commented 13 September 2019 at 16:29

Issue summary:

View changes

Comment #4

mibfire CreditAttribution: mibfire commented 13 September 2019 at 17:37

Issue summary:

View changes

Comment #5

drunken monkey

he/him

German

Vienna, Austria

CreditAttribution: drunken monkey as a volunteer commented 5 October 2019 at 14:17

Component:	Framework	» Plugins
Priority:	Major	» Normal
Status:	Active	» Needs review

File	Size
3081180-5--html_filter_breaking_spaces--interdiff.txt	2.76 KB
3081180-5--html_filter_breaking_spaces.patch	2.36 KB
7.x-1.x: PHP 5.3 & MySQL 5.5, D7 24 pass

1 file was hidden/shown/deleted

File	Size
search_api-html_filter_whitespace-3081180-2.patch	2.18 KB

Thanks for reporting this issue and providing a patch! You’re right, seems we hadn’t really taken non-breaking spaces and other “exotic” whitespace into account.
However, your solution seems a bit verbose. Also, if we are using the exact same code three times, we might want to split it off into its own helper method. (Can include the html_entities_decode() call there, too, it seems.)

Please test/review my attached revision and see if it still resolves the problem for you!

PS: In the future, please also remember to set status to “Needs review” when posting a working patch. (And please don’t misuse the “Priority” field!)

Comment #6

30 October 2019 at 11:12

drunken monkey committed 21e6121 on 7.x-1.x authored by mibfire

Issue #3081180 by mibfire, drunken monkey: Fixed HTML filter sometimes...

Comment #7

drunken monkey

he/him

German

Vienna, Austria

CreditAttribution: drunken monkey as a volunteer commented 30 October 2019 at 11:12

Status:

Needs review

» Fixed

Comment #8

mibfire CreditAttribution: mibfire commented 31 October 2019 at 11:55

Status:

Fixed

» Active

Also, if we are using the exact same code three times, we might want to split it off into its own helper method. (Can include the html_entities_decode() call there, too, it seems.)

I was thinking on the same but i was not sure where i should put it to. I didn't want to extend the current class because i thought it is a global function that we might also use somewhere else.

I checked your patch but you don't remove the spaces between words. Is there always only one word? So couldn't we have something like: "word1(double spaces)word2"?

I also checked that on my profile page that i have already 1 credit in "Search API" but i am not sure that it is this one or something else what i did earlier. I think i should have 2 with this one.

Thanks

Comment #9

drunken monkey

he/him

German

Vienna, Austria

CreditAttribution: drunken monkey as a volunteer commented 31 October 2019 at 15:02

Status:

Active

» Fixed

I checked your patch but you don't remove the spaces between words. Is there always only one word? So couldn't we have something like: "word1(double spaces)word2"?

I don’t think that’s true, preg_replace('/\s+/u', ' ', $token) should replace all whitespace with a single classical space.

I also checked that on my profile page that i have already 1 credit in "Search API" but i am not sure that it is this one or something else what i did earlier. I think i should have 2 with this one.

You can actually view the exact issues by clicking on “View all issue credits”. As you see (and I also see in the module’s commit log) it doesn’t seem like you were credited for another Search API issue yet.