Setup

  • Solr version: 7
  • Drupal Core version: 8.9.13
  • Search API version: 8.x-1.19
  • Search API Solr version: 4.1.11
  • Configured Solr Connector: Acquia Connector 3.0.2

Issue

Content like the following, which is just a snippet, overwhelms the part of SearchApiSolrBackend that adds boost_term, to the extent that the boost term values exceed the float value limit of PHP, which results in boost terms like word|INF which makes the node impossible to index because Solr rejects it:

<p><a href="/library/hs/periodicals/a-ak">A-AK</a>, <a href="/library/hs/periodicals/al-au">AL-AU</a>, <a href="/library/hs/periodicals/b-d">B-D</a>, <a href="/library/hs/periodicals/e-h">E-H</a>, <a href="/library/hs/periodicals/i-m">I-M</a>, <a href="/library/hs/periodicals/n-r">N-R</a>, <a href="/library/hs/periodicals/s-z">S-Z</a></p>

<p><strong>Journals List:  S - Z</strong></p>

<p><strong>Saalburg Jahrbuch.</strong><br />
<strong>            Holdings</strong>: v.1- , 1910- . Index v.17-42.<br />
<strong>            Lacks</strong>: v.4.<br />
<strong>            Location</strong>: Compact Shelving &mdash; Lower Level<br />
</p>
<strong>Saeculum.</strong><br />
<strong>            Holdings</strong>: v.1- , 1950- .<br />
<strong>            Location</strong>: Current issues in Periodicals Reading Room<br />
                            Bound issues in Compact Shelving &mdash; Lower Level<br />
<br />
<strong>Saggi e memorie di storia dell&#39;arte.</strong><br />
<strong>            Holdings</strong>: v.1, 1957; v.5, 1966; v.6, 1968; v.7, 1970; v.8, 1972.<br />
<strong>            Location</strong>: Compact Shelving &mdash; Lower Level<br />
<br />
<strong>Saint-&Eacute;tienne. Universit&eacute;. Centre Jean Palerne. M&eacute;moires.</strong><br />
           <strong>SEE</strong>: <a href="i-m#M%E9moires+%28Centre+Jean+Palerne%29">M&eacute;moires (Centre Jean-Palerne)</a>.<br />
</p>
...

The full HTML is here. Here is part of the devel output for boost term:

    [boost_term] => Array
        (
            [0] => Historical|3.0 Studies|3.0 Social|3.0 Science|3.0 Library|3.0 Journals|3.0 SZ|3.0
            [1] => AAK|9.0 ALAU|9.0 BD|9.0 EH|9.0 IM|9.0 NR|9.0 SZ|9.0
            [2] => Journals|18.0 List|18.0 S Z
            [3] => Saalburg|72.0 Jahrbuch|72.0
            [4] => Holdings|288.0
            [5] => v1|576.0 1910|576.0 Index|576.0 v1742|576.0
            [6] => Lacks|1152.0
            [7] => v4|2304.0
            [8] => Location|4608.0
            [9] => Compact|9216.0 Shelving|9216.0 Lower|9216.0 Level|9216.0
            [10] => Saeculum|18432.0
            [11] => Holdings|73728.0
            [12] => v1|147456.0 1950|147456.0
            [13] => Location|294912.0
            [14] => Current|589824.0 issues|589824.0 in|589824.0 Periodicals|589824.0 Reading|589824.0 Room|589824.0 Bound|589824.0 issues|589824.0 in|589824.0 Compact|589824.0 Shelving|589824.0 Lower|589824.0 Level|589824.0
            [15] => Saggi|1179648.0 e memorie|1179648.0 di|1179648.0 storia|1179648.0 dellarte|1179648.0
            [16] => Holdings|4718592.0
            [17] => v1|9437184.0 1957|9437184.0 v5|9437184.0 1966|9437184.0 v6|9437184.0 1968|9437184.0 v7|9437184.0 1970|9437184.0 v8|9437184.0 1972|9437184.0
            [18] => Location|18874368.0

As you can see, the boost term values increase rapidly. Boost values can become astonishingly large and eventually they exceed PHP_FLOAT_MAX and the following happens:

            [886] => v1|101120238836005269809773416981882641266011205065504744716304420651224630140594291762148518493854239011880064057427658763683069244958109350152226679734704194962494440051486835467919998525700927942192359160547815744971459758817635370829076124679215320904384662260079792759970325840394840421469887935413626077184.0 1962|101120238836005269809773416981882641266011205065504744716304420651224630140594291762148518493854239011880064057427658763683069244958109350152226679734704194962494440051486835467919998525700927942192359160547815744971459758817635370829076124679215320904384662260079792759970325840394840421469887935413626077184.0
            [887] => Location|INF
            [888] => Current|INF issues|INF in|INF Periodicals|INF Reading|INF Room|INF Bound|INF issues|INF in|INF Compact|INF Shelving|INF Lower|INF Level|INF

Comments

cilefen created an issue. See original summary.

cilefen’s picture

cilefen commented
Related issues: +#3085299: Pipe character in a boosted field breaks indexing
cilefen’s picture

cilefen commented
Issue summary: View changes
cilefen’s picture

cilefen commented
Issue summary: View changes
cilefen’s picture

cilefen commented
Title: boost_term values can run wild » boost values can exceed PHP_FLOAT_MAX which breaks the boost_term
Issue summary: View changes
cilefen’s picture

cilefen commented

The effect of these enormous boosts values affects this module however the root issue may be in search_api in HtmlFilter::parseHtml:

        $inner_boost = $boost * ($this->configuration['tags'][$m[2]] ?? 1);
        $ret = array_merge($ret, $this->parseHtml($text, $m[2], $inner_boost));

I don't understand why this is multiplying the boost in a recursive call.

cilefen’s picture

cilefen commented
Title: boost values can exceed PHP_FLOAT_MAX which breaks the boost_term » Because of multiplying inside recursion, HTML Filter's tag boost can produce astronomical boost values which besides being nonsensical, breaks the boost_term Search API Solr by exceeding PHP_FLOAT_MAX
Project: Search API Solr » Search API
Version: 4.1.11 » 8.x-1.x-dev
Component: Code » General code
cilefen’s picture

cilefen commented
Title: Because of multiplying inside recursion, HTML Filter's tag boost can produce astronomical boost values which besides being nonsensical, breaks the boost_term Search API Solr by exceeding PHP_FLOAT_MAX » Because of multiplying inside recursion, HTML Filter's tag boost can produce astronomical boost values, which besides being nonsensical, breaks the Solr boost_term in Search API Solr by exceeding PHP_FLOAT_MAX