Search API Compound

Search backends match whole tokens. That makes a title like Stadtpolizei invisible to a search for Polizei: the word is there, but it sits inside a compound. Prefix search only reaches the front of a word, so it finds Polizeiposten and not Stadtpolizei, and where a backend offers substring search it usually scores those hits at zero — the document turns up somewhere on page five, which for a visitor is the same as not turning up at all.

This module splits compound words while indexing and stores the parts in a second field, so the match becomes an ordinary token match that ranks normally:

Stadtpolizei        →  stadt, polizei
Stadtpolizei        →  stadt, polizei                                                                                                           Kantonspolizei      →  kantons, polizei
Abfallkalender      →  abfall, kalender                                                                                                                    Einwohnerkontrolle  →  einwohner, kontrolle

The module is backend-agnostic: the derived field is an ordinary multi-value string field, so Solr, Typesense, the database backend and the rest all handle it without special support. The automated tests run against the database backend.

Requirements

• Search API
• One language submodule, plus the library it needs (see Installation). Both libraries are suggested rather than required: the module installs without them, and the submodule tells you what is missing instead of silei>

Installation

composer require drupal/search_api_compound

# German
composer require vanderlee/syllable spell-checker/dictionary-de
drush en search_api_compound_de

# English
composer require loupe/matcher
drush en search_api_compound_en

Configuration

1. On the Processors tab of your index, enable Compound parts (German) or Compound parts (English).
2. On the Fields tab, every string field in the index now offers a second property, labelled … (compound parts, de). Add the ones you want. Adding the field is the on switch — there is no separate setting, because the field UI already asks exactly that question.
3. Reindex.
4. Tell your backend to search the new field, and weight it below the source field so that a page actually titled Polizei still beats Stadtpolizei. With Typesense, for example:

fulltextFields: [
  { name: "title", weight: 10 }
  { name: "title_compound_de", weight: 8 }
  { name: "body" }
]

The language code is part of the field name on purpose. Search API merges property definitions by key and attributes the survivor to a single processor, so if both language submodules offered the same name, one of them would never run and every item in its language would stay unsplit.

How it works

Two rules keep the output from turning into noise, and both ma

A word must decompose completely. Every part th nothing left over. This is stricter than Lucene'sDictionaryCompoundWordTokenFilter, which emits every dictionary word it finds at any position and therefore turns Polizeiposten into polizei plus the stray osten.

A word may only be broken where hyphenation allows a break — the approach of Lucene's HyphenationCompoundWordTokenFilter.
This rules out seams that are not morpheme boundaries without nee Masterplan has no break after Mast, so it cannever become Mast plus Plan, and Bahnhofstrasse has none before trasse. Names survive it too: a name only comes apart if every part is a word in its own right, so Hamburg and Stuttgart stay whole.

Parts go into their own field rather than being appended to that lets you weight them, and it keeps the stored value of thesource field — the one your frontend displays — free of stray words.

Performance

The German splitter loads a 380,000-word list into memory the ut a word it has not seen: about 34 MB and 60 ms. After that a word costs 0.003 ms. Results are cached per word, so once an index has been built, saving a node normally does not load the dictionary at all.

Language support

Shipped: German and English.

Not shipped, deliberately: French, Italian and Spanish. Those languages build compounds analytically — poste de police, posto di polizia, puesto de policía — so a plain whitespace tokenizer already produces the terms a visitor searches for, and there is nothing left to split. The closed compounds that do exist are few, and they split into non-words: parapluie → para + pluie, tournevis → tourne + vis, paraguas → para + guas. In every case the left half is not a term anyone would search for, so a decompounder would cost precision and gain no recall.

The difference is not marginal. In a German word list, 23.6% of the entries are 15 characters or longer (90,417 of 382,300) and almost all of those are compounds; in an Italian one the figure is 4.4%, and most of those are derivational suffixes rather than compounds. The Italian community's own list of compound words runs to 419 lines, mostly hyphenated forms and English loanwords.

English is looser than German

English decompounding is delegated to loupe/matcher, which ships its own dictionary and test suite.

No length threshold is layered on top, because measuring one showed it discards as many good splits as bad ones — airport and bedroom would go with them — trading one kind of noise for a loss of recall. Suppressing a fragment that bothers your site is what the stop words hook is for.

Contributing

Bug reports and patches are welcome in the issue queue. The module ships 42 automated tests; the English ones skip themselves when the library is absent rather than failing, so phpunit is meaningful with only the German dependencies installed.