Problem/Motivation
The Tika extractor sets the locale to en_US.UTF-8 to avoid stripping UTF-8 multibyte characters. This probably works for most systems, however if that locale is not present the indexing fails - the extract function throws an exception Tika Exctractor is not available (which has a typo btw). AFAIK the C.UTF-8 locale is more likely to be present on a system, so would be a more reliable choice.
I've set the priority to Minor since I presume this is an unusual scenario: I'm using Lando on WSL (Windows Subsystem for Linux on Win10) with the Pantheon recipe. The only locales present on the container Lando builds are:
$ locale -a
C
C.UTF-8
POSIXSteps to reproduce
- Use a bare minimum Linux install, or remove the en_US.UTF-8 locale.
- Create a file with UTF-8 characters in the filename, eg 'Drupal’s contributors.docx' (character 0x2019 apostrophe)
- Index using the Tika extractor
Proposed resolution
Change line 39 and 56 of /src/Plugin/search_api_attachments/TikaExtractor.php to use the C.UTF-8 locale:
// setlocale(LC_CTYPE, 'en_US.UTF-8');
setlocale(LC_CTYPE, 'C.UTF-8');
// shell_exec("LANG=en_US.utf-8");
shell_exec("LANG=c.utf-8");
Issue fork search_api_attachments-3255205
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
izus commentedhi,
thanks for this catch
we maybe can add a check for the locale before using it
please feel free to suggest a patch/MR for checking en_US.UTF-8, else C.UTF-8 before setting locale
the typo can also be corrected here
Thanks for any contribution
Comment #3
gaddman commentedComment #5
gaddman commentedThanks Izus. I've created a merge request. A few notes:
shell_exec, which AFAIK wouldn't achieve the desired result.Comment #6
izus commentedok
thanks for this
will keep the issue open if someone wants to review and RTBTC
will come back to merge it in few days.
Comment #9
izus commentedThis is now merged in 8 and 9 branch
Thanks