Problem/Motivation
Drupal core provides 2 logger modules: dblog and syslog. The former has simple but nice UI, yet slows the system and is not recommended for production. The latter is performant, but does not provide any UI at all.
One of contrib-space alternatives to core logging is Monolog module, that works with different storage backends like file system, elastic/opensearch and others.
The hosting platform I'm working with currently, does not support Elastic, but I can get a Solr instance.
Which led me to the question, why not have generic plugin for monolog for any search_api-compatible backend? Which led to the next question - why not have generic LoggerInterface plugin that would work with any search_api compatible backend?
Steps to reproduce
- Install Search API module (obviously)
- Configure at least one Search API server
Proposed resolution
Create a search_api_logger module (either as a submodule of search_api itself, or as a separate project - I'm open to both) that
- Exposes SearchApiLog implementation of LoggerInterface
- Exposes Log datasource plugin
- Exposes LogEntry DataType plugin and accompanying datadefinition plugin
- (optionally?) exposes dummy tracker that does not track anything, since all the data is stored in search_api
Remaining tasks
Implement all of the above :)
Issue fork search_api-3475570
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3475570-logger
changes, plain diff MR !169
1 hidden branch
Issue fork search_api_solr-3475570
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3475570-solrlog
changes, plain diff MR !48 /
changes, plain diff MR !47
- 3475570-solronly
compare
Comments
Comment #3
mkalkbrennerAs discussed on Slack, I like the idea to log into Solr. So to build an alternative to elastic, logstash, kibana that is integrated in drupal and allows queries, facets, etc.
It might seem to be an advantage to implement it as common functionality in search_api to provide it as functionality for any backend.
But it adds overhead. For example the tracker. It creates additional database queries.
I would prefer a super lightwight and fast implementation of the LoggerInterface that directly writes to Solr using solarium, leveraging a connection defined in search_api_solr. That backend is based on the DocumentDatasource for foreign indexes.
And a predefined View could be used to display the log. Facets will work as well. Or custom queries written with Search API.
That could be packed and added as sub_module to search_api_solr, including the server config and the View. So the user just has to adjust the credentials and connection settings.
Comment #4
valthebald@mkalkbrenner: PoC of lighter, Solr-only logger implementation https://git.drupalcode.org/sandbox/valthebald-3476169
TL;DR; where possible, I followed dblog patterns, replacing database specifics with solarium queries
Comment #5
valthebaldComment #6
mkalkbrennerI like this approach.
What do you think of adding it as sub-module of search_api_solr?
I have some more suggestions, put I need to play with module first.
Comment #7
valthebald@mkalkbrenner sure, I will create a MR later today
Comment #13
valthebaldComment #14
valthebald@mkalkbrenner did you have a chance to check this?
Comment #15
valthebaldI have created https://www.drupal.org/project/solrlog as a separate project, to allow installations without patching search_api_solr
Comment #17
mkalkbrenner