Add a new Search API processor plugin (ProcessorPluginBase) that allows the rendered output of each meta tag to be indexed, separate to the existing typed data plugin that only indexes values from the optional entity field.
The plugin should list all available meta tags with checkboxes (or a multi-value select list?) to control which ones are indexed, with the items grouped.
In https://www.drupal.org/project/metatag/issues/2901039, Search API integration was added to the Metatag module to allow for the indexing of entity metatags to a Search API backend.
But, this is not working as expected.
The issue is that the current Search API integration with Metatag module relies heavily on this computed field:
https://git.drupalcode.org/project/metatag/-/blob/8.x-1.x/src/Plugin/Fie...
The main issue with this implementation is this logic done in a preSave() method, that checks if the metatag values at the node level differ from the metatag defaults defined for the content type. This is an issue because if the node level metatags are the same as the defaults, the metatag value is not saved to the metatag computed field for the entity. But, it seems that the Search API integration is using that metatag computed field to get the metatags and their values to pass along to the search API index during indexing.
When this happens, you will notice that backer class of IndividualTag will not trigger any XDebug breakpoints nor index any of the metatags as expected, in the scenario that the node level metatags are the same as the defaults.
If the node level metatag values differ from the defaults, then the breakpoints will work and the metatags will be indexed. If the node field has the same value as the default configuration, the value is removed from what is stored for that node.
The Metatag field storage doesn't store the system defaults in the field, so that the defaults can be changed later if needed without having to worry how to update all of the per-entity changes.
We might need to rework the Search API integration to not be driven by the Field API field itself but rather just work from supported entity types via a separate property.
| Comment | File | Size | Author |
|---|---|---|---|
| #34 | metatag--2024-09-09--3315049--mr-140.patch | 13.75 KB | nuuou |
| #29 | 3315049-metatag-searchapi-support-29.patch | 11.34 KB | nuuou |
| #28 | metatag-n3315049-27.patch | 13.5 KB | kmitch |
| #25 | metatag-n3315049-24.interdiff.txt | 475 bytes | benabaird |
| #25 | metatag-n3315049-24.patch | 14.27 KB | benabaird |
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #2
damienmckennaComment #4
damienmckennaThanks to borisson_ for providing the recommendation to create a processor plugin.
Comment #7
damienmckennaThis is patch #9 from #2901039: Add a TypedData plugin to support Search API that tsplash and mErilainen worked on.
Comment #8
joshua.boltz commentedThis patch is a first attempt at a more flexible processor plugin for the Search API + Metatag integration.
Using this patch, a new Processor can be enabled.
With it enabled, a user can configure with metatags to include.
The metatags that get included in this configuration become made available as fields to add to the Fields listing for indexing.
Comment #9
joshua.boltz commentedI noticed an issue pretty quick when the value was multiple-value, like og_image, it was setting Array as the value. So, added logic in this patch to check if its an array, and if so, build a comma-separated string of the values.
Comment #11
damienmckennaImproved.
I don't know why you need to enable the processor and then add the field, seems like a bad UX?
Comment #13
joshua.boltz commentedNice @DamienMcKenna, I like how the metatags are now organized into their groupings in the processor configuration.
Yeah I figured there would be more ideas around the UX, but the idea is that you enable the processor, which gives you the config to choose which metatags you wish to allow/expose to the index. But, then you actually have to add those allowed/exposed fields to the index to set things like machine name, type, boost, etc. Unless you know of a more streamlined way of doing it, that is just what worked for me when developing the patch, but I am open to input for sure.
Comment #14
damienmckennaNew idea: this should include an option to control whether the rendered output should be used or if it should only include the values filled in on the node itself.
Comment #15
damienmckennaThis is going into v2.
Comment #16
damienmckennaFor continuity, let's add it to 8.x-1.x.
Comment #17
damienmckennaWorking on this.
Comment #18
damienmckennaAn option for controlling whether just the entity's values or all of the defaults are loaded, along with test coverage and improvements to the UI.
Comment #20
damienmckennaImprovements.
I posted a question in the Search API queue (#3332764: Should ProcessorPluginBase::addFieldValues() process data prior to calling addValue()?) as it's not working as I expected.
Comment #22
damienmckennaLeaving this for now.
Comment #23
berdirDrive-by comment: We implemented a custom processor that skips entities that have noindex set on metatags, because in pretty much all cases that we've seen, content that isn't indexed publicly should also not be indexed in your internal search, things like thank you pages and so on.
Probably doesn't really fit in this issue but I could look into creating an issue with our code if there's interested for that.
Comment #24
damienmckennaThat's an interesting idea, it kinda skips the need for search_api_exclude by using the checkbox from Metatag.
I'd be happy to include that, if you'd be willing to throw it into a new issue. Thank you!
Comment #25
benabaird commentedThanks for the patch, it's working nicely. I did find an issue with the "all metatags including defaults" option, it looks like the config value isn't being read. Attached a patch and interdiff.
Comment #26
damienmckennaComment #28
kmitch commentedA big thank you to everyone who's contributed to this issue! I (perhaps foolishly) upgraded directly from v1.22 to v2.0.0 and ran into a *lot* of search-api-integration issues with metatag fields, and this issue seemed to be the most current one to help me get back on track.
I'm not sure if this is still the right place to follow for official post-v2.0.0 search-api integration? Issue #3326104 seems to suggest that this current issue will be used to re-implement search-api integration in 2.x, but the recent patches and activity in this issue seem to be focused on the 1.x branch, and consequently the patches don't work as-is with the 2.x branch code. But perhaps that will be coming with future activity?
Regardless, for what it's worth I adapted the most recent patch from @benabird (metatag-n3315049-24.patch) to work with the 2.x branch. It seems to work as intended in a Drupal 9.5.10 (PHP 8.1.x) environment, but I suspect it has the same test failures that the #24 and previous patches triggered.
Comment #29
nuuou commentedRan into a similar situation as above, with a site that was previously using the old patches. Need to upgrade that site to the "new" metatags/Search API methodology eventually, but not today haha.
I re-rolled this patch for Metatag 2.0.2 support, since #28 doesn't apply anymore.
I did not include the README edits, since this approach shouldn't be supported anymore anyhow.
Comment #30
8bitplateau commented@nuuou what is the 'new methodology' you mention ? how do we achieve this without the patch ?
Comment #31
nuuou commentedY'know, I think was wrong about the "new methodology". There are quite a few related issues to this, I thought this was solved in a different way!
Looks like this is the issue to follow going forward on this.
https://www.drupal.org/node/3326104
Lemme re-roll that patch again with the README.
Also, tagging this to 2.x since that's the recommended version now.
Comment #32
nuuou commentedComment #34
nuuou commentedCreated an MR based on the above, against
2.0.x.Patch file created for folks who want that for Composer patches too.
Comment #35
nuuou commentedComment #36
idflood commentedThanks @nuuou, the merge request worked perfectly for my use case. I was trying to index keywords, and this patch made it work nicely : )
Comment #38
technotim2010 commentedHi I was able to apply the patch from #34 and it worked as planned. I was able to add a custom metatag to the search api fields during ingestion.
IMHO this should be committed to the module.
Comment #39
damienmckennaComment #40
damienmckenna