We have a requirement for allowing Metatag keywords to be searched and considering writing a patch that adds a Metatag submodule adding support for this, such as in https://drupal.stackexchange.com/questions/167438/how-to-add-metatag-mod... & http://agileadam.com/2015/11/adding-metatag-keywords-to-search-api-index/

Is this something you'd consider adding to Metatag?

CommentFileSizeAuthor
#77 metatag-n2692877-77.interdiff.txt646 bytesDamienMcKenna
#77 metatag-n2692877-77.patch9.58 KBDamienMcKenna
#74 metatag-n2692877-73.patch8.95 KBDamienMcKenna
#71 metatag-n2692877-71.patch388 bytesDamienMcKenna
#68 metatag-n2692877-68.patch9.55 KBDamienMcKenna
#62 2692877-62.patch9.55 KBstefan.r
#58 2692877-57.patch6.38 KBstefan.r
#57 testresults.png18.85 KBstefan.r
#54 2692877-52.patch9.82 KBstefan.r
#54 interdiff.txt998 bytesstefan.r
#51 2692877-51--search_api_integration.patch9.81 KBdrunken monkey
#51 2692877-51--search_api_integration--interdiff.txt2.16 KBdrunken monkey
#48 field.png166.89 KBstefan.r
#48 alt.png117.08 KBstefan.r
#44 metatag-n2692877-43.patch9.87 KBstefan.r
#44 interdiff-42-43.txt405 bytesstefan.r
#42 interdiff.txt4.12 KBstefan.r
#42 metatag-n2692877-42.patch9.87 KBstefan.r
#26 metatag-n2692877-26.patch9.07 KBDamienMcKenna
#24 metatag-n2692877-24.patch7.35 KBDamienMcKenna
#21 metatag-2692877-21.patch8.55 KBstefan.r
#13 metatag-2692877-13.patch8.34 KBstefan.r
#10 metatag-2692877-10.patch8.31 KBstefan.r
#7 metatag-2692877-7.patch8.04 KBstefan.r
#4 metatag-2692877-4.patch8 KBstefan.r
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

stefan.r created an issue. See original summary.

stefan.r’s picture

Issue summary: View changes
DamienMcKenna’s picture

I would *definitely* be interested in a patch for that!

stefan.r’s picture

Status: Needs review » Needs work

The last submitted patch, 4: metatag-2692877-4.patch, failed testing.

The last submitted patch, 4: metatag-2692877-4.patch, failed testing.

stefan.r’s picture

Status: Needs review » Needs work

The last submitted patch, 7: metatag-2692877-7.patch, failed testing.

The last submitted patch, 7: metatag-2692877-7.patch, failed testing.

stefan.r’s picture

Status: Needs review » Needs work

The last submitted patch, 10: metatag-2692877-10.patch, failed testing.

The last submitted patch, 10: metatag-2692877-10.patch, failed testing.

stefan.r’s picture

Status: Needs review » Needs work

The last submitted patch, 13: metatag-2692877-13.patch, failed testing.

The last submitted patch, 13: metatag-2692877-13.patch, failed testing.

stefan.r’s picture

My tests are green locally, I think there is an issue with the testbot. It is not fetching the dependencies (entity, search_api) as it is reading the dependencies from test_dependencies in the .info file before applying the patch, rather than after:

https://dispatcher.drupalci.org/job/default/106677/console

stefan.r’s picture

DamienMcKenna’s picture

Yeah, that's the problem, I see it all the time. I'll give it a spin locally. Thanks.

DamienMcKenna’s picture

Would you mind reworking this so that it applies to the main module? I don't think it needs to be a submodule.

stefan.r’s picture

Yes, will do.

stefan.r’s picture

Status: Needs review » Needs work

The last submitted patch, 21: metatag-2692877-21.patch, failed testing.

The last submitted patch, 21: metatag-2692877-21.patch, failed testing.

DamienMcKenna’s picture

Title: Add Search API support for Metatag keywords » Add Search API support
Status: Needs review » Needs work

After reviewing the functionality, this needs to be extended to use metatag_get_info() to list all meta tags provided by the module, rather than hardcoding it to only work with the useless 'keywords' field.

DamienMcKenna’s picture

DamienMcKenna’s picture

Status: Needs work » Needs review
stefan.r’s picture

Looking good so far!

I'm willing to look into adding support for other fields (our specific requirement was just the keywords field), but for now should we limit ourselves to adding just Page title, Description, Abstract and Keywords?

Does this look good to you, other than the missing support for other fields?

The last submitted patch, 24: metatag-n2692877-24.patch, failed testing.

The last submitted patch, 24: metatag-n2692877-24.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 26: metatag-n2692877-26.patch, failed testing.

The last submitted patch, 26: metatag-n2692877-26.patch, failed testing.

DamienMcKenna’s picture

Yes, this is definitely going in the right direction, it just needs to dynamically load the meta tags via the APIs rather than hardcoding the definitions for a few tags.

stefan.r’s picture

Just to be clear: all of them? i.e. everything under metatag_get_info()['tags'] ?

Please can you point me to the right API hooks you'd like me to use.

DamienMcKenna’s picture

Thinking on it further, is the goal to index the strings that are being manually entered, i.e. overridden values that would be found in the {metatag} table, or the final processed meta tags that would appear on the HTML page?

stefan.r’s picture

In our case, the manual metatags... Not sure what the point would be of indexing the <meta> tags?

DamienMcKenna’s picture

Sorry, I didn't mean the actual HTML tags, but the final values with all tokens processed.

The next question is: should it exclude values that are just tokens?

Dave Reid’s picture

+++ b/metatag.module
@@ -2974,6 +2974,19 @@ function metatag_translations_delete($metatags, $context) {
 /**
+ * Implements hook_search_api_alter_callback_info().
+ */
+function metatag_search_api_alter_callback_info() {
+  return array(
+    'search_api_metatag_alter_callback' => array(
+      'name' => t('Metatag keywords'),
+      'description' => t("Adds the item's Metatag keywords to the indexed data."),
+      'class' => 'MetatagSearchAlterCallback',
+    ),
+  );
+}

This should be able to go into metatag.search_api.inc as well, since search_api implements hook_hook_info().

stefan.r’s picture

Ah! Definitely the processed values then -- makes much more sense when searching.

Why would we want to ignore token-only values?

DamienMcKenna’s picture

The question about tokens was related to the fact that just pulling the $entity->metatags[LANGCODE][METATAG]['value'] variable will give you the meta tag values before they're processed, so there could be raw tokens. To get the final meta tag strings we'd need to take a different approach, and I'm honestly not sure off-hand the best way of doing this.

DamienMcKenna’s picture

Thinking on it further, I think should be handled as a custom data type via hook_search_api_data_type_info rather than a data alternation thing. It might then list all available meta tags individually, or maybe it could work as a "related field" that then makes all of the tags available?

stefan.r’s picture

Status: Needs review » Needs work

The last submitted patch, 42: metatag-n2692877-42.patch, failed testing.

stefan.r’s picture

The last submitted patch, 42: metatag-n2692877-42.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 44: metatag-n2692877-43.patch, failed testing.

The last submitted patch, 44: metatag-n2692877-43.patch, failed testing.

stefan.r’s picture

FileSize
117.08 KB
166.89 KB

Screenshots, for illustration:

Data alteration toggle, which will make the Meta tag fields appear:

alteration

(from hook_search_api_alter_callback_info() docs: "Registers one or more callbacks that can be called at index time to add additional data to the indexed items (e.g. comments or attachments to nodes), alter the data in other forms or remove items from the array.")

Fields:

fields

stefan.r’s picture

@DamienMcKenna, is this close to what you are looking for? Or still waiting for @drunken_monkey feedback?

DamienMcKenna’s picture

Status: Needs work » Needs review

Yes, this is definitely the kind of improvement I was looking for, thank you.

I'll test it out next week.

drunken monkey’s picture

I don't know about the Metatag-specific code, of course, but as far as integration with the Search API is concerned, the latest patch looks quite good. Just a few things I noticed:

  1. +++ b/metatag.search_api.inc
    @@ -0,0 +1,78 @@
    +    // Only works on entities.
    +    return $index->datasource() instanceof SearchApiEntityDataSourceController;
    

    You can use (bool) $index->getEntityType() instead. That's more portable than checking for a specific class.

  2. +++ b/metatag.search_api.inc
    @@ -0,0 +1,78 @@
    +    $tags = metatag_get_info()['tags'];
    

    I could be mistaken, but I don't think that's valid in PHP 5.2.

  3. +++ b/metatag.search_api.inc
    @@ -0,0 +1,78 @@
    +        if (isset($item->metatags[$item->language][$tag])) {
    

    This will trigger an error if $item->language isn't set.

Those should all be fixed in the attached patch. Regarding field_get_items(), I'm not completely sure – it should do what you want, but I'm not 100% certain. Please try it out.

The test looks also sensible, though I wonder why you're using the UI for creating the server and node, but not for the index. Personally, I'd create all three programmatically, but at least it should be consistent – in my opinion. But not a real "error", of course, so that's up to you.

Status: Needs review » Needs work

The last submitted patch, 51: 2692877-51--search_api_integration.patch, failed testing.

The last submitted patch, 51: 2692877-51--search_api_integration.patch, failed testing.

stefan.r’s picture

Status: Needs work » Needs review
FileSize
998 bytes
9.82 KB

@drunken monkey many thanks for the review, changes look great to me!

field_get_items() doesn't seem to work here though (as field_language() === FALSE), so one of the tests failed. Maybe just a check for $item->language being set instead?

Status: Needs review » Needs work

The last submitted patch, 54: 2692877-52.patch, failed testing.

The last submitted patch, 54: 2692877-52.patch, failed testing.

stefan.r’s picture

Status: Needs work » Needs review
FileSize
18.85 KB

Re-roll, tests are still green:

results

stefan.r’s picture

Forgot to attach the actual reroll, here it is :)

stefan.r’s picture

Triggering a re-test

DamienMcKenna’s picture

Did you forget to include the metatag.search_api.test file? I don't see it in patch #57.

DamienMcKenna’s picture

Status: Needs review » Needs work
stefan.r’s picture

Status: Needs work » Needs review
FileSize
9.55 KB

Status: Needs review » Needs work

The last submitted patch, 62: 2692877-62.patch, failed testing.

The last submitted patch, 62: 2692877-62.patch, failed testing.

DamienMcKenna’s picture

I've re-queued the last patch for review.

DamienMcKenna’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 62: 2692877-62.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 68: metatag-n2692877-68.patch, failed testing.

stefan.r’s picture

Oops! Passes locally though, doesn't it?

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
388 bytes

Ah! It's because we're adding new test dependencies, I forgot about this issue.

DamienMcKenna’s picture

The patch in #71 will add the test dependencies, we'll commit that and then do a new patch with the rest.

  • DamienMcKenna committed 9bdb9da on 7.x-1.x
    Issue #2692877 by stefan.r, DamienMcKenna, Drunkey Monkey: Search API...
DamienMcKenna’s picture

FileSize
8.95 KB

And the final tests, which should work fine.

DamienMcKenna’s picture

BTW I updated the test file to match the other "with [modulename]" tests, but otherwise it's 100% yours.

Status: Needs review » Needs work

The last submitted patch, 74: metatag-n2692877-73.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
9.58 KB
646 bytes

Forgot to add the metatag.info changes back again.

Status: Needs review » Needs work

The last submitted patch, 77: metatag-n2692877-77.patch, failed testing.

stefan.r’s picture

Woohoo! Thanks :)

The last submitted patch, 77: metatag-n2692877-77.patch, failed testing.

DamienMcKenna’s picture

The testbot failure are being caused by #2737555: Test failures in contrib modules when other projects include forked versions of said contrib modules, so hopefully that'll be fixed soon.

BTW would you mind adding a section to the README.txt file explaining how to use the new functionality? I think that'd really help people a lot. Thanks.

DamienMcKenna’s picture

The tests are passing again (thanks drumm!), so I think a little extra bit to the README.txt file and it'll be good to go.

DamienMcKenna’s picture

Status: Needs work » Reviewed & tested by the community

Duh, there's already some instructions in the README.txt file.

DamienMcKenna’s picture

Status: Reviewed & tested by the community » Fixed

And it's committed! Woohoo! Great work, stefan.r!

DamienMcKenna’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.