Add IndexNow capability to search engine integration

Hi. I wrote the referenced “unofficial post” about the Bing deprecation, and advocated for the adoption of IndexNow. I’d just like to add one comment regarding implementing the IndexNow in Drupal.

Please don’t hardcode a search engine to use with IndexNow. As stated on the IndexNow website, if you submit to URLs to one IndexNow-compatible search engines then that search engine will immediate forward it to other IndexNow-compatible search engines. So, Drupal can help maintain competition in the search engine market by not favoring any one search engine over another. If everyone submit their URLs to just Yandex or Bing it can skew the market, which is easily solved by picking a random provider from a list. E.g.

 $indexnow_endpoints = [
  'www.bing.com/indexnow',
  'yandex.com/indexnow',
  // 'www.google.com/indexnow' and others in the future
];

array_rand ( $indexnow_endpoints ); 

Ideally and to also future-proof things a bit, failed submission requests can be reattempted using a different entry from the list.

Oh, another implementation note that I jut ran into: Yandex’s API returns HTTP 202 Accepted instead of HTTP 200 OK as stated in its documentation.

Hi @gbyte, I added two commits to the issue fork, please review. Changes include code cleanup and handling of summary of IndexNow setting.

Also pay attention to the following points:

1. Perhaps a separate entity for IndexNow is redundant. Maybe it would be better to implement new functionality inside the simple_sitemap_engine entity?
2. I don't fully understand why the IndexNow key can be stored in both config and state. Can we choose only one location?
3. "indexNow" should be changed to "IndexNow" in all descriptions to avoid confusion.

@gbyte Just a note for code simplicity :)

->condition('index_now_url', NULL, 'IS NOT NULL')

can be replaced by

->exists('index_now_url')

If the backwards compatibility is preserved and there are no changes related to core_version_requirement - the best is to go with 4.0.2. imho.

For example... when you drop the D8 support then 4.1.0 release makes sense.

The question remains how drupal.org lists 4.1.x on the release page in relation to 4.0.x

Disclaimer: I have a little maintainer's experience.... so I might be wrong. :)

In our case we have the the 4.x branch. And both 4.0.1 and 4.1.0 releases are releases of the same 4.x branch. So, I expect the 4.1.0 to replace 4.0.1 at the project page automatically. The similar example for this is the Admin Toolbar module.

If we would have 4.0.x and 4.1.x branches created separately... then both these branches and their releases would be displayed at the project page one above the other. The example for this is the Webform module.

I agree with @devad. Releases are related to branches, so the new release should replace the current one.

https://www.drupal.org/node/1015226
https://semver.org

Based on the documentation, the release should be version 4.1.0.

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.

@gbyte I think tests can be implemented within this issue #3105922: Fix tests for simple_sitemap_engines

Well apparently that's not the case - minor releases create new entries on the project page. Only marking the previous release as unsupported hides it.

Thanks for the reply. It's good to know. It gives more flexibility to maintainers to hide or show the previous minor release which is good.

Thank you so much for implementing this so quickly!

I'm not sure I understand the procedure though. Is this how it's done?:
1. generate tag (through bing or throug the ui of the module)
2. ideally add the tag to settings.php
3. add a file called .txt containing the tag to the root folder (web) -> this one is unclear

Is the last step necessary? Does the file really have to go in the web folder, and not in the private one or so? If so, it would perhaps be a good idea to add this to the explanation in the message on the UI?

Oh, even better! It seemed too easy to be true.
In that case perhaps a simple 'you're all set!' after the instructions would already suffice :-)
Thanks!

It's supposed to be "simple" sitemap after all.

Mission accomplished I'd say. :-)

Which instructions are you referring to though?

The system message you get when generating a key. The one saying the key is now saved in Drupal State, but you should consider saving it in settings.php.