This problem we notice is caused by having leftover junk in the index, the clear all indexed data isn't complete.
Visible symptom:
Search for a phrase or keywords "word1 word2" that has more than one page of results.
The items visually counted in the page do not match the expected number of items.
Displaying results 1 to 20 of 138 results.
We count using our eyes, 9 results on a page instead of 20.
burst a few brain cells and a few hours.
Then we remember, oh, we did a sql-drop and re-loaded from another dump file.
So, go to the search_api interface, clear the index, re-index everything.
SAME PROBLEM!!
Burst a few more brain cells and a few more hours.
Then get a brainwave, use the solr web ui Documents tab on the selected core to clear out the index.
<delete><query>*:*</query></delete>
Now that's done, index everything again:
ok now we go back to the search interface, count the results per page, it now matches the expected number of items per page.
Now, the reason why I am raising this issue is that for Developer Experience reasons, it seems as if the search_api clear all indexed data ui is misleading, it is not clearing out all of the drupal indexed contents. Is there a way that the DX (developer experience) and or end user experience can be improved here? It's kind of a normal thing to have to take a dump from prod and restore it to say a dev / qa or a local environment where a site was likely previously indexed and yet the clear all indexed data functionality is insufficient to clear out noise that the search_api_solr module is processing.
So, we do have a workaround thankfully for this situation which is to delete the indexed data via the solr ui + Documents tab using the xml mentioned above then re-indexing. The reason for this issue is to find a way to improve the clear index functionality in a way that does not cause regressions for cases where multiple index sources go into the same core. Maybe there's a signature or something such as sitename or indexname that the clear all indexed data functionality could rely on that would avoid a regression but yet sufficiently clear index ?
| Comment | File | Size | Author |
|---|---|---|---|
| #16 | Search-API-Solr-Admin-tip.png | 47.69 KB | ressa |
| indexing_completed.png | 31.69 KB | joseph.olstad | |
| delete_indexed_data.png | 15.95 KB | joseph.olstad |
Comments
Comment #2
joseph.olstadComment #3
joseph.olstadComment #4
joseph.olstadComment #5
joseph.olstadComment #6
joseph.olstadComment #7
mkalkbrennerDeleteing all is too risky. Some people index data from different site into one collection or core.
The behaviour to limit the delete to a virtual index ID is a Search API design decision.
DDEV is taking care about droping all Solr content.
Or use the search_api_solr_admin module to delete everything.
Comment #9
ressaThere are a few tips on How to delete Solr Core and Collection, or all documents, which I created last year, after also having to clear data, perhaps it could be useful?
It would be great to get the documentation page added to the menu, so that users visiting https://www.drupal.org/docs/8/modules/search-api-solr/search-api-solr-ho... can discover it.
Comment #10
mkalkbrennerThank @ressa. I just "reviewed" the howto. It should be vissile in the menu shortly.
Comment #11
ressaThank you @mkalkbrenner, it's great to see it in the list and in the menu. I very much appreciate you maintaining Solr, it's an amazing tool.
Comment #12
joseph.olstadI still think we should offer a delete all button with warning info bubble. Administrator only button
Comment #13
mkalkbrennerIt exists in the admin module. You just have to enable it as it is a sub-module. And you will also get the drush command.
I dont't want to get this functionality into the main module. And when you're using Solr Could, which you must with Solr 10, you will enable the admin module anyway as a developer.
Comment #14
ressaIf everyone needs to enable the Admin module anyway, why not just include it? I don't get it.
Comment #15
mkalkbrennerEvery developer, not every user.
Enable it in your dev environment and maybe on your backend or drush server, but not on a frontend webserver that just uses an index or contributes to a multi-site setup.
Like you probably should disable all the *_ui core modules on the production frontend.
Comment #16
ressaThanks for a fast response @mkalkbrenner. And I see ... I struggle to imagine a scenario where it could be problematic, but I trust you have good reasons to sticking to keeping the Admin module features separately.
But if moving Admin tools into the standard Search API Solr module will not happen, we could alert the users, since they probably want to install the Admin module as well, for the extra features. We could check if Admin module is installed, and show an alert above if not?
And actually, I have become uncertain what the Admin module adds. The
search_api_solr_admin.info.ymlsays:But I am not exactly sure which tasks, because if I uninstall it, I still see the same three buttons under an index, and even "Delete all indexed data on this server" under a Solr server page ...Oops, I was looking at a regular DB, not Solr, sorry! I see an extra "Delete collection" button.
See: https://www.drupal.org/docs/8/modules/search-api-solr/search-api-solr-ho...
It also provides Drush commands, and it would be amazing if it was documented which features and Drush commands it adds, and I created #3587376: Document Search API Solr Admin features.
@mkalkbrenner:
Are you saying I should see an additional button in the interface, and if so, where?
@joseph.olstad: Just to make sure, did you see this "Delete all indexed data on this server" button under your Solr server, but it did not work for you?
Comment #17
ressaI have added my alert suggestion ("You may want to install the Search API Solr Admin module...") in #3529563: Alert users if Search API Solr Admin module is not installed.