Deleted nodes do not result in any indication of stuff waiting to happen (0 nodes to index, 100% indexed). After a while node disappears from index though.

The wait for the commit might be be 4 min, or whatever the autocommit time is. Especially, when someone deletes/unpublishes stuff, it might be frustrating to still see it coming up in results without an indication on when is it gonna disappear.

The same stats.jsp page should have this info, such as:

deletesById : 1
deletesByQuery : 0

Thus, we shgould be able to grab and display them as we do with the pending docs.

Comments

gábor hojtsy’s picture

Peter says it is also possible to say what is the expected lag. That would be highly useful for the user to know.

pwolanin’s picture

Yes- we can display the autocommit time, since it would also be in the XML we have parsed.

pwolanin’s picture

Status: Active » Needs review
StatusFileSize
new2.31 KB
pwolanin’s picture

StatusFileSize
new37.99 KB

screenshot:

JacobSingh’s picture

Would be overcomplicating matters to track when the last update happened so we could actually say when it is supposed to be committed?

I suppose so :)

Anyway, what is there is a huge step up. Works for me.

Noyz’s picture

Looks like you have all the information here, but nothing it all blends together. I think users will care about whether search works, and if it doesn't work, how long will it take before its starts working again. IMHO, why its not working is secondary, as are number of terms/fields/documents in index.

Another way to address this is to hijack the deletion confirmation and append the string with "Search will be re-indexed in approximately 240 seconds." Or something like that?

pwolanin’s picture

Jeff - I don't really understand your comment about the deletion form.

Peobably the information here can and should be re-worded and better organized. If we want to have it avilable to use elsewhere, we should think about caching it as we are with the Luke info.

Note that the top line, however, comes form apacehsolr_search.module, while the remainder comes from apachesolr.module.

Noyz’s picture

Meaning when something is deleted, there's a confirmation. "X we deleted successfully" (or something to that effect). Change it to, "X was deleted successfully. The search index will reflect the deletion in approximately 240 seconds"

Noyz’s picture

StatusFileSize
new416.38 KB

Hopefully this visual will clarify.

pwolanin’s picture

StatusFileSize
new8.15 KB

This refactors a bunch of the code and moves the report to the reports section.

JacobSingh’s picture

StatusFileSize
new9.84 KB

@Noys: Since we have muliple slaves, it will be pretty difficult to tell them when all of the slaves are up2 date. This is a much larger story.

I've attached a new patch containing Peter's changes, plus the last two notifications you wanted with a slight modification.

Here is a screenshot with what will be shown when someone deletes / unpublishes a node:
http://img.skitch.com/20090327-8wf82ksjctymcb2s1thpq8xjqt.jpg

Assuming we have 2 days, and the current functionality how could we improve this?

Here is a screenshot of Peter's patch in action:
http://img.skitch.com/20090327-rjnbuus5gr7tkwffu5w3p835m2.jpg

The reports page is just like the old index-settings page minus the form at the bottom.

----------------------

@pwolanin: the patch looks okay to me.

pwolanin’s picture

I think any such extra notices should only show for admins and should have an option to disable them in the settings page - this is really about initial hand-holding IMHO.

Noyz’s picture

Re: http://img.skitch.com/20090327-8wf82ksjctymcb2s1thpq8xjqt.jpg. The Apache Solr message make sense, but can the message in sentence form vs bullet form? Assuming it's a bug to say the blog was deleted, then followed up with "the item was deleted"

Re: http://img.skitch.com/20090327-rjnbuus5gr7tkwffu5w3p835m2.jpg: Of the changes I see, I don't think they're significant enough. Looks like two items were replaced with one link.

JacobSingh’s picture

@Noyz: Sentence vs. bullet - No. One message is generated by node module, one by us. Can't really change that AFAIK.

re: not significant enough

Needs more info here. Given the constraints I mentioned (we can't get authoritative info on when all slave servers are caught up in any easy way), what should we do?

@pwolanin: I agree about the deletion messages, but I'm trying to follow Jeff's screenshots. I'm not totally adverse to it, although it adds to the problem of people ignoring messages. Anyway, it might help.

-J

Noyz’s picture

I believe Search starts producing accurate results as soon as one slave is updated - true? Can you benchmark off of one slave instead of all slaves?

pwolanin’s picture

@Noyz - potentially searches are distributed (unevenly) to slaves and master (depending on the load balancer config), while admin queries like this might go only to the master.

pwolanin’s picture

Status: Needs review » Needs work

" in aproximately 5 minutes"

Is no good - the whole point is to get the approximate time from Solr, not hard code it.

Noyz’s picture

Sorry, Peter, I'm not subscribed to this issue, so I don't know when you reply.

I think you're over engineeringthe solution. Users will not hold us accountable if the time isn't completely accurate. All they only need to know is if it's not working, and if its not, how long will it be until it starts working. To address the later, I believe the following options will work (so long as the data is not surrounded by a bunch of other data):

- ...Apache Solr will be re-indexed in approximately 5 minutes. //it may be wrong, it may be longer, but it at least tells them why it's not working. // you could even say 15 minutes, and if it comes in early they'll be happy.

- ...Apache Solr will be re-indexed in approximately [get the approximate time from Solr] ///more precise yes, but I thought you couldn't do this because you had no idea how long it would take to persist across slaves, and distribution is unevenly spread.

pwolanin’s picture

StatusFileSize
new15.29 KB

@Jeff - we can get the autocommit time - at that point the content should be gone from the master, though there may be unknown additional lag for slave replication.

Given that an admin can edit solrconfig.xml to set the autocommit time to anything for their Solr install (e.g. 10 sec to 10 min), it seems silly to assume the default value is always present.

pwolanin’s picture

Status: Needs work » Needs review
StatusFileSize
new15.45 KB

better patch - vs. 6.x HEAD

pwolanin’s picture

Status: Needs review » Fixed

committed to 6.x

pwolanin’s picture

StatusFileSize
new954 bytes

follow-up patch

pwolanin’s picture

committed

pwolanin’s picture

Status: Fixed » Needs review
StatusFileSize
new1001 bytes

Apparently the smpleXML object doesn't serialize/unserialize right.

This at least works.

pwolanin’s picture

Status: Needs review » Fixed

committed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.