Dirty and queued items don't get removed from the tracking table when deleted

I fixed this by upgrading to latest dev, disabling the index then enabling it again. Seems to work fine now.

Popped up again today, using latest dev of sapi an entity api.

Deleting nodes using node_delete and recreated (with another node type and structure) with node_save re-using the same nid by setting $node->is_new = true. I have also been experiencing situations where it will not index all nodes. It stays at eg 3331 of 3410 total, and then I have to disable an re-enable the index. Selecting re-index does not help. I'm thinking these issues could be related?

WD node: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '12043-2' for key 'PRIMARY': INSERT INTO {search_api_item} (item_id, index_id, changed) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1,
:db_insert_placeholder_2), (:db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array
(
    [:db_insert_placeholder_0] => 12043
    [:db_insert_placeholder_1] => 2
    [:db_insert_placeholder_2] => 1
    [:db_insert_placeholder_3] => 12043
    [:db_insert_placeholder_4] => 3
    [:db_insert_placeholder_5] => 1
)
 in SearchApiAbstractDataSourceController->trackItemInsert() (line 490 of sites/all/modules/contrib/search_api/includes/datasource.inc).

Deleting nodes is still giving me problems, even when deleting from from the UI.

I just noticed that re-indexing was not done correctly. I was seeing a higher count of a node type than I knew existed (as a faset) after deleting a lot of nodes of that type. Clicking on that facet gave me a WSOD.

This is on a view in a panel, no caching on the view or panel is enabled:
Trying to clear the index using "drush sapi-c" did not help. Even when 0% of the index was indexed it would return all items and all facets.
Disabling and reenabling the index, reindexing all 8989 items (a pain without batch indexing) did not help.
Event tried deleting the index manually, using "curl http://localhost:8983/solr/<core-name>/update?commit=true -d '<delete><query>*:*</query></delete>' did not help - it would still show all items in the view. Clearing the cache did not help.

Finaly managed to clear the index using the "Empty index"-button on index status UI. Reindexing and everything seems to work again. The I would delete a node, and the view would still show the previous count on that node type and clicking on it now gave me a WSOD again. Cleared the cache manually and the facet updated its cound and the view worked correctly. So there are some issues here with (re)indexing and cache.

Ok, here is what I did:

1) Emptied the index in the UI
2) Ran "drush sapi-i 1 1000" x 3 times to index 3000 of about 9000 items
3) Looked at the view, facets shows counts that looks about right
4) Indexed the remaining items up to 100%
5) Looke at the view. Facet counts increased, but were not showing all items.
6) cache clear all fixed number 5 - or perhaps 120 seconds did.
7) Facet for node type shows 6 items of one of the node types. Clicking it lists all six.
8) Deleted one item of this node type.
9) Facets still shows 6 items, but index has been reduced by 1.
10) Clicking on the facets now produces a fatal error. "Fatal error: Cannot access empty property in /home/quickstart/websites/nbk.dev/modules/field/field.attach.inc on line 314"
11) Waited 120 pluss seconds.
12) Refresh. Fasets now shows 5 items.
13) View works as expected, no fatal errors.

As for what is going on in the delete functions, i step-debugged it.

a POST is sent to Solr with the message "article_index-30889" and response "{"responseHeader":{"status":0,"QTime":66}}". So looking good.

BUT! A bit earlier, TrackItemDelete on the datasource controler runs a query:

db_delete($this->table)
->condition($this->itemIdColumn, $item_ids, 'IN')
->condition($this->indexIdColumn, $index_ids, 'IN')
->condition($this->changedColumn, 0)
->execute();

table is search_api_item, $item_ids is an array containing the entity id, index_ids is an array containing numeric index id's. I have two indexes on this dev server, and one of the indexes does not seem to work at the moment - stuck at 150 indexed items. Looking in the database the "changed" columns is set to -1 for all items except the 150 it has indexed. So this item-index combo does not get deleted. I guess that would cause a duplicate item if I tried to recreate the node with the same NID?

Patch from #1225926-4: Fix performance problems in indexing workflow fixed the views "Fatal error"-issue!

Now that we have established that duplicated entries is a result of TrackItemDelete not deleting items with the changed column set to -1, - how would we go about to fix this?

Is there a good reason why these records are not deleted when this columns is not set to 0? Or could I just remove the line/condition and all will be well?