Closed (fixed)
Project:
Search API
Version:
8.x-1.x-dev
Component:
Tests
Priority:
Normal
Category:
Bug report
Assigned:
Unassigned
Reporter:
Created:
15 Aug 2021 at 08:53 UTC
Updated:
9 Nov 2021 at 15:55 UTC
Jump to comment: Most recent, Most recent file
Comments
Comment #2
drunken monkeyComment #3
drunken monkeyComment #4
drunken monkeySeems the fail on Postgres is actually due to a Core bug: #1148856: Postgres schema doesn't support keylength on a unique index.
However, probably better to just change this for MySQL anyways, I guess? After all, it’s not 100% accurate and could, in very hypothetical scenarios, even lead to bugs (tasks not getting created since their
datahas the first 255 characters in common with another task’s).Comment #5
diegopino commentedYes, thanks this fix works.
But, personally, I feel using the whole data column as a key may be a design issue. Thinking loud here, it would be maybe better to apply an md5 hash on data itself, index that as an extra field/column and use that as unique to avoid double tracking. The hypothetical scenarios or something not being tracked because of a partial match do worry me specially because they would be quite hard to debug. I have always seen trackers like queues, that are by design totally unaware of any other entry. That makes them atomic and failsafe. Repeating an Solr document indexing less process intensive that querying the DB for its existence. Again, my 25cents and I came late to the discussion.
Comment #6
drunken monkeyFirst off, indexing doesn’t use our tasks system, so that’s completely independent from that.
The task types actually used by this module can be found in the three
*TaskManagerclasses, and of all of those only the server task manager’sdeleteItemstype has adatacolumn (in this case used for the deleted items’ IDs) that could conceivably (in my opinion) exceed 255 characters in length. And then for that to be falsely flagged as a duplicate there would need to be a second task with an identical length of the array of deleted items, identical item IDs for the first 255 characters but differences later on. (And do note that deleting all items for an index or datasource has its own task type. And that the task system will only be used if the initial attempt to just delete those items from the server right away failed for some reason.) I’d say that’s extremely far-fetched – and even if it should happen, the actual consequences would be minimal in most cases, as any undeleted items will just get deleted the next time any part of the Search API tries to load them.So, I think we should just go ahead with this. You’re right, though, a problem here would be extremely difficult to debug. I’ll remind myself to put something about this in the release notes to ask for other opinions. If the majority ends up thinking that this is a bad move, we can also easily just drop the whole
UNIQUEconstraint again.Unfortunately we now have a second test failure, this time against Drupal 8. Let’s fix both of them at once.
I’ll commit this once the test bot is happy.
Comment #7
drunken monkeyUrgh.
Comment #8
drunken monkeyFinally. Committed.
Thanks again for your input, DiegoPino!
Comment #10
diegopino commentedSorry for my late response. Too many moving targets lately. Clever fix by identifying the driver! Thanks so much
Comment #12
grimreaperHello,
If this may help anyone. I encountered an issue when deploying on prod for my personal website. And this is what I had done.
Add in the my.cnf:
Restart MariaDB and then convert existing tables with:
Then the hook_update_N was ok.
Comment #13
wellsFYI I have opened #3248297: Default tests failing over at the Facets module which seems to be (somehow) related to this change. Tests that were passing as of 6 Oct. 2021 now appear to be failing with a similar key length error despite (because of?) this patch. I'm confused here about how this patch fixed an issue here but maybe causes the same issue in Facets. You can see in the failed test example on that issue the new
UNIQUE KEY `task__unique` (`type`, `server_id`, `index_id`, `data`(255))part of the query.Anyone have thoughts on how these are related? I'm stumped at the moment.
Comment #14
mrshowermanI seem to have the same issue as Grimreaper.
The update hook 8107 fails with this message:
General error: 1709 Index column size too large. The maximum column size is 767 bytes: ALTER TABLE "search_api_task" ADD UNIQUE KEY `task__unique` (`type`, `server_id`, `index_id`, `data`(255));I assume it is due to the collation set to
utf8_mb4, which leads to a necessary column size of 4 * 255 = 1020 bytes, exceeding the 767 bytes in our MariaDB 10.5.12 environment.Comment #15
wellsIt seems that the Facets tests failures and the issues noted by @Grimreaper and @mrshowerman are all the same thing — this commit made the module essentially incompatible with MySQL 5.6 in certain (default? and testbot) configurations.
With the deprecation of D8 maybe this is acceptable? Drupal 9+ requires MySQL 5.7+ where this doesn’t seem to be an issue.
Comment #16
mrshowermanThanks @wells for pointing me in the right direction. Turns out that back when I installed Search API, I was still using MariaDB 10.1.48, so the search_api_task table got created with the COMPACT row format. Even after upgrading to 10.5.12 later on, the row format didn't change, leading to the above error.
I fixed it by running the following:
Now the update script was able to create the index.