System messages removed all issue tags during D7 upgrade

Confirmed. This looks like it's happening whenever an issue is closed after two weeks without activity.

This looks like it's happening whenever an issue is closed after two weeks without activity.

Also seems to happen on "failed test" reports on issues being actively worked on - which is really unfortunate :-)

I'd even bump this to critical...

Likely related to a critical D7 core bug: #1126000: [meta] hook_field_*() called on fields not set in the $entity object (partial updates) Edit: I stand corrected. :)

I didn't lose tags on a test failure just now.

Could this be a caching problem? Or maybe avoided by a warm cache that contains the tags?

This is an itermittent issue ... looks like the drupal.org tag monster may have survived the port.

I think this was the d.o tag monster *during* the port. It only seems to have happened on System Message updates that came during the migration. I haven't seen it in any System Message updates that were triggered since migration.

Do we have any idea how many issues were affected?

According to

mysql> SELECT COUNT(*) FROM field_data_field_issue_changes fic INNER JOIN comment ON fic.revision_id = comment.cid WHERE field_
issue_changes_field_name="taxonomy_vocabulary_9" AND uid=180064;
+----------+
| COUNT(*) |
+----------+
|    24434 |
+----------+

There are 24434 comments posted by 'System Message' user where there were also changes to the issue tags. Sometimes those were added, sometimes removed. We could theoretically get old field data from those change records and update issues, however that might get messy.

I'm trying to clean out https://drupal.org/project/issues/search?status%5B%5D=8&issue_tags=Drupa... (needs review). This one never seemed to get the memo that we're no longer using 'needs review' to mean 'needs triage'.

Re: #9 : Thanks for the query. Good to know the scope of the problem. Honestly, I can't think of a single case where 'System message' should ever be changing issue tags. Seems like it shouldn't be *that* complicated to try to write something that can revert all of those. Thankfully, the same query on my d.o dev site still has 303 hits, so I can investigate a few of those and it should be possible to develop a DB update or drush command or something to repair them. I'll assign this issue to myself if I actually start working on this. Until then, anyone else is totally welcome to run with this.

Thanks!
-Derek

BTW, I've never seen a problem with this on auto-close messages that happened since D7 was live. This was only a problem during the migration itself, so I'm giving this a more accurate title.

...moved to its own issue: #2166385: Issue tags field does not autocomplete beyond a certain length.

klonos - #12 is a separate issue from this. I suspect it could be a core bug. Issue tags are the core taxonomy module. Does the same behavior happen on a term field with multiple autocomplete?

This affects 17,707 issues.

SELECT c.nid, max(c.cid), concat('https://drupal.org/node/', c.nid, '#comment-', max(c.cid)) FROM comment c INNER JOIN field_data_field_issue_changes fic ON fic.entity_id = c.cid AND fic.field_issue_changes_field_name = 'taxonomy_vocabulary_9' AND fic.field_issue_changes_new_value = 'N;' LEFT JOIN field_data_taxonomy_vocabulary_9 tags ON tags.entity_id = c.nid WHERE c.uid = 180064 AND tags.taxonomy_vocabulary_9_tid IS NULL GROUP BY c.nid ORDER BY NULL;

Attached is the script I am using to add a new system message restoring the tags.

It has run on
#15380: Allow to configure content of issue notification e-mails
#19646: Aggregator: Feed Suspend Option
#20046: User picture messes up poll display
#20446: Cursor focus on user login page

And is running on the first batch of 100.

This takes 16 minutes for a batch of 500, so will be done in 9ish hours.

I don't think this fix is doing the right thing.

I didn't know about this issue until today when I saw that (so far) more than 125 issues in the Ubercart queue had new system messages restoring tags. I expect to see more as the batch progresses.

The thing is, all these issues are OLD: 4.5 - 5.5 years old. All of them had the tags removed many years ago, NOT during the d.o D7 upgrade.

From what I can tell, removing tags is something that used to happen automatically, a long time ago, when issues were automatically closed? This fix is going back and trying to "correct" something that was intentional way back when it happened.

So while I think the fix isn't breaking anything per se by restoring tags, it's certainly not addressing just D7 upgrade problems. Now I have to visit page 3 of my issue queue to see my current issues ...

Sorry for all the noise. For me, I almost always have the '- Open issues -' filter selected, so I haven't seen any bubble up yet.

This is indeed a D7 migration bug. For example, https://web.archive.org/web/20120915093126/http://drupal.org/node/920110 doesn't shows the tags are preserved, but they were removed in #920110: google fonts and ssl.

The more-thorough fix would be updating the issues, without creating a new revision or System Message comment, and updating the nodechanges field. However, that is a lot more complex and I'm afraid of causing more unintended side-effects.

I had 135 issues flood the top of my queue.
If there was still a full pager, it would be mitigated.
But that's 3 pages.

Nothing that can be done now, except fix the pager issue I guess.

...moved to its own issue: #2166385: Issue tags field does not autocomplete beyond a certain length.

Edit: turns out it was #2171455: d.o does not autocomplete tags starting with # hash anymore

I'm seeing issues where removed tags were re-added. Seems to be related to tags that were removed automatically when an issue is closed by the system. Eg: #636966: mikeker [mikeker] @#13 "module review" was removed and @#14 it was added. Or #891432: Views3 compatibility screencast @#2 and #3 where "views" and "views3" tags (irrelevant tags, I'll grant you) were removed and readded.

EDIT: Sorry for the noise, I just realized that what I was seeing was exactly what this issue was designed to fix. That'll teach me to post before fully reading the thread...

I paused this script overnight, and realized that back-dating the comments shouldn't be a problem. I'll see about changing that for the existing comments, and update the script to handle it for the remaining 9,103 issues.

For the record, backdating comments was running the results of:

SELECT concat('UPDATE comment SET created = ', max(c2.created), ', changed = ', max(c2.changed), ' WHERE cid = ', c.cid, ';') FROM comment c INNER JOIN field_data_comment_body b ON b.entity_id = c.cid AND b.comment_body_value = 'Restoring issue tags, see #2125755: System messages removed all issue tags during D7 upgrade.' INNER JOIN comment c2 ON c2.nid = c.nid AND c2.uid = 180064 AND c2.created < c.created WHERE c.uid = 180064 AND c.created > unix_timestamp('2014-01-01') GROUP BY c.nid ORDER BY NULL;

SELECT concat('UPDATE node_comment_statistics SET last_comment_timestamp = ', max(c.changed), ' WHERE nid = ', ncs.nid, ';'), ncs.last_comment_timestamp FROM node_comment_statistics ncs INNER JOIN comment c ON c.nid = ncs.nid WHERE ncs.last_comment_uid = 180064 GROUP BY ncs.nid HAVING last_comment_timestamp > max(c.changed) ORDER BY NULL;

And then

UPDATE search_api_db_project_issues pi INNER JOIN node_comment_statistics ncs ON pi.item_id = ncs.nid AND pi.last_comment_timestamp <> ncs.last_comment_timestamp SET pi.last_comment_timestamp = ncs.last_comment_timestamp;

While looking for a good way to backdate the new comments, I stumbled on setting $node->nodechanges_cid. This will update the existing comment, like #1120160: File/Image Upload button breaks ajax upload.. I can run with this. Removing the empty "Issue tags" change is likely doable.

All done.

Superb! Thanx.

Nice catch drumm - all the noise is gone. I greatly appreciate that.

For the record, this is the script I used for the second round.