In D6, we have some custom logic for giving comments their nice monotonically increasing comment numbers on a given issue. E.g. instead of:

https://drupal.org/node/1624874#comment-6105090

We can just refer to "#1" or #1624874-1: Port project issue text filter. So, we call this comment #1 not #6105090.

In D6, we've got {project_issue_comments}.comment_number to hold this value for each issue comment, and {project_issue_nodes}.last_comment_id to store the latest value in an issue so that we can come up with the right next number when inserting a new comment on an issue.

This is sort of a PITA from a porting/code perspective, since I'm not immediately sure the cleanest way to replicate this functionality via D7 tools. However, we definitely need this from a UI/UX perspective, since these sequential comment numbers are really critical to how people use the issue queues.

This issue is about figuring out how this should work and actually implementing it in D7.

Proposed solutions

A: straight port -- custom storage and logic

We could just do a straight port of the existing code, both storage (custom DB tables) and logic.
Pros:

  1. it all works now and is heavily tested in practice
  2. less of a problem with data migration

Cons:

  1. it's a continuation of custom-magic in project_issue instead of using flexible tools
  2. we were hoping to entirely kill the tables that this stuff would live in, and maybe it's silly to have these tables just for this one feature

B: core fields for storage, custom code for logic

We could use core fields for storing this info (a hidden integer field on issue nodes to hold the last comment number, and a read-only integer field on comment entities to hold each comment's number), and then have some custom code (similar to what we have now) inside hook_comment_insert() (or whatever it's called) to grab the right value of the comment number while inserting a comment and then increment the issue-wide max. We still need to be careful about locking to avoid race conditions.

Pros:

  1. it'd be more consistent and flexible for displaying this number when rendering comments if it was using a core field
  2. could hopefully get rid of the custom DB table for storage -- although it's less obvious that using a core field to store the issue-wide max is worth doing, especially if we might have to use a custom table for locking, anyway...

Cons:

  1. would need a data migration path from the old tables into the new core fields
  2. would still need custom logic, but then would have to conditionally check to ensure that the fields are defined on the right entities to store the results

C: evaluate Comment Easy Reply module

Yay, the Comment Easy Reply module already exists for exactly this, with both D6 and D7 versions! Who knew?!? We should check it out and see if it's up to snuff for Drupal.org deployment, but this seems extremely promising.
Pros:
- Already exists and claims to solve exactly this problem
- Proudly found elsewhere!
- Adds some other potentially useful functionality
Cons:
- none?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dww’s picture

Note: I just pushed a fix for #1624874: Port project issue text filter and marked that fixed, even though there's a little code block in there that's commented out with a @todo pointing here. So, whenever we resolve WTF we're going to do here, we should be sure to also fix the [#nid-comment] filter thing, too.

dww’s picture

Added some proposed solutions with pros/cons to the summary. As always, folks are welcome to add other proposals and/or flesh out pros + cons and/or other details of the two proposals I already listed.

Thanks,
-Derek

Drave Robber’s picture

Wait. At the moment, is this functionality NOT provided by Comment Easy Reply module?

If it's not, maybe it should.

Senpai’s picture

Agreed. Comment_easy_reply is almost EXACTLY what I was going to propose for "linkable" comment numbers. I love it.

Senpai’s picture

Issue summary: View changes

added some proposed solutions with pros/cons

dww’s picture

comment_easy_reply says: "Posted by kongoji on April 8, 2012 at 2:58pm" -- Project* has been around for about 10 years. ;) So, uhh, yeah, project_issue has been doing this itself for a while. ;) But I'd be thrilled to outsource this! So I just added option C to the summary to eval that module for possible d.o deployment. We should probably just fire off one of those "can we deploy this?" issues in the infra queue, but it seems extremely promising. Anyone want to do an initial evaluation? Code review/security audit?

Yay,
-Derek

Senpai’s picture

Easy there, smarty pants. Project_issue is probably still going to need to do a touch of custom work via fields just for data migration purposes, but outsourcing to this new method adds the relationability [yes, new word] of linking a comment about a comment to that original comment. It's something I've always though was lacking in an issue comment, namely that when I say "Up in #32, you said..." but the "#32" part *doesn't link to anything*. This way, it would.

I'll get someone to do a security review first. Then, it's a deep dive to see exactly how this module handles it's data architecture, storage, and retrieval. No sense replacing the innards of project_issue's numbering system with something that doesn't improve the odds of adhering to TNG standards. Then, after that, we can open a 'pretty puhweeze' Infrastructure issue.

greggles’s picture

I'll get someone to do a security review first. Then, it's a deep dive to see exactly how this module handles it's data architecture, storage, and retrieval.

Please, no. Let's reverse those. If it doesn't match the needs well then let's not waste the security team's time.

dww’s picture

@greggles: agreed. That's why I said: "Anyone want to do an initial evaluation? Code review/security audit?" ... Initial evaluation comes first, then code review/security audit. I was just looking for volunteers, since I'm trying not be a SPoF and just do everything myself.

bdragon’s picture

Assigned: Unassigned » bdragon

Reviewing.

bdragon’s picture

Assigned: bdragon » Unassigned

Comment easy reply does not have persistent numbering, so it would not work. The #s next to the comments depend on whether or not you can see unpublished comments, comments have been deleted, etc.

It's not usable in this state for our purposes.

Something sorta like https://drupal.org/project/serial but with the ability to put it on any entity (comment) and partition it by an entity property (comment nid) would work...

Senpai’s picture

Status: Active » Needs work
Issue tags: +sprint 6

I've emailed the maintainer and asked to begin a discussion about modifying this module to incorporate a 1-to-1 comment ID to realnumber scheme for drupal.org. We shall see if they reply soon, and if not, we'll just make something happen during the upcoming sprint.

kongoji’s picture

Hi everyone, I'm the maintainer of Comment Easy Reply module.
I'm really surprised (and honoured) to see you evaluating my module and I would really happy to talk with you about any possible improvement on the module.

1-to-1 persistent comment ID is a good idea, in my opinion it could be done in two ways:

  1. Adding a new table with 3 fields: cid, nid and comment_number
  2. Adding a new field comment_number to comment table

The second option sounds better to me because avoids creating a new table with as many rows as comment table has. Moreover, with a hook_shema_alter() on comment table schema, it should be possible to automatically load the comment number during comment entity load.

I'm really interested to hear your opinions and thoughts about this and all other new implementations/ideas you have on the module.

dww’s picture

Thanks for the reply, kongoji!

I've *never* understood why core makes it easy to alter other module's tables via hook_schema_alter(). ;) This just seems like a recipe for disaster. If that module then changes their schema in the future and has to migrate its own data, it's not going to know about your data and it might be lost, etc. I understand the concerns about performance, however in this case, I don't think the extra JOIN is a problem since we're just rendering all the comments for a single entity and there's not going to be a WHERE clause that spans tables. It's not like we display these comment numbers in global views of issues across the site or something (where the performance would really suck). So, I think it's cleaner and safer to just go with a separate table. Plus, it might be the case that not every comment bundle (e.g. set of comments for a given node type) will want to use this functionality, so it'd probably be more space efficient to only store these extra numbers for the comments that need them.

Moreover, if you look in the 6.x-1.x branch of project_issue there's a {project_issue_comments} table that includes all the additional fields the project_issue module injects into comments. However, the persistent comment numbering functionality also depends on the {project_issues} table, which stores the last comment number for a given issue node. We had to do some pretty tricky magic to ensure that there were no race conditions when two users post comments simultaneously and the ordering gets screwed up. Feel free to study that code and copy it for your module. Point being, I don't think there's a safe way to accomplish this feature with a single column in a {comments} table, even if altering the core schema was actually a wise thing to do. ;) Therefore, I don't think a single {comment_easy_reply_comment} table with cid, nid and comment_number will be sufficient. I think you'll need table like that, plus something like {comment_easy_reply_node} with nid, last_comment_number, and db_lock. Again, see the 6.x-1.x branch of project_issue for more.

Should we just open up a new issue in your issue queue about this persistent numbering stuff, instead of continuing to discuss that here?

Thanks again!
-Derek

kongoji’s picture

Hi Derek,
Thank to you for such interesting argumentations, there is always something new to learn :-)
I watched the problem mainly from a performance side, but now I see your concerns about schema changes by core modules.
I'm looking at project_issue_comments and its way of locking the table, it's very interesting and it works really well, but do you think it's a bad idea using the drupal standard locking mechanism (lock_acquire, lock_release, ...)?
What I'm thinking about is something like (I write meta-code to make it short)

// Comment has just been added.
if (lock_acquire('comment_easy_reply' . $comment->nid)) {
    // $next_id = SELECT FROM {comment_easy_reply_serial} WHERE nid=$comment->nid
    // INSERT {comment_easy_reply_number} VALUES($comment->nid, $comment->cid, $next_id)
    // $next_id++
    // UPDATE {comment_easy_reply_serial} set number=$next_id 
    // ...
    lock_release('comment_easy_reply' . $comment->nid);
}

I don't want to force anyone's decisions in any way, I'm just curious by nature and I love to hear from people who knows drupal a lot better than me.

It's ok for me to move the issue on Comment Easy Reply issue queue

greggles’s picture

Project_issue's locking code was added in 2009 while the 6.x lock.inc was added in 2010.

I think it makes sense to move to the core locking functions, but I'll leave it to dww/hunmonk to say if they feel #14 makes sense or something more like what project_issue has now makes sense. I did just check and there are no messages in drupal.org's dblog currently for project_issue meaning that the locking code has not failed to grab a lock, though our site only holds less than 24 hours of data so that's hardly a conclusive test.

Senpai’s picture

Title: Figure out and port comment numbering functionality to D7 » Figure out and port project_issue comment numbering functionality to D7
Status: Needs work » Postponed

The discussion and implementation details have moved to #1678398: Persistent (atomic) numbering of comments needed

bdragon’s picture

Status: Postponed » Needs review
FileSize
3.22 KB

I'm not convinced. We have special link handling already for comments, that work the same way as project links.

Here is an implementation of using {comment}.thread as the basis for numbering.

I have already fixed the data glitches on drupal.org that would prevent rethreading issues to match the comment_number. (assuming more don't crop up.)

bdragon’s picture

Furthermore, supporting threading is a matter of adjusting the regexp slightly.

kongoji’s picture

Hi bdragon,
the idea of using comment thread to get comment number is brilliant!
I just finished to implement {comment}.thread numbering in Comment Easy Reply.
At this stage I don't know if it could still be interesting for you, anyways it seems to work pretty well. With the new comment numbering system I've been also able to give a proprer link fragment in comment's permalink.

bdragon’s picture

Assigned: Unassigned » bdragon
FileSize
4.3 KB

Here's an updated patch that should play nice with comment_easy_reply.

Also removed a todo that is todone.

Senpai’s picture

Assigned: bdragon » chx
Issue tags: +sprint 7

@chx, could you please review #20 before we commit it?

dww’s picture

Assigned: chx » Unassigned
Status: Needs review » Active

A very informal and quick skim of the d.o database reveals all sorts of cases where the {comment}.thread value once piped through vancode2int() doesn't match {comment}.subject.

For example:

#1699272-59: cod_bof view should work with single column sessions schedule and tabs
nid: 1699272
cid: 6354178
subject: #59
thread: '118/'
vancode2int('118'): 44

comment_save() is computing the thread value by doing some funky MAX() logic. So, if you delete comments, comment_save() will re-use the same thread value. All future comments in the issue are no longer going to have a thread number that matches the (desired) comment number.

So, I don't think we can use this approach at all, and need to return to the drawing board for either comment_easy_reply or our own code.

Sorry,
-Derek

chx’s picture

Derek, I am not sure whether it is such a problem that we reuse the comment numbers after deletion. I have found a bit weird we ddn't, as the issue you linked has a huge gap in the numbering because of spammer cleanup.

bdragon’s picture

dww: Don't worry about that, part of the migration will be to re-thread the issues to match comment_number. I already have it under consideration. I spent several hours a while back combing through the database and cleaning up the glitches that would prevent rethreading like this.

bdragon’s picture

dww: And for reassigning a comment number, who cares? The only possible place this could matter is a spam delete and a followup referencing the spam comment happening simultaneously causing the followup to reference itself... Big deal...

bdragon’s picture

Assigned: Unassigned » bdragon

Back to me.

dww’s picture

Priority: Normal » Critical

Bump. This seems like a launch blocker to me.

bdragon’s picture

I tweaked the migration job that strips the extra comment titles, hopefully it won't bail now.

bdragon’s picture

I committed a modified version of #20 which is closer to how core does the permalink, because I can't remember why I was attempting to preserve the query parameters. If some reason does come up it can always be changed again.

http://drupalcode.org/project/project_issue.git/commit/f3665dc65b6f55f1a...

Bluecheese will need to be updated...

Senpai’s picture

Issue tags: +bluecheese

@bdragon, could you please elaborate on how the Bluecheese Team needs to update their stuff based upon what you've done in commit #31?

bdragon’s picture

@Senpai: Here is one possible implementation.

bdragon’s picture

To elaborate on that patch:

If we want to avoid having a separate tpl for issue comments, we can alter the link into the comment title, to make up for the fact that bluecheese discards the permalink.

Senpai’s picture

Status: Active » Fixed

I've cross-linked the Bluecheese Team (posted in #1796078: Re-theme project and issue pages to the patch in #33 above, so I'm marking this 'fixed' because there seems to be no further development needed.

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

added section about comment_easy_reply