Problem/Motivation

Despite the fact that we ask users to search for similar, previously filed issues before filing a new, we still get a lot of duplicate issues filed every day. This wastes precious time & resources because people are then lead to "hunt" duplicates and in general spent time cleaning up the issue queues.

Proposed resolution

Since most people don't seem to actually bother searching for duplicates before filing new issues, we should figure a way to "force" this search. This could be done by *automatically* doing the search in the background and then presenting the results to them à la AJAX.

Some words need to be excluded from the search (like "a", "the", "to" and "of", for starters) to reduce the number of false-positives.

chx mentioned here that we could use something like Sphinx for indexing things and perhaps a stemmer too for better results.

moshe weitzman later noted that apachesolr actually has a built in 'more like this' feature.

Here are a few screenshots of how Ubuntu Launchpad does it:

enter issue titlenope, just reporting newreport new bug

The actual bug submit form is not shown until only after the user clicks a "No, I need to report a new bug" button.

Here's another that shows how they have also added a "Recently reported" & a "Recently fixed" section, right bellow the main search section:

recently filed/fixed issues

Here's how bugzilla does it with AJAX + issues' statuses + quick-links to subscribe to them:

bugzilla duplicate bugs search

There already exist contrib projects that we can use that do part of the job for us (so we don't re-invent the wheel):

Uniqueness
Similar entries

Remaining tasks

???

User interface changes

The plain issue title field would become a special autocomplete field that does this search for duplicate/similar issues.

API changes

???

Exploring solutions: Managing Duplication

Original report by javanaut

I see a lot of the same issues coming up in the forums over and over again. I also see a lot of sigs requesting that users search issues before requesting help. Perhaps there should be a "preview" option for forum/project support submissions that searches the database using the content that they provided. Matching search results would display in between the preview and the submit button, so at least users have to see the first page of search results before submitting. This may be a heavy search request, especially if the user is verbose in their description, but perhaps this one step would stave off a bunch of the redundant support requests.

Files: 
CommentFileSizeAuthor
#14 recently_reported_and_fixed_issues.png68.08 KBklonos

Comments

pfaocle’s picture

Version:» x.y.z

+1

I like this idea: I've seen it implemented in various search engines and Support Ticket type systems. It should help users submitting support requests similar to existing ones, and hopefully eliminate duplicate issues.

aclight’s picture

Title:Automatically search issues for support requests» Automatically search for duplicate issues before submitting new issue
Project:Project» Project issue tracking
Version:x.y.z» 5.x-2.x-dev

This belongs in the project issue tracking queue now. It's still relevant, though might be tricky to implement and would be a pain for people already pretty familiar with the issue queues of certain projects.

Also, #102523: Suggest duplicates upon preview was marked a duplicate of this issue.

catch’s picture

Project:Project issue tracking» Drupal.org webmasters
Version:5.x-2.x-dev»
Component:Issues» Redesign
Category:feature» task
Priority:Minor» Normal

I think I might have a duplicate of this somewhere too. I reckon this'd be an ideal thing to try to address as part of the redesign efforts. I'm also not sure it'd have to be project issue specific - we could have a 'similar topics' block on preview for forum topics too maybe - so a generic module which does this, which we could then install on the project* related subdomain, would make sense.

oadaeh’s picture

@catch: maybe this is what you were thinking of: http://drupal.org/node/192714

catch’s picture

Category:task» feature
Priority:Normal» Critical

Marked #192714: Add a "Report a problem" link in d.o navigation block (in order to help avoid duplicate issues) as duplicate - if you've not read it yet, there's some good proposals in there.

Considering this issue itself has two duplicates, let's bump this to critical.

waddles’s picture

I would like to raise this again with another feature - automatically search for similar issues when marking an issue 'reviewed & tested' or 'fixed'

I found an issue a few days ago after enabling a module. I searched and raised a new issue but the root cause turned out to be in a totally different module. After fixing my own issue, I found numerous other issues reported, all for different modules but all due to the same root cause - they didn't change their D5 code to match the D6 API.

Perhaps this "find similar issues on closure" feature could provide a list of Original Posts with checkboxes to automatically add a comment saying "A similar issue was resolved in [#issuenum] - please check there to see if it applies to your case too."

Off topic, I also think that comments should be typed. I should be able to quickly subscribe, say thanks and "me too/me too fixed" an issue without having to create a full comment. The advice comment would be another type.

Issue submissions should be "guided" better so we don't end up with issues like #483914: Philbar: What is your email address? :D Plus, here are some issues you could maybe have a look at if you have time... and #493994: Any chance for Drupal 5 - LOL

FWIW, Ubuntu's Launchpad does exactly what the OP suggested. *sigh* if only I could make the Priority "critical and serious" :)

moshe weitzman’s picture

For inspiration, check out these screenshots of the Launchpad bug submission process.

Some notes:

All you are asked to provide at first is a title. Thats good - make it low effort for those are going to find a duplicate anyway

Review a bunch of possible dupes in the same project. Dupes can be suggested by apachesolr More Like This or some other algorithm

We just clicked the button, 'No, I just need to report a bug', and the current page instantly shows the bug form. No page reload needed.

smzur22’s picture

+1

greggles’s picture

The uniqueness module was built to do exactly this http://drupal.org/project/uniqueness

I'll ping Ben to see if he can provide advice on how we could potentially implement it here, but it was built in part with this use in mind.

smzur22’s picture

wow! looks great =)

benone’s picture

+1

benone’s picture

http://drupal.org/node/1030148 - this is my type.

Any other modules which could be used to create the new module discussed in this thread ?

klonos’s picture

...coming from #1213868: Return possible duplicate issues as the user types (AJAX?) a title for a new issue.

Thanx for the link to the Similar entries project Ben. I wasn't aware of it.

PS: Check out how bugzilla does it.

klonos’s picture

StatusFileSize
new68.08 KB

...a "Recently reported" & a "Recently fixed" section could be a very useful addition too:

klonos’s picture

...hope you all like the new summary ;)

klonos’s picture

Title:Automatically search for duplicate issues before submitting new issue» Automatically search for duplicate issues/questions before submitting new issue/question
Component:Redesign» Site organization

...better suited component and title adjusted because I think that we should provide a solution that could scale to other parts of d.o.

#1460738: [meta] Issues for improving support options on d.o. could use this feature request here -though it should not be blocked by it-.

dww’s picture

Issue summary:View changes

...new summary. Complying to the Issue summary standards template too ;)

klonos’s picture

Thanx Derek. I just added it to the list of related articles in the issue summary ;)

klonos’s picture

Issue summary:View changes

...added #1128044 to the list of related issues.

mgifford’s picture

This would be particularly important for high traffic projects like Core, Views & Dates. See the discussion #2183117: Dates are Important - So Why Are there over 1000 open issues?

mgifford’s picture

Project:Drupal.org webmasters» Drupal.org customizations
Version:» 7.x-3.x-dev
Component:Site organization» Code
Issue tags:+issue queue

Moving projects. Not sure if this is better in customizations or infrastructures, but think it's more likely to get addressed there.

Meta issue here - https://drupal.org/node/1080550

EDIT - Earlier discussion on GDO https://groups.drupal.org/node/136179