As part of the Bug Smash Initiative, we are triaging issues that are marked "Postponed (maintainer needs more info)", or tagged with "Needs steps to reproduce" and "Needs screenshots".

We had a discussion in Slack #bug-smash about guidelines for when it is OK to close an old issue and wanted to record that as well as opening it up for wider feedback.

Consensus on Slack was that if an issue meets these three criteria, it should be closed:

  1. Report consists only of an error or description of an error with no steps to reproduce
  2. Someone replied asking for steps to reproduce >6 months ago
  3. No steps were provided and/or it is only reproducible in the reporter's site

This proposal only applies to a relatively narrow set of issues, and many of these are already closed as part of the initiative triage. So I think there is a more broad discussion to be had about stale issues and not just ones that meet these somewhat obvious criteria.

The idea of automating this came up as well, which is something that has been discussed before in #156704: Dealing with "postponed (maintainer needs more info)" issues and #1230642: Automatically mark "postponed (maintainer needs more info)" issues to "closed (cannot reproduce)".

Comments

pameeela created an issue. See original summary.

pameeela’s picture

Issue summary: View changes
pameeela’s picture

I think I am -1 on automating this, because:

  1. It's really easy to spot the ones that are stale and can be closed without further ado, and doing this manually does not take that much time
  2. In doing triage I have found plenty that were postponed and had no recent activity but were perfectly valid

But it is certainly possible that (as @catch said in #1230642: Automatically mark "postponed (maintainer needs more info)" issues to "closed (cannot reproduce)":

closing the issue automatically will bump it in people's trackers/subscriptions - potentially reminding them to actually post back the more information (then they can mark it back to some kind of active state after that).

I guess it is a question of what is worse:

  1. Having a queue full of stale issues distracting from real ones
  2. In cleaning up the queue, potentially closing an issue that shouldn't be closed
cilefen’s picture

+1 to this. Also, I suggest going beyond bugs and also considering the issue type. For example (I just made these up):

  • Support requests: Close any with no comments in n months. Suggest the usual support channels when closing. I suggest automating this one.
  • Feature requests: Close those with fewer than n comments and those without comments in n months. If there is community interest in a feature, there ought to be activity.
catch’s picture

I think the suggestion in the issue summary is good, although I do think 6 months is more than enough time for someone to get back with steps to reproduce.

With support requests we run the risk of closing them before anyone replied, but this is already the case. Eventually with much less in the queues, there's more chance of any one active issue being replied to and resolved, or people finding what they're looking for instead of creating duplicates.

IMO both of these could be automated, a big question is whether that could be enabled for contrib modules too (maybe opt-in?).

Feature requests are a lot trickier because there is a lot of overlap between tasks and feature requests, and some things are valid but get forgotten. For example I just opened #3156396: Use assertSameSize() to check same size of two countable variables, it is arguably a task, someone might work on it next week or in five years.

larowlan’s picture

+1 from me too, and I think if we made it automated you would need to be able to configure it per project, so that's out of scope here.

I think here we just need to agree on the idea and the number of months and move it to Rtbc then fixed

pameeela’s picture

Issue summary: View changes
cilefen’s picture

@catch I would argue that's a task. In my day job, I consider a feature to be something an end-user may notice. But on the other hand, if a testing framework adds API, that's a feature.

quietone’s picture

+1.

@larowlan, is that the number of months since the last comment?

johnwebdev’s picture

+1 on automation.

The number of issues looks scary at first sight, but then you realize many of them are either outdated, duplicates etc. For instance there are > 1000 feature/tasks for 7.x that can be closed.

Support requests could be automatically closed after n time, if the last reply is not from the one who created it.

I also think the flow described in issue looks great 👍

andrewmacpherson’s picture

Please DON'T close accessibility issues on any of the grounds mentioned in the comments so far, because...

  • We don't get much feedback from users with disabilities, and I don't want to bury it deeper by closing these issues. People with disabilities already face an uphill struggle to have their voices heard; a ton of procedural hurdles and micro-aggressions. I think the sort of policy described here just adds to that.
  • It won't be any easier to keep track of accessibility if someone takes them off the accessibility maintainers' list. I'd rather have lots of "needs more info" in my issue list, than have those issues move even further away from my crappy attention span. The way my brain works, the "back burner" is a very useful place.

A problem with closing issues by policy (especially automatically) is that they may have been wrongly classified in the first place. It's fairly common to find issues with the wrong category, component, or project. These issues likely won't have been seen by others who could provide more information. Closing them makes it even less likely they will ever be seen.

I'm against closing issues merely because they are old...

If issues were marked "postponed", that explicitly means that it's worth returning to later. These shouldn't be closed just because time has passed. Rather, they need to be re-assessed for relevance, preferably by someone who understands the topic/subsystem/module. It's great when a reason has been given for postponement, but this isn't always the case.

If they were marked "needs steps to reproduce" and no steps have been provided yet, that doesn't mean the issue is no longer relevant, or that the bug has gone away. It just means that it still needs more information to describe the problem. There are a lots of reasons why the original reporter may not have been able to provide steps to reproduce:

  • Users cannot remember the sequence of steps which they took.
  • Users don't want to make the bug happen again! Maybe they can't risk trying to reproduce it, or shouldn't set up specimen content on a production web site.
  • Not all CMS users reporting bugs are experienced testers. Describing steps-to-reproduce is a tester's skill, not a content editor's skill.
  • The user literally does not know what control they activated. Keyboard users and speech control users can inadvertently activate an invisible control (we've had these before). Perhaps focus was moved programmatically, meaning they activated a different control to the one they thought they would activate (our ajax behaviours sometimes lead to this). If the control was visible, but focus wasn't visible, the user doesn't know what control they activated. Failures of WCAG "label in name" or "character key shortcuts" success criteria are other possible causes of accidentally activating a control without knowing which control it was.
  • Users may have trouble articulating the problem. Perhaps they are unclear what terms to use, or aren't using their preferred language or communication style.
  • Our issue queue has a complex UI, which itself may be a source of difficulty for someone trying to report a problem.

#3:

closing the issue automatically will bump it in people's trackers/subscriptions

The "your issues" link on d.o user profiles leads to a list of open issues only. Closing an issue won't attract attention to it. On the contrary, it will just disappear from the default "your issues" list.

Re. #4:

Feature requests: Close those with fewer than n comments and those without comments in n months. If there is community interest in a feature, there ought to be activity.

There are features which we must introduce because we can't achieve WCAG or ATAG conformance without them. It's often a struggle to get community interest in these, let alone find contributors to help make them happen (and we certainly can't rely on companies for that). Please don't make me have to add a certain number of comments to get it over the N threshold, or hustle for relevancy.

pameeela’s picture

Issue summary: View changes
Status: Active » Needs review

Updated the IS to reduce >1 year to >6 months.

I think that we have reached a consensus that automating this is problematic as we are not going to be able to find a set of rules that avoids closing valid issues.

Marking this needs review, if people are happy with this policy then the issue can go to RTBC then fixed.

I think that we should open separate policy issues for support and feature requests that were mentioned here, as this issue is pretty specific to a common type of bug report.

pameeela’s picture

mmjvb’s picture

Suggest to make it clear in the title that this issue is about bug reports.

pameeela’s picture

Title: [policy, no patch] Adopt a set of guidelines for closing old issues » [policy, no patch] Adopt a set of guidelines for closing stale bug reports

Good point @mmjvb. Updated.

sime’s picture

I support this policy. I have been looking through the "Needs more info" there is a lot of noise from old "Postponed maintainer needs more info" issues.

Can the point in #11 be handled? Perhaps via a list of tags that should not be closed?

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

quietone’s picture

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

+1 for this.

pameeela’s picture

I think that this issue should be marked RTBC and then fixed. I totally understand the concerns raised in #11 but I think several years of this initiative have shown that we are taking a cautious approach to this and treating every issue with care.

For my part, having triaged probably >1000 issues, I have never come across an accessibility bug that matched the description here. There are a large number of bugs against accessibility, but they tend to be well documented, and even if they are old or stale, we would never close one that was established to be valid.

On a broader level, I think the approach taken by the initiative has been exemplary. Of course, mistakes are made -- I reopened an issue a few weeks ago that was closed in error because it wasn't understood. But even that shows the system working, as I wouldn't have ever seen the issue except that I saw it was updated!

Of course, I created this issue so I won't mark it RTBC but curious whether anyone else has input now that we have a few years of progress to point to.

quietone’s picture

Reading #11 and #16 I decided to look at the evidence. There are 25 issues tagged bugsmash and accessibility that have been closed for reasons other than fixed.

won't fix: 2
works as designed: 7
cannot reproduce: 6 (includes several IE 11 issues)
outdated: 10 (includes several IE 11 issues)

I opened all the issues and read how they were closed. I didn't find that any of them were closed improperly. I do agree with @pameeela in #14 that the Bugsmash triagers are "taking a cautious approach to this and treating every issue with care.".

I have added the proposed text to the initiative docs, https://www.drupal.org/community-initiatives/bug-smash-initiative/issue-...

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

The concerns in #11 are very valid, but as remarked in #24 and #25 we've been very careful about this in the past and will continue to do so.

I think writing down, and consistently following this policy, while still being careful is a good improvement in the maintenance of the issue queues.

lauriii’s picture

+1 for the proposed policy, as a more manageable issue queue benefits everyone by increasing our efficiency. I believe that it's essential for us to emphasize that closing stale issues doesn't mean disregarding bugs or problems. Issues are being closed because they are lacking next steps due to insufficient information.

We should always respect reported problems and strive to understand them. However, if we cannot proceed on an issue due to inadequate information, it holds little value in the queue. While #11 brings up valid concerns, I don't believe that maintaining a queue filled with unactionable issues addresses those issues.

g-brodiei’s picture

Status: Reviewed & tested by the community » Needs work

Saw this issue from bug smash initiative channel from @quietone to check wordings on policy page.

+1 on the policy change. I’ve read through the threads and notice a slight wording diff on the policy page.

On policy page:
If the stale issue meets these criteria, it should be closed:

On the issue summary:
if an issue meets these three criteria, it should be closed:

I believe the word `three` would make it clearer for future readers (not native English speaker) to understand the criteria, thanks!

quietone’s picture

Status: Needs work » Needs review

Thanks!

I have updated the page.

g-brodiei’s picture

Status: Needs review » Reviewed & tested by the community

Checked the page, wording changed as suggestion, thanks!

Set to RTBC

g-brodiei’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.