Continuing from #2453587: [policy, no patch] Changes to the project application review process and #2532162: [policy, no patch] Mandate all projects have Automated Testing enabled, it would be useful if all project releases were required to pass all tests provided by the module and the testbots. By doing so the complaint that old projects don't have to follow the same standards as new projects under the project application review process would be, in part, made moot.

Two aspects of this in particular should be considered:

  • Part of the implementation of this should automatically create an issue titled "fix tests to allow further releases" should the tests fail.
  • There would need to be a way of defining files that should not be tested via the project's info file. The rationale for this would be thus: once we get to the point of having automated code reviews we'll need to be able to indicate that certain files are from a 3rd party source and should not be modified, e.g. GPL's libraries, etc.

Comments

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented

By doing so the complaint that old projects don't have to follow the same standards as new projects under the project application review process would be, in part, made moot.

As pointed out in #2532162: [policy, no patch] Mandate all projects have Automated Testing enabled, passing tests (presumably based upon the Drupal Testing framework (in core for D7)) is currently not a requirement for new projects, so the "same standards" part of the rationale is not correct.

As for introducing this as a requirement for all projects, I do not think it will improve anything. For arguments against making this a requirement, please see #2532162: [policy, no patch] Mandate all projects have Automated Testing enabled.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented

This is just a groundwork issue, it'd be entirely worthwhile if #2532246: [policy, no patch] Add code reviews as part of Automated Testing were to be approved.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Parent issue: » #2532352: [policy, no patch] Simplify, automate parts of the project application & release processes

I've added the parent issue for this: #2532352: [policy, no patch] Simplify, automate parts of the project application & release processes

Apologies for working backwards.

jthorson’s picture

jthorson commented
Status: Active » Needs review

As pointed out in #2532162: [policy, no patch] Mandate all projects have Automated Testing enabled, we don't mandate that contributors use one particular development methodology over another ... while we may recommend that contributors adopt a test-driven development methodology in accordance with current best practices, I would argue that forcing them to write tests would be overly prescriptive.

As such, I would suggest that this proposal becomes a non-starter as currently written.

Flagging for discussion in the next TWG meeting, where my recommendation would be to close this issue.

jthorson’s picture

jthorson commented
Status: Needs review » Closed (won't fix)

As per the comments provided in the issue referenced in comment #4, the TWG has agreed that policy could be perceived as being overly prescriptive and increasing the barrier to contribution; and as such, would recommend against such a policy at this time.