Problem/Motivation

We have currently an entry for simpletest as a component, but well, we are moving away from it.

Proposed resolution

Let's add a phpunit component and add a couple of people.

Remaining tasks

User interface changes

API changes

Data model changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner created an issue. See original summary.

dawehner’s picture

Let's ask @alexpott, @berdir, @klausi whether they are fine with that.

Drupal.org already has that specific component.

dawehner’s picture

Status: Active » Needs review
FileSize
641 bytes
alexpott’s picture

I'm fine with this.

klausi’s picture

Yes, I'm also fine with this.

Wim Leers’s picture

Nice, thanks guys! While there are some rough edges still for sure, using phpunit instead of run-tests.sh is so much better. Thanks for all your hard work on this!

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

Ok with me too.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 3: 2808123-3.patch, failed testing.

dawehner’s picture

Status: Needs work » Reviewed & tested by the community

Back to RTBC

xjm’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +Project governance, +Component maintainers
Related issues: +#2702321: Group core components into broad categories, +#2786145: Clarify how multiple maintainers share a role

Nice, +1 for these maintainers. Thank you all!

I'm wondering if we should instead update it to be an entry for "Testing framework", since there are multiple related APIs, including BTB, JSTB, etc. and the test runner itself? This is in line with our idea to build maintainer teams within core, and could also help with some ambiguity with the specific component and what it includes.

klausi’s picture

FileSize
776 bytes

Sure, a section for Testing Framework also works.

Note that we also had an abandoned "Testing" section in the file for topic maintainers, but I think it is a duplicate and we can consolidate that under the Testing Framework subsystem.

Note that I did not touch the entry for Simpletest module, I think we can leave that as is as long as we have the module. Just in case people search for simpletest in the file.

We don't mention the word "PHPUnit" anywhere in the file, but that should also be OK for now, we will have a special place for the PHPUnit initiative anyway #2808121: Add PHPUnit initiative to MAINTAINERS.txt.

xjm’s picture

Well, so the "Testing" section is supposed to be for the Testing topic maintainer, which is a different role than Subsystem maintainer that has to be approved by Dries. (The idea being that the topic maintainers are each responsible for a core gate.) So we might have different maintainers for APIs for testing, vs. the person responsible for maintaining testing best practices across core (in the same way that Bojhan and yoroy maintain our usability best practices).

It could also be worth removing the redundant-ish simpletest entry, depending on what alexpott and Berdir think. (Otherwise we can also do that once we actually deprecate it entirely; I can see either way.) :)

dawehner’s picture

Well, so the "Testing" section is supposed to be for the Testing topic maintainer, which is a different role than Subsystem maintainer that has to be approved by Dries. (The idea being that the topic maintainers are each responsible for a core gate.) So we might have different maintainers for APIs for testing, vs. the person responsible for maintaining testing best practices across core (in the same way that Bojhan and yoroy maintain our usability best practices).

It is a tricky question, because well, there might be also tests written in javascript at some point. Those should have its own subsystem I guess?

What about just making it really obvious:

  • Rename testing framework to "PHPUnit based testing"
  • Keep the existing Testing topic, without anyone in there
  • Keep the simpletest in order to make it clear that there is some legacy framework we have to maintain
xjm’s picture

Also see #2786145: Clarify how multiple maintainers share a role.

Ideally I think our PHPUnit-based tests, our legacy SimpleTests, our test runner, and maybe even future hypothetical qunit tests would all be maintained by a single team of people, even if some of those people individually said "I am SO not doing anything with SimpleTest" or "JS what?"

OTOH @dawehner's suggestion is the minimal scope to add the entry. I just am trying to encourage each governance change to MAINTAINERS.txt go in the direction we want of fewer components and more teams, rather than away from it.

dawehner’s picture

Ideally I think our PHPUnit-based tests, our legacy SimpleTests, our test runner, and maybe even future hypothetical qunit tests would all be maintained by a single team of people, even if some of those people individually said "I am SO not doing anything with SimpleTest" or "JS what?"

So you argue certainly more in the direction of the patch? I've adapted it and removed the Simpletest side of things.

On top of that though I'm wondering whether this discussion: topic maintainer vs. subsystem maintainer actually opens up a different question. Conceptually tests
are in parallel to the actual files which produce the value. Given that its not an integral part of the system, and it belongs more into the topic region. This also again opens
up the wormhole of actually having the testing framework be its own package.

PS: I still think it would be beneficial to categorize comments from committers to know what are discussions and what are decisions.

xjm’s picture

@dawehner, to me the direction of the current patch makes sense (if we undo the removal of the Testing topic which is actually sort of an out of scope governance change), but it matters more what the maintainers for the proposed component think than what I think.

If I make a decision, I'll mark something Needs work. If I'm not sure and just want to propose a suggestion, I'll mark it Needs review. (I'm not sure how any of my extremely wishy-washy "I think" and "I wonder" and "I can see either way" and "it could be worth" could be construed as a decision or even a recommendation, but I guess I'm relying a lot on subtleties of language and tone that might not be very clear in text or the same across languages.) :)

Does an issue exist already for the idea of making the testing framework its own component? It is a can of worms for sure, and I'm not sure we could change it in 8.x even if we all agreed it was the best choice and solved every concern, but it keeps coming up in a lot of different contexts so it seems like it's worth discussing and exploring somewhere in the queue. Maybe the ideas queue.

dawehner’s picture

Here is the patch file which removes the simpletest component.

Does an issue exist already for the idea of making the testing framework its own component? It is a can of worms for sure, and I'm not sure we could change it in 8.x even if we all agreed it was the best choice and solved every concern, but it keeps coming up in a lot of different contexts so it seems like it's worth discussing and exploring somewhere in the queue. Maybe the ideas queue.

I don't think so, well, I just fear that this further defers work on the BTB conversions.

xjm’s picture

#17 looks good to me, if the other maintainers for the component are comfortable with it.

klausi’s picture

Status: Needs review » Reviewed & tested by the community

Good enough for me.

klausi’s picture

Title: Add a phpunit component to MAINTAINERS.txt » Add a Testing framework component to MAINTAINERS.txt
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed 1b5802d to 8.3.x and 86a61e2 to 8.2.x. Thanks!

  • alexpott committed 1b5802d on 8.3.x
    Issue #2808123 by dawehner, klausi, xjm: Add a Testing framework...

  • alexpott committed 86a61e2 on 8.2.x
    Issue #2808123 by dawehner, klausi, xjm: Add a Testing framework...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.