Problem/Motivation

Features maintainers want to have a solid set of test coverage in place prior to releasing a first beta.

Proposed resolution

This meta issue will track overall progress. Specific tests can be added in child tickets.

Remaining tasks

A solid start has been made on unit tests-- thanks @jhedstrom!

That said, there's plenty left to do, in both unit and functional testing.

Comments

nedjo’s picture

Version: » 8.x-1.x-dev
nedjo’s picture

Project: Configuration Packager » Features
Version: 8.x-1.x-dev » 8.x-3.x-dev
Issue summary: View changes
jhedstrom’s picture

Status: Active » Needs review
StatusFileSize
new2.56 KB

As suggested during a presentation today, here is a starter that can serve as an example for others to contribute tests. It tests a single method on the FeaturesManager class.

jhedstrom’s picture

StatusFileSize
new1.86 KB
new3.97 KB

I realized I couldn't stop at just one test after doing all the setup!

nedjo’s picture

Priority: Normal » Major

This looks like a great start, thanks!

Does this need any updating given subsequent code changes?

Raising priority, since lack of tests is a beta blocker. We should aim to get this applied as soon as possible.

+++ b/src/FeaturesManagerInterface.php
@@ -31,7 +31,7 @@ interface FeaturesManagerInterface {
+  const STATUS_DEFAULT = self::STATUS_NO_EXPORT;

This bug fix is now covered in #2488838: Notice: Use of undefined constant STATUS_NO_EXPORT so should be removed.

@jhedstrom: can you explain a bit your thoughts on what will be possible to cover in unit tests (PHPUnit)and what will need functional tests (Simpletest)?

Also, since this is the base testing for Features and will serve as a model for what comes, what would you think about fleshing out the code comments a bit for the benefit of other developers less familiar with what's involved? Some suggestions....

  1. +++ b/tests/src/Unit/FeaturesManagerTest.php
    @@ -0,0 +1,110 @@
    +  public function setUp() {
    

    Some inline documentation of the steps here would be good.

  2. +++ b/tests/src/Unit/FeaturesManagerTest.php
    @@ -0,0 +1,110 @@
    +    return [
    

    A comment for each of the arrays here would be good. What do they represent?

  3. +++ b/tests/src/Unit/FeaturesManagerTest.php
    @@ -0,0 +1,110 @@
    +    $packages = ['foo' => 'bar'];
    

    What does this data structure represent?

jhedstrom’s picture

@jhedstrom: can you explain a bit your thoughts on what will be possible to cover in unit tests (PHPUnit)and what will need functional tests (Simpletest)?

I think a good goal would be to have some reasonable amount of PHPUnit coverage for most (if not all) classes. This has to be balanced with the difficulty in mocking certain items, so folks shouldn't go crazy.

In Drupal 8 SimpleTest has effectively been split into 2 different types of tests: Web tests (which do a full Drupal site install for every test method), and Kernel (these are new to D8 and only install database tables as requested, which makes them almost as fast as PHPUnit).

I think the majority of the functional and integration tests will be doable as Kernel tests. Using web tests can be saved for testing the various UIs provided by features.

I'll try to flesh out the comments in the above patch sometime this week.

mpotter’s picture

Status: Needs review » Fixed

I removed the STATUS_NO_EXPORT line and then committed this. We can improve comments through Features in other issues. But I really wanted to get the start of test coverage so people can work on improving this.

Thanks!

mpotter’s picture

Status: Fixed » Needs work

Looks like the tests are not working (https://qa.drupal.org/pifr/test/997213), so re-opening this for more work.

nedjo’s picture

Almost all of the config module testing in core is done with simpletest. I recently added some basic simpletest testing to Configuration Share, #2572323: Add tests for config_share.

Yes, it would be ideal to have unit test coverage. But a potential interim approach would be to focus first on functional tests, using test modules that provide configuration, drawing where appropriate on the core tests for config module. Those could at least cover broad areas of functionality.

jhedstrom’s picture

When possible, using KernelTestBase is much faster than the full WebTestBase since it doesn't do a full site install, but only instantiates the container, and module schemas as needed. I agree that trying to do too much in unit tests as a starting point isn't the best way to proceed.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new614 bytes

This one-liner gets current tests passing again.

Status: Needs review » Needs work

The last submitted patch, 12: 2383435-12.patch, failed testing.

nedjo’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 12: 2383435-12.patch, failed testing.

nedjo’s picture

@jhedstrom: thanks for fixing this up. I'm not sure what's up with the test bot. All features 3.x tests are passing for me locally.

jhedstrom’s picture

Test bot just can't apply the patch since it was committed above :)

nedjo’s picture

Title: Test coverage » META: Features 3.x test coverage
Issue summary: View changes
Status: Needs work » Active

Ah, thanks, that'd do it!

Changing this to a meta issue. Thinking we can open child tickets for specific tests.

mpotter’s picture

mpotter’s picture

Status: Active » Fixed

I think tests are working well now and we have good coverage. Closing this.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.