Problem/Motivation

Adding the patch in #40 here: #2105583: Add some sane strictness to phpunit tests to catch risky tests we see risky tests including EntityManagerTest

You can reproduce this even without the above patch as follows.

php vendor/bin/phpunit -c core/phpunit.xml.dist --verbose --report-useless-tests --filter EntityManager
PHPUnit 4.8.11 by Sebastian Bergmann and contributors.

Runtime:	PHP 5.6.4-4ubuntu6.3 with Xdebug 2.2.6
Configuration:	/.../core/phpunit.xml.dist

.............................................R......

Time: 4.46 seconds, Memory: 177.25Mb

There was 1 risky test:

1) Drupal\Tests\Core\Entity\EntityManagerTest::testOnFieldDefinitionUpdate
This test did not perform any assertions

OK, but incomplete, skipped, or risky tests!
Tests: 52, Assertions: 128, Risky: 1.

Digging in, it turns out that a lot of the tests aren't asserting everything they think they are because the phpunit method getMockClass() doesn't seem to work for settings up assertions. I can't find any documentation of this, or even any documentation of getMockClass() but it explains the why this test is "risky" despite asserting method calls.

You can see this in the fact that the mock assertion is even asserting the wrong number of arguments to onFieldDefinitionUpdate().

Proposed resolution

Remove usage of getMockClass and replace them with prophecied mocks.

Remaining tasks

N/A

User interface changes

N/A

API changes

N/A

Data model changes

N/A

RC review: Testing fixes.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

neclimdul created an issue. See original summary.

neclimdul’s picture

neclimdul’s picture

Status: Active » Needs review
Mile23’s picture

I looked at this one, too, but I don't know enough prophecy and can't learn it right now. I'd wager that it's best to use one mocking system or the other but not both. :-)

Also, this illustrates how bad an idea it is to hold mocks as class properties (eg $this->entityManager;), because it's unclear where they came from, what has modified them, or what expectations they have. They should be either factory functions or defined in the test function.

But that's just me griping and way out of scope here. :-)

I can't quite review the prophecy use, but I can verify that the patch works to make the strict error go away.

$ ./vendor/bin/phpunit -c core/ -v --report-useless-tests --filter EntityManagerTest
PHPUnit 4.8.11 by Sebastian Bergmann and contributors.

Runtime:	PHP 5.5.26 with Xdebug 2.2.7
Configuration:	/Users/paul/pj2/drupal/core/phpunit.xml.dist

...................................................

Time: 16.94 seconds, Memory: 177.75Mb

OK (51 tests, 131 assertions)
neclimdul’s picture

It was my initial suspicion it was from mixing. I think it turns out from what I can tell its a weakness in getMockClass() specifically. So it would have been happening before EntityManagerTest started using prophecy. I didn't dig into the details but because its just handing around a classname and the the mock is instantiated elsewhere, my suspicion is that the problem is that the mock never gets registered in the mockObjects list.

This test is the only place we use that method in our entire test suite and its easy enough to work around currently by mocking and pushing the storage handler into the test manager.

Will see if I can get a prophet to review the modified code. ;)

tim.plunkett’s picture

Status: Needs review » Reviewed & tested by the community

LGTM. Getting rid of usages of getMockClass is always a good thing.
Thanks @neclimdul!

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed ff04d6a and pushed to 8.0.x. Thanks!

  • alexpott committed ff04d6a on 8.0.x
    Issue #2597844 by neclimdul: EntityManagerTest doesn't assert everything...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.