Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir’s picture

Status: Active » Needs review
FileSize
17.04 KB

Here is a patch.

Notes:
- Some of these tests have a quite a list of dependences, I added a base test class for entity tests to simplify the dependency/setUp() code.
- Moved over EntityPropertiesTest from field.module to the Entity tests and renamed to EntityLabelTest, that's the only thing that it does.
- As you can see in the screenshots, the test time in the UI goes down from 2min to 48s. It's still that high because there are a few web/form tests remaining and two that I didn't manage to convert:
- The entity access tests are weird. They do a drupalLogin() but that doesn't log in the user within the tests, I guess they are relying on the fact that by default the gobal user object for tests has uid 1. I suggest a separate issue to figure out what to do with those.
- Entity info tests special cases during the module enable process and should probably remain a web test.

klausi’s picture

Status: Needs review » Needs work
+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityApiTest.php
@@ -46,10 +37,10 @@ public function testCRUD() {
-   * @param \Drupal\user\Plugin\Core\Entity\User $user1

@param docs always have to be fully namespaced.

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityCrudHookTest.php
@@ -190,6 +200,8 @@ public function testCommentHooks() {
+    $this->installSchema('file', 'file_managed');

I don't like setup tasks in the actual test method. On the other hand, you would have to create separate classes for each then, which is also tedious :(

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.php
@@ -0,0 +1,47 @@
+/**
+ * @file
+ * Definition of Drupal\system\Tests\Entity\EntityApiTest.

Should be "Contains ...", see http://drupal.org/node/1354#files

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.php
@@ -0,0 +1,47 @@
+   * @return Drupal\user\Plugin\Core\Entity\User

missing "\"

Otherwise this looks like a good idea.

Berdir’s picture

Status: Needs work » Needs review

Thanks for the review!

Fixed the documentation issues. Yes, the hook tests is ugly but I think refactoring that into separate classes is out of scope in this issue and doing those table installations in the test methods makes the whole test class ~twice as fast.

Berdir’s picture

Status: Needs review » Needs work

The last submitted patch, entity-tests-dubt-1893108-3.patch, failed testing.

dawehner’s picture

This looks good from my perspective, beside of the ugly hook test case already mentioned before.

Berdir’s picture

Status: Needs work » Needs review
FileSize
1.17 KB
16.85 KB

role table is no more, this should pass again.

Berdir’s picture

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityCrudHookTest.phpundefined
@@ -41,6 +40,12 @@ public static function getInfo() {
+    $this->installSchema('user', 'users_data');

user data is a service, so we could write a memory implementation of that too but that's something for another issue I think :)

sun’s picture

Issue tags: +Test suite performance
alexpott’s picture

Status: Needs review » Reviewed & tested by the community

This halves the time the Entity API group of tests takes to tun... great work!

With patch

Drupal test run
---------------

Tests to be run:
 - Entity display configuration entities (Drupal\entity\Tests\EntityDisplayTest)
 - Entity properties (Drupal\field\Tests\EntityPropertiesTest)
 - Entity access (Drupal\system\Tests\Entity\EntityAccessTest)
 - Entity info (Drupal\system\Tests\Entity\EntityApiInfoTest)
 - Entity CRUD (Drupal\system\Tests\Entity\EntityApiTest)
 - Entity CRUD hooks (Drupal\system\Tests\Entity\EntityCrudHookTest)
 - Entity Field API (Drupal\system\Tests\Entity\EntityFieldTest)
 - Entity form (Drupal\system\Tests\Entity\EntityFormTest)
 - Entity Query relationship (Drupal\system\Tests\Entity\EntityQueryRelationshipTest)
 - Entity Query (Drupal\system\Tests\Entity\EntityQueryTest)
 - Entity revisions (Drupal\system\Tests\Entity\EntityRevisionsTest)
 - Entity translation form (Drupal\system\Tests\Entity\EntityTranslationFormTest)
 - Entity Translation (Drupal\system\Tests\Entity\EntityTranslationTest)
 - Entity UUIDs (Drupal\system\Tests\Entity\EntityUUIDTest)

Test run started:
 Tuesday, January 22, 2013 - 12:25

Test summary
------------

Entity display configuration entities 29 passes, 0 fails, and 0 exceptions
Entity properties 4 passes, 0 fails, and 0 exceptions
Entity access 34 passes, 0 fails, and 0 exceptions
Entity info 6 passes, 0 fails, and 0 exceptions
Entity CRUD 24 passes, 0 fails, and 0 exceptions
Entity CRUD hooks 149 passes, 0 fails, and 0 exceptions
Entity Field API 640 passes, 0 fails, and 0 exceptions
Entity form 77 passes, 0 fails, and 0 exceptions
Entity Query relationship 17 passes, 0 fails, and 0 exceptions
Entity Query 57 passes, 0 fails, and 0 exceptions
Entity revisions 53 passes, 0 fails, and 0 exceptions
Entity translation form 36 passes, 0 fails, and 0 exceptions
Entity Translation 136 passes, 0 fail, and 0 exceptions
Entity UUIDs 76 passes, 0 fails, and 0 exceptions

Test run duration: 2 min 1 sec

without patch

Drupal test run
---------------

Tests to be run:
 - Entity display configuration entities (Drupal\entity\Tests\EntityDisplayTest)
 - Entity properties (Drupal\field\Tests\EntityPropertiesTest)
 - Entity access (Drupal\system\Tests\Entity\EntityAccessTest)
 - Entity info (Drupal\system\Tests\Entity\EntityApiInfoTest)
 - Entity CRUD (Drupal\system\Tests\Entity\EntityApiTest)
 - Entity CRUD hooks (Drupal\system\Tests\Entity\EntityCrudHookTest)
 - Entity Field API (Drupal\system\Tests\Entity\EntityFieldTest)
 - Entity form (Drupal\system\Tests\Entity\EntityFormTest)
 - Entity Query relationship (Drupal\system\Tests\Entity\EntityQueryRelationshipTest)
 - Entity Query (Drupal\system\Tests\Entity\EntityQueryTest)
 - Entity revisions (Drupal\system\Tests\Entity\EntityRevisionsTest)
 - Entity translation form (Drupal\system\Tests\Entity\EntityTranslationFormTest)
 - Entity Translation (Drupal\system\Tests\Entity\EntityTranslationTest)
 - Entity UUIDs (Drupal\system\Tests\Entity\EntityUUIDTest)

Test run started:
 Tuesday, January 22, 2013 - 10:42

Test summary
------------

Entity display configuration entities 29 passes, 0 fails, and 0 exceptions
Entity properties 4 passes, 0 fails, and 0 exceptions
Entity access 34 passes, 0 fails, and 0 exceptions
Entity info 6 passes, 0 fails, and 0 exceptions
Entity CRUD 26 passes, 0 fails, and 0 exceptions
Entity CRUD hooks 157 passes, 0 fails, and 0 exceptions
Entity Field API 682 passes, 0 fails, and 0 exceptions
Entity form 77 passes, 0 fails, and 0 exceptions
Entity Query relationship 20 passes, 0 fails, and 0 exceptions
Entity Query 62 passes, 0 fails, and 0 exceptions
Entity revisions 53 passes, 0 fails, and 0 exceptions
Entity translation form 36 passes, 0 fails, and 0 exceptions
Entity Translation 138 passes, 0 fails, and 0 exceptions
Entity UUIDs 77 passes, 0 fails, and 0 exceptions

Test run duration: 4 min 8 sec

EDIT: The entity translation test had one of it's random failures... replaced "with patch" test data.

sun’s picture

Component: configuration system » entity system
Status: Reviewed & tested by the community » Postponed

Thanks for converting these!

However, since there are new calls to enableModules() and config_install_default_config() in this patch, let's wait for #1891516: Remove $install parameter from DrupalUnitTestBase::enableModules(), encourage individual schema tables, please.

Berdir’s picture

Status: Postponed » Needs review
FileSize
18.35 KB
32.25 KB

Fine with me.

But while we wait on there, here's an update that moves the label test specific types to entity_test.module (nicely shows the problem with hardcoded label() methods of EntityTest and a bunch of config entities. They essentially remove support for the label key/callback in entity info).

Also fixed EntityAccessTest:
- The hardest part was to actually make that test fail when it's supposed to as I had to create a non-uid 1 user
- Added very simple support to create users with permissions left out all the checks and validations done by WebTest. Not sure if we want to move that to TestBase?
- Also fixed the bogus isset() check that resulted in always displaying "null" in the test assertions..

Berdir’s picture

Removed an unecessary $modules.

Berdir’s picture

Ok, here is a new patch that removes all instances of enableModule() and instead uses installSchema() only.

This should mean that, apart from a few small changes like loading default config, this shouldn't conflict with #1891516: Remove $install parameter from DrupalUnitTestBase::enableModules(), encourage individual schema tables anymore.

Berdir’s picture

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityApiTest.phpundefined
@@ -22,6 +22,17 @@ public static function getInfo() {
+  public function setUp() {
+    parent::setUp();
+    $this->installSchema('entity_test', 'entity_test_mul');
+    $this->installSchema('entity_test', 'entity_test_mul_property_data');
+    $this->installSchema('entity_test', 'entity_test_rev');
+    $this->installSchema('entity_test', 'entity_test_rev_revision');
+    $this->installSchema('entity_test', 'entity_test_mulrev');
+    $this->installSchema('entity_test', 'entity_test_mulrev_property_data');
+    $this->installSchema('entity_test', 'entity_test_mulrev_property_revision');

This is why I asked for a $this->installSchema() in the other issue :)

Good news is that, thanks to these changes, we're down to 30s from initially ~2min on my system.

tim.plunkett’s picture

Well, the other issue does allow you to specify an array of tables from a module, so the hunk shown in #15 could be one method call after that other issue goes in.

Berdir’s picture

Issue tags: -Test suite performance

#14: entity-tests-dubt-1893108-14.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +Test suite performance

The last submitted patch, entity-tests-dubt-1893108-14.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
35.75 KB

Rerolled.

aspilicious’s picture

+++ b/core/modules/system/tests/modules/entity_test/entity_test.moduleundefined
@@ -251,3 +251,20 @@ function entity_test_entity_test_insert($entity) {
\ No newline at end of file

:(

Berdir’s picture

Status: Needs review » Needs work

The last submitted patch, drupal-1893108-19.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review
FileSize
10.9 KB
36.93 KB

Re-rolled, changed a lot of installSchema() to use an array() and adjusted EntityUriTest to use the new base class.

Execution time is now 36 seconds instead of 2m 13s.

dawehner’s picture

FileSize
4.26 KB
36.45 KB

Some small points which are not worth to be reviewed.

fago’s picture

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.php
@@ -0,0 +1,58 @@
+/**
+ * Tests the basic Entity API.
+ */
+class EntityUnitBaseTest extends DrupalUnitTestBase {

Documentation does not say it's just a base-test case, what it is - right? So should it be abstract also? Else improvements look just great!

dawehner’s picture

FileSize
36.75 KB
1.23 KB

Good point.

Berdir’s picture

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.phpundefined
@@ -32,6 +32,12 @@ public function setUp() {
+   *   Array of permission names to assign to user. Note that the user always
+   *   has the default permissions derived from the "authenticated users" role.

I'm not sure if that role even exists in a DUBT test and if it does, then it probably has no permissions :)

Would have to check to be sure but those permissions would be saved in a table that does not exist :)

(It works in the EntityAccessTest because we install the table there manually but only there. So you might want to document that this is necessary. I'm not doing it by default because all other tests don't need it.)

Status: Needs review » Needs work

The last submitted patch, drupal-1893108-26.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review
FileSize
1.13 KB
36.74 KB

Updated the comment.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
FileSize
36.77 KB
992 bytes

Some tiny nitpicking.

fago’s picture

Great, I agree this is RTBC!

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Faster tests++ :)

Committed and pushed to 8.x. Thanks!

sun’s picture

Status: Fixed » Needs work

Thanks all! This is a great improvement :)

Sorry for my late review, but I've discovered some issues:

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.php
@@ -0,0 +1,66 @@
+abstract class EntityUnitBaseTest extends DrupalUnitTestBase {

s/EntityUnitBaseTest/EntityUnitTestBase/

The *Test suffix normally denotes a final test class. The *Base suffix denotes an abstract/base class.

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.php
@@ -0,0 +1,66 @@
+  protected function createUser($values = array(), $permissions = array()) {

Really not happy about the "dual-purpose" here.

For unit tests, there should really be a cleanly separated createUserRole() method.

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitBaseTest.php
@@ -0,0 +1,66 @@
+    $account = entity_create('user', $values + array(
...
+      'status' => 1,
+    ));
+    $account->enforceIsNew();

1) $status defaults to 1 already, so I don't really see why we need to explicitly set it?

2) enforceIsNew() on a newly created entity object looks superfluous to me?

Berdir’s picture

Status: Needs work » Needs review
FileSize
11.7 KB

Thanks for the review.

- Renamed the test class, you're totally right of course, not sure why I missed that. A bit unfortunate because other patches are already starting to pick up that class.
- Separated the user role creation into a separate method.
- removed status but kept enforceIsNew(). The reason I'm using it that because there is no default user in a DUBT test, the first created user would be uid 1 which would make it impossible to do permission related tests. Adding that call allows me to specify a fixed uid when creating a new user.

sun’s picture

Much better, thanks! :)

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityQueryAggregateTest.php
@@ -12,7 +12,7 @@
- * @todo Use EntityUnitBaseTest provided by http://drupal.org/node/1893108.
+ * @todo Use EntityUnitTestBase provided by http://drupal.org/node/1893108.

Eh? This @todo refers to this issue here? ;)

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitTestBase.php
@@ -0,0 +1,73 @@
+    $account->enforceIsNew();

I did not really understand your reply. I understand the concern about uid 1, but fail to see how and why this method call would work around that situation in any way. I fear that some non-trivial magic is going on with this additional method call.

We should make sure to document that magic right above this call.

Berdir’s picture

Hah, fixed the @todo. That was a search/replace :)

Let me try to explain the enforceIsNew().

$user = entity_create('user', array('uid' => 2));
$user->save();

What will happen here is that the storage controller assumes this is an existing entity because the uid is set. So it will try to do an update query to update that entity, which won't do anything.

To prevent that, I'm calling enforceIsNew(). It doesn't hurt when there is no uid and it allows me to specificy a specific uid when needed.

Added an inline comment, is that clear enough?

sun’s picture

Ah-ha! :)

So you're actually not preventing uid 1 from getting created (by default). Instead, you require all callers to pass a random 'uid' in $values, which in turn gets unset and enforced to be new? I guess that could use some additional clarification in the phpDoc?

That said... I still do not understand why this trick would skip over uid 1 from being created (if there is none yet). enforceIsNew() only unsets the entity ID, no? In turn, the new ID would still be 1 if there is none yet...?

So... actually less ah-ha than I thought :P :D

Berdir’s picture

No, enforceIsNew() does not reset the uid, it does set a flag that says "I'm a new entity" *despite* having an id set.

Passing an uid is not required. If there is none, it is new anyway, the user storage controller will detect that and request a new id from db_next_id(). But if there is one, it will save it as a new entity with that id.

You're welcome to provide a better description. I don't think I can explain it better :)

sun’s picture

Thanks! Hm... Perhaps I slowly get it now... :)

So let me try:

 * @param yada $values
 *   Yada.
 *   To perform access checks and prevent the new user from being uid 1
 *   (which is exempt from all user access conditions), pass a value
 *    other than 1 as 'uid' in $values.

...

  // Force the entity to be new, so that callers are able to specify
  // a 'uid' in $values to skip over uid 1. If the specified uid exists
  // already, UserStorageController will automatically choose and insert
  // the next available ID.
  $account->enforceIsNew();

Right, or still wrong? Thanks for enlightening me/us! :)

Berdir’s picture

Close :)

*If* you specify a uid, you are responsible for using one that is not yet used. If you try to create two new users with uid 2, it will explode. I think this is a case of "We assume you know what you are doing" and picking a different uid would be weird as you explicitly chose that it should be X and said it is new, so it must not yet exist.

A default is only created if you didn't specify any.

Maybe it would help to do something like this:

if (!empty($values['uid'])) {
  $user->enforceIsNew();
}

To explicitly document it but technically, it makes absolutely no difference if enforceIsNew() is always called or not, so it's kinda useless.

What about this? We shouldn't go too far to document this IMHO, EntityInterface::enforceIsNew() already documents how it works.

sun’s picture

Status: Needs review » Reviewed & tested by the community

Thank you!

I think my understanding level hovers around ~90% by now (still some gaps, but I'm also super busy with $dayjob right now), so I think we can move forward with this. :)

Perhaps @webchick wants to commit this ;)

Berdir’s picture

Thanks for the useful review. If you have remaining questions, maybe it can be better discussed in IRC :)

webchick’s picture

Status: Reviewed & tested by the community » Needs review
+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityAccessTest.phpundefined
@@ -62,7 +62,10 @@ function assertEntityAccess($ops, AccessibleInterface $object, User $account = N
-    $user = $this->createUser(array('uid' => 2), array('view test entity'));
+    $role = $this->createUserRole(array('view test entity'));
+    $user = $this->createUser(array('uid' => 2));
+    $user->roles[$role->id()] = $role->id();
+    $user->save();

Going from 1 line of code to 4 lines of code makes me :( but I guess I can see the logic in granular methods.

+++ b/core/modules/system/lib/Drupal/system/Tests/Entity/EntityUnitTestBase.phpundefined
@@ -0,0 +1,79 @@
+   * @param array $values
+   *   (optional) The values used to create the entity. To perform access checks
+   *   and prevent the new user from becoming uid 1 (which exempts from all user
+   *   access conditions), pass a unique value other than 1 as 'uid' in $values.
...
+  protected function createUser($values = array()) {

Whenever you need a book and a half to explain something, it's usually a sign that you should probably change the implementation. ;)

So I can understand totally why a user with the UID of 1 coming out of that function would totally screw any permission-based tests in an odd way. However, I would also posit that creating a non-UID1 user is the 99% case.

Therefore, how I would do this instead is something like:

if (!isset($values['uid'])) {
  // Always add one so you never end up with uid 1 unless you meant to.
  $values['uid'] = $database->getNextId() + 1; // I'm making up this function, but something like that.
}

So that you could still create a uid1 if you really wanted to for some reason, but you wouldn't need to worry about this detail at the "developer calling the function" level.

In other words, this:

$web_user = $this->createUser();

...is way cleaner than this:

$web_user = $this->createUser(array('uid' => 2)); // oops, no wait, 3, oops. I meant 4.. or 5 maybe?

Wondering if we should split that part out into its own patch and just do the rename here, which is totally uncontroversial and also blocks a few other things.

sun’s picture

Thanks for the review, @webchick!

Note that the double-saving of $user is a bug... The roles can be specified in the values being passed to createUser() already. Well spotted.

Regarding uid 1: It's actually only access/permission-related tests that may want to skip over uid 1. The vast majority of tests should actually use uid 1 or uid X, because user permissions should be irrelevant.

Closely related anecdote: In D8, tests are finally able to log in with $this->root_user (uid 1), which allows plenty of web tests to skip over the creation of test users and just log in. Since the facility is new and isn't widely known yet, we still have plenty of tests that still create custom "admin" users, even though that's not strictly required anymore. Leveraging the root user not only speeds up tests but also simplifies them.

Due to that, I do not think that we generally want to skip over uid 1. Instead, I imagine that there can be many tests that just need a user record.

If we wanted to keep this super simple, I could foresee the following two paths:

A) Make the test setup automatically create a $this->root_user, similar to a web test, and case closed (though with negative test performance impact).

B) Make each test that needs to create non-uid 1 users simply create $this->root_user first. Provide a helper method for that, but otherwise, as usual, in a unit test, you're on your own, case closed.

Both options would be a lot more clear than the currently proposed "magic." ;) I personally don't have strong preference between both, but I think I'd lean more towards B), partially because I think that a unit test shouldn't do anything automatically.

Berdir’s picture

Not sure about the root_user thing. I think an important aspect of writing tests is making sure that the access checks are correct, those are bugs that you often don't catch when testing with uid 1. So I would encourage to write tests with users that have an explicit permission set.

Berdir’s picture

As suggeted by @webchick, re-rolled with just the re-name, opened #1932780: Clean up EntityUnitTestBase::createUser() for the createUser part as this needs more discussion.

Status: Needs review » Needs work
Issue tags: -Test suite performance

The last submitted patch, entity-dubt-fixes-1893108-46.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review
Issue tags: +Test suite performance

#46: entity-dubt-fixes-1893108-46.patch queued for re-testing.

ParisLiakos’s picture

Status: Needs review » Reviewed & tested by the community

couldnt find anything to complain about and since most webchick's concerns are in another issue, i am rtbc'ing this

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Perfect, thanks!

Committed and pushed to 8.x.

Automatically closed -- issue fixed for 2 weeks with no activity.