Problem/Motivation

The ability to mark entities uncacheable is broken ^8.7 and tests are broken.

Currently there are two tests failing on HEAD:

  • \Drupal\Tests\field_encrypt\Unit\EncryptedFieldValueManagerTest
  • \Drupal\Tests\field_encrypt\Functional\FieldEncryptCacheTest

This prevents contributors and maintainers from benefiting of existing tests until we get the branch passing.

Proposed resolution

Identify if the failures are correct, and if so fix the bugs, otherwise fix the tests.

Remaining tasks

Do it.

User interface changes

None.

API changes

None.

Data model changes

None.

Release notes snippet

None.

Comments

Manuel Garcia created an issue. See original summary.

manuel garcia’s picture

Status: Active » Needs review
StatusFileSize
new1.92 KB

This should get EncryptedFieldValueManagerTest passing again.

As far as FieldEncryptCacheTest goes I fear it may be a legitimate failure. In any case I've not been able to figure it out, and would love some help on it :)

Status: Needs review » Needs work

The last submitted patch, 2: 3124830-2.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

manuel garcia’s picture

Status: Needs work » Needs review
StatusFileSize
new11.36 KB
new13.28 KB

I've spent some time trying to figure out why FieldEncryptCacheTest is failing, but I cant figure out why.

Perhaps this is expected behaviour? The testDynamicPageCache is passing which is great.

the node entity type isPersistentlyCacheable returns false, yet \Drupal::cache('entity')->get($cid) does return the test node entity in it.

I have also noticed that when we set the node to be uncacheable it was failing as well, and managed to fix that by saving the node after changing its field storage settings.

Took the liberty of improving the base test class:

  1. Since #2692387: Define and use base class for simple tests got in, we can now greatly simplify our base test class by extending EncryptTestBase.
  2. Use NodeCreationTrait.
  3. Use the EntityTypeManager instead of the deprecated entity manager.

And removing setFieldStorageSettings() from FieldEncryptCacheTest since we have a better version on FieldEncryptTestBase.

manuel garcia’s picture

StatusFileSize
new2.35 KB
new24.19 KB
PHP Fatal error:  Class 'Drupal\Tests\encrypt\Functional\EncryptTestBase' not found in /var/www/html/modules/contrib/field_encrypt/tests/src/Functional/FieldEncryptTestBase.php on line 14

Meaning we need to pull in Encrypt module for the test bot, so adding it to composer.json as requirement.

manuel garcia’s picture

StatusFileSize
new708 bytes
new13.97 KB

Argh wrong patch! Please ignore the files on #5.

multitasking--

manuel garcia’s picture

StatusFileSize
new353 bytes
new13.97 KB

Docs for this: https://www.drupal.org/docs/develop/using-composer/managing-dependencies...

Let's see if it works this time...

manuel garcia’s picture

Status: Needs review » Postponed

OK, its not finding the encrypt base class because it needs this patch to be applied, which i had on my machine from working on it yesterday.

#3124749: Convert simpletests to PHPUnit

I'd say lets wait for that to get in before working more on this.

manuel garcia’s picture

manuel garcia’s picture

Status: Postponed » Needs review

#3124749: Convert simpletests to PHPUnit got in, so this is now unblocked :)

manuel garcia’s picture

StatusFileSize
new335 bytes
new13.97 KB

OK its now failing because the base test class is only available on the latest tarball of Encrypt. I believe for tests there is benefit in testing against Encrypt HEAD so we can identify issues before a release is cut, in theory catching bugs before they hit sites.

Status: Needs review » Needs work

The last submitted patch, 11: 3124830-11.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

manuel garcia’s picture

Status: Needs work » Needs review
StatusFileSize
new2.43 KB
new14.73 KB

OK, great to see the tests running now.

About the last failing test, I've come to the conclusion that we should only be testing what the module does, which is altering the entity type definition on field_encrypt_entity_type_alter() to mark them as not cacheable. We're already testing this which should be enough imho:

// Check whether node entities are marked as uncacheable.
$this->entityTypeManager->clearCachedDefinitions();
$definition = $this->entityTypeManager->getDefinition('node');
$this->assertFalse($definition->isPersistentlyCacheable());
$this->assertFalse($definition->isRenderCacheable());
$this->assertFalse($definition->isStaticallyCacheable());

In other words, testing core's caching mechanism is out of scope, so I'm removing that part of the test.

Also removing an unused use statement.

vijaycs85’s picture

  1. +++ b/composer.json
    @@ -1,5 +1,8 @@
    +    "drupal/encrypt": "3.x-dev"
    

    I don't think we could do require-dev here and make encrypt as a dependency in the info file. Either we should move to `require` or make it test_dependecies in info file?

  2. 
    +++ b/tests/src/Unit/EncryptedFieldValueManagerTest.php
    @@ -105,14 +106,18 @@ class EncryptedFieldValueManagerTest extends UnitTestCase {
    +        'getEncryptedValue',
    +        'hasTranslation',
    +        'getTranslation',
    +        'setEncryptedValue',
    +        'save',
    

    mocking more methods since they are not relevant?

vijaycs85’s picture

I also agree on on #13 on testing only what is updated by this module.

manuel garcia’s picture

Thanks @vijaycs85 for the review!

Re #14.1:
The reason for that is that since #3124749: Convert simpletests to PHPUnit got in 9 days ago, the EncryptTestBase class we're extending is only available on Encrypt dev until they make a release. I explained this on #11 also noting that there could be benefits of testing against dev, up to the maintainers of course.

Re #14.2:
Not a Unit tests expert here... I added these because otherwise there are failures on testGetEncryptedFieldValue() like:
Trying to configure method "setEncryptedValue" which cannot be configured because it does not exist, has not been specified, is final, or is static If it's not the right fix then please let me know - happy to learn :)

Re #15: 👍

jeroent’s picture

I postponed #3111904: Drupal 9 deprecated code report on this issue, since this issue already fixes some deprecations and it makes it easier to test Drupal 9 compatibility once the tests are fixed.

nerdstein’s picture

I'm coordinating with @rlhawk to get a release of encrypt out for Drupal 9 support. I believe this would help address the concern in #16.

rlhawk’s picture

The new version of Encrypt has been released.

suzymasri’s picture

StatusFileSize
new14.4 KB
new361 bytes

Updated patch to remove Encrypt dependency from composer.

manuel garcia’s picture

Thanks @suzymasri, great to see we can now just depend on the latest encrypt module release.

I think think this is ready to go :)

vijaycs85’s picture

Status: Needs review » Reviewed & tested by the community

Looks good. I tried to see if we could clean up drupal ci related files, it seems we are not there yet.

tame4tex’s picture

Status: Reviewed & tested by the community » Needs work
Related issues: +#3107865: Unable to turn off entity caching since 8.7.0 release, +#3081143: Drupal Core 8.7 compability

I do have some concern regarding the changes to FieldEncryptCacheTest::testEntityCache. Part of the reason that test was failing was because this module no longer prevents the entity from being cached. I think this is something we need to be concerned with because of the security implications.

Sure, the changes are testing the entity definition is correctly set but are we not just perpetuating the false assumption that this module prevents entities with encrypted fields from being cached when it doesn't.

In a nutshell field_encrypt_entity_type_alter() is no longer doing what it is expected to do, so is it not correct that the test fails?

If we fix this issue Drupal Core 8.7 compability then I am assuming this test will pass.

This issue provides more explanation as to why changes to the entity cache settings are no longer working:
Unable to turn off entity caching since 8.7.0 release

Thoughts?

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new8.47 KB
new17.13 KB

@tame4tex that's a great point. I've merged in #3081143: Drupal Core 8.7 compability and fixed it so that entity definitions are maintained as entity types become cacheable or uncacheable according to field_encrypt.

Also these changes have the added benefit of not needing a cache clear if field encrypt cacheability changes.

alexpott’s picture

Title: Fix failing tests on 8.x-2.x » Fix project on Drupal ^8.7 and fix failing tests on 8.x-2.x
Priority: Normal » Critical
Issue summary: View changes

Marked #3107865: Unable to turn off entity caching since 8.7.0 release as a duplicate - @mdolink should get credit here.
Marked #3081143: Drupal Core 8.7 compability as a duplicate - @ytsurk should get credit here.

vijaycs85’s picture

Status: Needs review » Reviewed & tested by the community

Thanks @alexpott. It looks much better and seems cleaned other issues as well. I think we are good to go.

alexpott credited ytsurk.

alexpott’s picture

Crediting people as per #25

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Crediting all the contributors and committing - thanks!

  • alexpott committed b000309 on 8.x-2.x authored by Manuel Garcia
    Issue #3124830 by Manuel Garcia, alexpott, suzymasri, vijaycs85, ytsurk...
manuel garcia’s picture

Fantastic, thanks so much @alexpott for that!

tame4tex’s picture

Excellent @alexpott and very happy to hear you are on board as a new maintainer of this module. Thank you to everyone for the efforts on this. It is fantastic to have passing tests again!

There is still an issue with caching though on existing sites. The current code doesn't update the last installed definition of existing entity types that are already listed in uncacheable_entity_types. We need an update script to do that.

I am also thinking we need to ensure the cache properties are set correctly on any entity type definition update.

I have created a new issue for this:
Cache issue persists after update to 8.x-2.0-alpha2

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.