Work in progress, I have some weird notices related to stream wrappers that I couldn't figure out yet.

Difference is huge...

php core/scripts/run-tests.sh --class "Drupal\file\Tests\CopyTest,Drupal\file\Tests\DeleteTest,Drupal\file\Tests\LoadTest,Drupal\file\Tests\MoveTest,Drupal\file\Tests\SaveDataTest,Drupal\file\Tests\SaveTest,Drupal\file\Tests\SpaceUsedTest,Drupal\file\Tests\ValidateTest,Drupal\file\Tests\UsageTest,Drupal\file\Tests\ValidatorTest"

HEAD

File copying 110 passes, 0 fails, and 0 exceptions
File delete 24 passes, 0 fails, and 0 exceptions
File loading 42 passes, 0 fails, and 0 exceptions
^[xFile moving 110 passes, 0 fails, and 0 exceptions
File save data 93 passes, 0 fails, and 0 exceptions
File saving 16 passes, 0 fails, and 0 exceptions
File space used tests 12 passes, 0 fails, and 0 exceptions
File validate 12 passes, 0 fails, and 0 exceptions
File usage 34 passes, 0 fails, and 0 exceptions
File validator tests 29 passes, 0 fails, and 0 exceptions

Test run duration: 5 min 21 sec

With Patch:

File copying 122 passes, 0 fails, and 0 exceptions
File delete 30 passes, 0 fails, and 0 exceptions
File loading 60 passes, 0 fails, and 0 exceptions
File moving 125 passes, 0 fails, and 0 exceptions
File save data 108 passes, 0 fails, and 0 exceptions
File saving 19 passes, 0 fails, and 0 exceptions
File space used tests 15 passes, 0 fails, and 0 exceptions
File validate 15 passes, 0 fails, and 0 exceptions
File usage 45 passes, 0 fails, and 0 exceptions
File validator tests 39 passes, 0 fails, and 2 exceptions

Test run duration: 19 sec
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir’s picture

Status: Active » Needs review
FileSize
8.51 KB
jibran’s picture

Please convert it. +1000000000000000

+++ b/core/modules/file/lib/Drupal/file/Tests/FileManagedUnitTestBase.php
@@ -8,25 +8,36 @@
+ * Base class for file unit tests that use the file_test module to test uploads and

More then 80 chars.

Status: Needs review » Needs work

The last submitted patch, 1: file-managed-dubt-2194085-1.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 1: file-managed-dubt-2194085-1.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review
FileSize
10.3 KB
3.08 KB

That was a fun one to figure out. DrupalUnitTestBase::(un)registerStreamWrapper() was completely messed up, no idea how no other tests are failing.

Status: Needs review » Needs work

The last submitted patch, 6: file-managed-dubt-2194085-6.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review
FileSize
10.77 KB
2.27 KB

Yeah, now I'm officially scared...

Status: Needs review » Needs work

The last submitted patch, 8: file-managed-dubt-2194085-8.patch, failed testing.

The last submitted patch, 8: file-managed-dubt-2194085-8.patch, failed testing.

Berdir’s picture

Berdir’s picture

Status: Needs work » Needs review

Green :)

sun’s picture

  1. +++ b/core/modules/simpletest/lib/Drupal/simpletest/DrupalUnitTestBase.php
    @@ -200,6 +200,7 @@ protected function tearDown() {
    +    $this->streamWrappers = array();
    

    Can we move this into setUp(), please?

    Whenever tests need to deal with global state constructs, they should ensure a clean environment in setUp(), because tearDown() is not necessarily reached (in case an exception is thrown).

  2. +++ b/core/modules/simpletest/lib/Drupal/simpletest/DrupalUnitTestBase.php
    @@ -448,9 +452,18 @@ protected function unregisterStreamWrapper($scheme) {
    +    $wrappers = &drupal_static('file_get_stream_wrappers', array());
    +    // @todo Some tests seem to reset the static cache to an explict
    +    //   NULL, for example the module_uninstall() call in
    +    //   \Drupal\field\Tests\FieldInfoTest::testInstanceDisabledEntityType().
    +    if (is_array($wrappers)) {
    

    The circumstance mentioned in the @todo is a bug in itself.

    file_get_stream_wrappers() is only able to operate on arrays.

    The tests should use drupal_static_reset('file_get_stream_wrappers') instead of setting NULL.

    Seems like an easy fix that we can quickly grep for?

sun’s picture

FileSize
18.37 KB
2.51 KB

Moved reset of $wrappers into setUp(), reverted additional changes.

Somehow, this passes for me locally?

Status: Needs review » Needs work

The last submitted patch, 14: test.file-managed.14.patch, failed testing.

Berdir’s picture

  1. +++ b/core/modules/simpletest/lib/Drupal/simpletest/DrupalUnitTestBase.php
    @@ -427,15 +427,12 @@ protected function registerStreamWrapper($scheme, $class, $type = STREAM_WRAPPER
         // @see https://drupal.org/node/2028109
    -    // Add the stream wrapper to the file_get_stream_wrappers() static cache,
    -    // only set the ALL type so that other types will be recalculated when
    -    // requested.
         $wrappers = &drupal_static('file_get_stream_wrappers', array());
    -    $wrappers[STREAM_WRAPPERS_ALL][$scheme] = array(
    +    $wrappers[$scheme] = array(
           'type' => $type,
           'class' => $class,
         );
    -    $wrappers[$type][$scheme] = $wrappers[STREAM_WRAPPERS_ALL][$scheme];
    +    $wrappers[STREAM_WRAPPERS_ALL] = $wrappers;
       }
    

    Nope, the validator test fails and the reason is that this code is completely weird, first it's adding it to the top level and then adds all of it recursively to the ALL wrapper.

    Maybe that function worked like this at one point, but it doesn't in HEAD. And fixing that then requires the other fixes, which are somehow magically hidden by this weird stuff ;)

  2. +++ b/core/modules/simpletest/lib/Drupal/simpletest/DrupalUnitTestBase.php
    @@ -452,18 +449,9 @@ protected function unregisterStreamWrapper($scheme) {
    -        }
    -      }
    -    }
    +    unset($wrappers[$scheme]);
    +    unset($wrappers[STREAM_WRAPPERS_ALL][$scheme]);
       }
     
    

    The unset relies the same weirdness and doesn't require any checks because unset($wrapper[whatever]) works fine as long as $wrapper is defined, even if the variable is NULL.

sun’s picture

Status: Needs work » Needs review
FileSize
20.54 KB
3.29 KB

That code exactly resembles what file_get_stream_wrappers() is doing internally at the end of the function.

Note that the static variable of file_get_stream_wrappers() contains both the registry of custom stream wrappers AND the type-filtered lists - on the same nesting level of the array.

What needs to be addressed is that a custom stream wrapper is not removed from all type-filtered lists.

→ Fixed unregistration of type-specific stream wrappers in DrupalUnitTestBase.

sun’s picture

FileSize
20.68 KB
1.52 KB

Fixed DrupalUnitTestBase does not properly resemble internals of file_get_stream_wrappers().

The last submitted patch, 17: test.file-managed.17.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 18: test.file-managed.18.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
20.85 KB
1.25 KB

Alright, this patch will pass + properly documents the possible edge-case. :-)

The possibility of a straw call to drupal_static_reset() at any time is actually what made me introduce this "parallel" registry in the DrupalUnitTestBase class in the first place, since the previously available information is simply gone. :)

→ Fixed drupal_static_reset() causes static variable in file_get_stream_wrappers() to be reset to NULL.

Berdir’s picture

Changes look good to me.

I'm still a bit confused by the drupal_static() behavior though.

I would assume that after a call to drupal_static_reset(), drupal_static('bla', array()) would return the passed default value and not NULL, which apparently still exists as an explicit value in there? I can see that file_get_stream_wrappers() calls it with no default value, so that would initialize it to NULL, but it also immediately .... oh I see. If file_get_stream_wrappers() is called and is empty, then it loads all stream wrappers but if system.module isn't around, then it doesn't find any and $wrappers_storage is never initialized as no value is ever set.

I think if we make sure that the default value is an empty array(), possibly even pre-initialize STREAM_WRAPPERS_ALL => array(), then we could IMHO better deal with that situation because the current would behavior would result in a constant re-initialization every time it is called and it is always an array?

Berdir’s picture

Status: Needs review » Needs work

The result is that the comment in the patch is wrong, the problem is not that file_get_stream_wrappers() is *not* called, the problem is that it *is* called but no stream wrapper hook was found.

sun’s picture

Status: Needs work » Needs review
FileSize
20.91 KB
1.17 KB

Fixed and clarified comment in unregisterStreamWrapper().

Berdir’s picture

What do you think about this?

This doesn't need that comment/check at all and it prevents us from invoking the hook possibly multiple times, even if it's just something that could happen in a DUBT test.

sun’s picture

Status: Needs review » Reviewed & tested by the community

Sure, that works, too :-)

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed 056096a and pushed to 8.x. Thanks!

Fixed on commit

diff --git a/core/modules/file/lib/Drupal/file/Tests/FileManagedUnitTestBase.php b/core/modules/file/lib/Drupal/file/Tests/FileManagedUnitTestBase.php
index de0b3f2..324b1b5 100644
--- a/core/modules/file/lib/Drupal/file/Tests/FileManagedUnitTestBase.php
+++ b/core/modules/file/lib/Drupal/file/Tests/FileManagedUnitTestBase.php
@@ -2,7 +2,7 @@
 
 /**
  * @file
- * Definition of Drupal\file\Tests\FileManagedTestBase.
+ * Contains Drupal\file\Tests\FileManagedUnitTestBase.
  */
 
 namespace Drupal\file\Tests;

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.