[TESTING BROKEN] FileFieldValidateTestCase fails randomly [#667264]

The automatic test FileFieldValidateTestCase fail random..

In some patch that don't touch in FileField, or only in javascript, that nothing related with filefield have to be re-tested because the simpletest fail in FileFieldValidateTestCase

Error message:

FileFieldValidateTestCase	101	1	7
Message	Group	Filename	Line	Function	Status
array_flip(): Can only flip STRING and INTEGER values!	Warning	entity.inc	108	DrupalDefaultEntityController->load()	
Trying to get property of non-object	Notice	file.test	374	FileFieldValidateTestCase->testRequired()	
Undefined property: stdClass::$uri	Notice	file.test	146	FileFieldTestCase->assertFileExists()	
File exists after uploading to the required multiple value field.	Other	file.test	375	FileFieldValidateTestCase->testRequired()	
Undefined property: stdClass::$fid	Notice	file.test	154	FileFieldTestCase->assertFileEntryExists()	
array_flip(): Can only flip STRING and INTEGER values!	Warning	entity.inc	108	DrupalDefaultEntityController->load()	
Trying to get property of non-object	Notice	file.test	156	FileFieldTestCase->assertFileEntryExists()	
Undefined property: stdClass::$uri	Notice	file.test	156	FileFieldTestCase->assertFileEntryExists()

Comment	File	Size	Author
#22	667264-fix-remaining-failures.patch	2.58 KB	Damien Tournoud
#22
#19	667264-random-file-related-failures.patch	7.03 KB	Damien Tournoud
#19
#13	667264-random-file-related-failures.patch	6.31 KB	Damien Tournoud
#13
#9	667264-random-file-related-failures.patch	5.89 KB	Damien Tournoud
#9

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

int CreditAttribution: int commented 24 December 2009 at 17:44

Title:

The automatic test FileFieldValidateTestCase fail randomly

» The automatic test FileFieldValidateTestCase fail random

Like:
http://qa.drupal.org/pifr/test/22976
http://qa.drupal.org/pifr/test/23248
http://qa.drupal.org/pifr/test/23258
http://qa.drupal.org/pifr/test/23282
http://qa.drupal.org/pifr/test/23286
http://drupal.org/node/582622#comment-2405160

Two testing servers and one give OK in testing HEAD, and other no.
http://qa.drupal.org/pifr/test/32
http://qa.drupal.org/pifr/test/33

Comment #2

int CreditAttribution: int commented 24 December 2009 at 11:24

Title:

The automatic test FileFieldValidateTestCase fail random

» The automatic test FileFieldValidateTestCase fail randomly

Comment #3

int CreditAttribution: int commented 24 December 2009 at 11:31

I don't know if this is true, but automatic test should never be created with random content. The automatic test should be reproduced, exacly the same variables.

e.g. Instead be one variable with random content, should be 3 variables with static content, with that 3 variables with almost possible text.

Comment #4

Dave Reid

he/him

English

Nebraska USA

CreditAttribution: Dave Reid commented 24 December 2009 at 20:26

Title:	The automatic test FileFieldValidateTestCase fail random	» The automatic test FileFieldValidateTestCase fail randomly
Priority:	Normal	» Critical

Test bot has been disabled until we can figure this out.
http://qa.drupal.org/node/68

Comment #5

Anonymous (not verified) CreditAttribution: Anonymous commented 24 December 2009 at 21:44

subscribe. nasty stuff.

Comment #6

sun

German

Karlsruhe

CreditAttribution: sun commented 24 December 2009 at 22:21

Title:	The automatic test FileFieldValidateTestCase fail randomly	» [TESTING BROKEN] FileFieldValidateTestCase fails randomly
Component:	field system	» file.module

Also investigated this today... I ran this test multiple times on my local machine (Windows), always passed 100%. AFAICS, the test does not seem to do anything scary.

What I didn't test is running tests in concurrency. Did anyone try this?

Comment #7

arhak CreditAttribution: arhak commented 24 December 2009 at 22:38

also http://qa.drupal.org/pifr/test/23068

and a different failure is at http://qa.drupal.org/pifr/test/23044 (my own patch not related with FileField at all)
this case HAD (because now it passed) 2 fails and 14 exceptions, which might help to alight the issue
- 1 fail was "File exists after uploading to the required multiple value field." with 7 exception around like the other tests failing
- the other fail was similar but for for the single value field (also with 7 exceptions around it, the same exception's messages it got for the multiple value field)

Comment #8

Damien Tournoud CreditAttribution: Damien Tournoud commented 25 December 2009 at 17:19

The probable culprit is FileFieldRevisionTestCase::testRevisions(). Files are shared between the test instances, and this test actually physically removes files.

The only correct fix is to copy all the test files to the new files directory in DrupalWebTestCase::setUp().

Comment #9

Damien Tournoud CreditAttribution: Damien Tournoud commented 25 December 2009 at 17:38

Status:

Active

» Needs review

File	Size
667264-random-file-related-failures.patch	5.89 KB

If I'm right, this should fix it.

Comment #10

Anonymous (not verified) CreditAttribution: Anonymous commented 25 December 2009 at 17:59

Status:

Needs review

» Active

from #drupal:

(16:56:04) beejeebus: anyone got any clues about reproducing that?
(17:17:07) DamZ: beejeebus: I think I know what's going on
(17:17:33) DamZ: beejeebus: it's probable that FileFieldRevisionTestCase::testRevisions() is running at the same time as the failing one
(17:18:07) DamZ: beejeebus: the files are shared, and FileFieldRevisionTestCase::testRevisions() is removing the file needed by FileFieldValidateTestCase
(17:55:13) beejeebus: DamZ: oh, its a concurrency issue?
(17:56:10) DamZ: beejeebus: yeah, of course
(17:56:58) beejeebus: DamZ: oh, that should be easy to fix then

Comment #11

Damien Tournoud CreditAttribution: Damien Tournoud commented 25 December 2009 at 18:27

Status:

Active

» Needs review

Why changing the status?

Comment #12

cburschka

they

CreditAttribution: cburschka commented 26 December 2009 at 00:24

This is fairly ironic - unpatched filefield works fine for me, but the patched version runs into the similar errors as the test server.

Note the memory size failure claiming a memory limit of 64M, when the memory limit is actually set to 128M. This is unusual...

File field validation tests 92 passes, 10 fails, and 53 exceptions

Fail       PHP Fatal  database.inc                   1956                      
    Allowed memory size of 67108864 bytes exhausted (tried to allocate 48 bytes)
--
Exception  Notice     file.test                      29                        
    Trying to get property of non-object
Exception  Notice     file.test                      29                        
    Trying to get property of non-object
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      410                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      411                       
    File exists after uploading a file ( bytes) under the max limit (1M).
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      417                       
    Node save failed when file ( bytes) exceeded the max upload size (1M).
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      410                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      411                       
    File exists after uploading a file ( bytes) under the max limit (1024K).
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      417                       
    Node save failed when file ( bytes) exceeded the max upload size (1024K).
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      410                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      411                       
    File exists after uploading a file ( bytes) under the max limit (1048576).
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      417                       
    Node save failed when file ( bytes) exceeded the max upload size (1048576).
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      426                       
    Trying to get property of non-object
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      427                       
    File exists after uploading a file ( bytes) with no max limit.
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      29                        
    Trying to get property of non-object
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      453                       
    Undefined index:  und
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      454                       
    File exists after uploading a file with no extension checking.
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Fail       Other      file.test                      463                       
    Node save failed when file uploaded with the wrong extension.
Exception  Notice     file.test                      104                       
    Undefined property: stdClass::$uri
--
Exception  Notice     file.test                      471                       
    Undefined index:  und
Exception  Notice     file.test                      146                       
    Undefined property: stdClass::$uri
Fail       Other      file.test                      472                       
    File exists after uploading a file with extension checking.
Exception  Notice     file.test                      154                       
    Undefined property: stdClass::$fid
Exception  Warning    entity.inc                     108                       
    array_flip(): Can only flip STRING and INTEGER values!
Exception  Notice     file.test                      156                       
    Trying to get property of non-object
Exception  Notice     file.test                      156                       
    Undefined property: stdClass::$uri

Comment #13

Damien Tournoud CreditAttribution: Damien Tournoud commented 26 December 2009 at 01:26

File	Size
667264-random-file-related-failures.patch	6.31 KB

Nice catch! I forgot to reset the copiedTestFiles indicator, so when several tests of the same class are run, files are only copied the first time.

This actually proves that the random errors in the test bot are actually due to a missing file.

Comment #14

cburschka

they

CreditAttribution: cburschka commented 26 December 2009 at 02:08

I can run the FileFieldValidateTestCase class with 102 passes and 0 fails, but I am not convinced the test completes, because there are three occurrences of

Fail       PHP Fatal  database.inc                   1956                      
    Allowed memory size of 67108864 bytes exhausted (tried to allocate 48 bytes)

Oddly, these failures do not get counted. Multiple different checks show that PHP's memory limit is set to 256M (and in fact any deliberately induced crashes elsewhere show a limit of 256MB was exceeded). This is weird.

Comment #15

cburschka

they

CreditAttribution: cburschka commented 26 December 2009 at 11:59

Okay, so here's what happens.

I have this file in the folder for whatever reason:

Array
(
    [sites/clean/files/simpletest/mpletest734835/error.log] => stdClass Object
        (
            [uri] => sites/clean/files/simpletest/mpletest734835/error.log
            [filename] => error.log
            [name] => error
        )

It contains this:

[23-Dec-2009 10:50:49] PHP Fatal error:  Allowed memory size of 67108864 bytes exhausted (tried to allocate 48 bytes) in /home/arancaytar/Documents/devel/drupal/core/CLEAN/includes/database/database.inc on line 1956

I've cleaned out and reinstalled my site now and hope that the error goes away.

Edit:

All is well. 102 passes, no failures, no exceptions, and the phantom fails are gone.

Comment #16

bleen CreditAttribution: bleen commented 26 December 2009 at 17:54

superscribe

Comment #17

cburschka

they

CreditAttribution: cburschka commented 26 December 2009 at 23:08

Note that this is no longer the only test case that fails right now: #654854: Clean up conditions in comment_reply(), avoid node_view() when unnecessary.

Comment #18

bleen CreditAttribution: bleen commented 26 December 2009 at 23:11

@Arancaytar

#658314: $page variable in node.tpl.php is buggy

Comment #19

Damien Tournoud CreditAttribution: Damien Tournoud commented 27 December 2009 at 14:56

File	Size
667264-random-file-related-failures.patch	7.03 KB

In fact, there is no reason to pregenerate the files. The generation is fairly quick and those files are only used in a couple of tests. So I suggest we simply remove the pregeneration and generate the test files on-demand, the first time DrupalWebTestCase::drupalGetTestFiles() is called.

This should be more robust, and would avoid Arancaytar's issue where a phantom error.log was copied over and over to the new test environment.

Comment #20

int CreditAttribution: int commented 28 December 2009 at 11:49

We should commit the #19 path, or remove the breaking test. The testbot can't be so long stopped.

Comment #21

Dries CreditAttribution: Dries commented 28 December 2009 at 12:08

Status:

Needs review

» Fixed

Committed to CVS HEAD. Bring back the testbots p-uh-lease! Thanks DamZ.

Comment #22

Damien Tournoud CreditAttribution: Damien Tournoud commented 28 December 2009 at 13:29

Status:

Fixed

» Needs review

File	Size
667264-fix-remaining-failures.patch	2.58 KB

This should fix the remaining test failures. Some tests were doing unholy things with the test files.

Comment #23

int CreditAttribution: int commented 28 December 2009 at 13:30

Status:

Needs review

» Needs work

http://qa.drupal.org/pifr/test/32
Fail:
FileScanDirectoryTest (Found two, expected javascript files.)
ImageEffectsUnitTest (Division by zero)
ImageToolkitUnitTest (Division by zero)

Comment #24

int CreditAttribution: int commented 28 December 2009 at 13:30

Status:

Needs work

» Needs review

cross post

Comment #25

cburschka

they

CreditAttribution: cburschka commented 28 December 2009 at 13:31

Cross-post.

Comment #26

Damien Tournoud CreditAttribution: Damien Tournoud commented 28 December 2009 at 13:57

I ran the tests manually on qa-1 (my test slave). 100% pass.

Comment #27

Dries CreditAttribution: Dries commented 28 December 2009 at 14:00

Status:

Needs review

» Fixed

Committed the patch in #22. Good thing we're making progress. Thanks @DamZ. :)

Comment #28

int CreditAttribution: int commented 28 December 2009 at 23:45

Title:	[TESTING BROKEN] FileFieldValidateTestCase (and FieldInfoTestCase?) fails randomly	» [TESTING BROKEN] FileFieldValidateTestCase fails randomly
Status:	Active	» Fixed

Fixed issues should not be tested. should be automaticly removed from the queue.

Comment #29

bleen CreditAttribution: bleen commented 28 December 2009 at 20:41

Title:	[TESTING BROKEN] FileFieldValidateTestCase fails randomly	» [TESTING BROKEN] FileFieldValidateTestCase (and FieldInfoTestCase?) fails randomly
Status:	Fixed	» Active

I believe that this error is still occurring but now with "FieldInfoTestCase" ...

the patch in comment #3 of #658148: The "access user profiles" permission is poorly named is nothing more then a find/replace on the string "access user profiles" but it is failing inexplicably on "FieldInfoTestCase"

I'm not 100% certain, but I suspect that its related to this issue ... thoughts?

Comment #30

int CreditAttribution: int commented 28 December 2009 at 23:53

Title:	[TESTING BROKEN] FileFieldValidateTestCase fails randomly	» [TESTING BROKEN] FileFieldValidateTestCase (and FieldInfoTestCase?) fails randomly
Status:	Fixed	» Active

You have to find more 2 or 3 test fails like this..
But actualy is true, because the first test give the error, but re-tested don't give none.. But I don't find any issue with this problem

Comment #31

int CreditAttribution: int commented 29 December 2009 at 11:34

see this #669842: qa.drupal.org #32 is re-testing and re-testing over and over again, without ask for it

Comment #32

cburschka

they

CreditAttribution: cburschka commented 31 December 2009 at 15:54

Title:	[TESTING BROKEN] FileFieldValidateTestCase (and FieldInfoTestCase?) fails randomly	» [TESTING BROKEN] FileFieldValidateTestCase fails randomly
Status:	Active	» Fixed

The error doesn't appear to be happening anymore. Even if it is, this would be an unrelated error because FieldInfoTestCase is not copying any files anywhere.

If a non-file-copying tests is failing erratically again, let's open a new issue.