Says it in the title, sometimes the test bot fails a patch with an error in the Party CRUD Test Case, it says the test failed due to a fatal error. Often just retesting the patch fixes it.

This is irritating. We should fix it.

Test Links

http://qa.drupal.org/pifr/test/300398
http://qa.drupal.org/pifr/test/321668
http://qa.drupal.org/pifr/test/321893
http://qa.drupal.org/pifr/test/328213

Comments

captaindav’s picture

captaindav CreditAttribution: captaindav commented

Why don't you comment out part of the test and commit, then we could hone in on where the problem is?

rlmumford’s picture

rlmumford
Primary language English
Location Manchester
CreditAttribution: rlmumford commented

It really is exceptionally intermittent, It's happens maybe 1 our of 7 times, so ensuring we've found the problem could be arduous that way. Next time I spot it, I'll pull apart the QA logs more.

rlmumford’s picture

rlmumford
Primary language English
Location Manchester
CreditAttribution: rlmumford commented

These are the 2 fails

Message |	Group	|Filename	|Line	|Function	|Status
The test did not complete due to a fatal error.	|Completion check	|party.test	|90	|PartyTestCase->testPartyCRUD()	
Test Party found	|Other	|party.test	|101	|PartyTestCase->testPartyCRUD()

I also noticed this at the bottom of the full log:

Fatal error: Cannot access empty property in /var/lib/drupaltestbot/sites/default/files/checkout/modules/field/field.module on line 936
FATAL PartyUserTestCase: test runner returned a non-zero error code (255).
andrewbelcher’s picture

andrewbelcher CreditAttribution: andrewbelcher commented

I'm stumped... I tried putting an exit in modules/field/field.module on line 936 and it doesn't seem to get run at all...

andrewbelcher’s picture

andrewbelcher CreditAttribution: andrewbelcher commented
Priority: Critical » Major

Marking this down as I've not seen it in a while and I suspect it may be test server related... Need some more examples, so if you see it happen, please post post a link in this issue.

andrewbelcher’s picture

andrewbelcher CreditAttribution: andrewbelcher commented
Issue summary: View changes

Added a failed QA link

rlmumford’s picture

rlmumford
Primary language English
Location Manchester
CreditAttribution: rlmumford commented
Issue summary: View changes

Updated issue summary.

andrewbelcher’s picture

andrewbelcher CreditAttribution: andrewbelcher commented
Priority: Major » Critical

I've posted a support request on the testbot issue queue to see if people who know it better than me have any pointers!

Also marking as critical as it seems to be happening a lot more...

#1760296: field_get_items() causing fatal error on testbot but doesn't get called locally

joachim’s picture

joachim CreditAttribution: joachim commented

An intermittent test sounds like a race condition to me.

Have you turned out verbose output so you can see the pages the test sees?
On D7 there's the very useful debug() too, which shows up in the test results.

andrewbelcher’s picture

andrewbelcher CreditAttribution: andrewbelcher commented

I have never been able to reproduce this issue locally. What's even weirder is that locally, an exit in the function that's reporting the problem has never been run. It seems like there is something different happening when it fails, but I can't figure it out.

I have looked through all the verbose output and like I said, can't see anything what-so-ever...

What kind of thing did you have in mind? The only 'race condition' I can see potentially happening in there is where we create a party through the UI and then check it exits by checking that it was created using $this->assertRaw('Party 1', t('Test Party found'));. We do a similar for delete.

However, I thought tests got run in separate databases?

Even more than that, surely that should just result in a failed test? Or actually it should succeed without it necessarily having succeeded. It certainly shouldn't throw a fatal error in a piece of code that doesn't even seem to be getting run normally?

andrewbelcher’s picture

andrewbelcher CreditAttribution: andrewbelcher commented
Issue summary: View changes

Added link

rlmumford’s picture

rlmumford
Primary language English
Location Manchester
CreditAttribution: rlmumford commented
Priority: Critical » Major

We had this fixed, but it's come up again on a new test bot, one to keep an eye on. I can force retests on qa.d.o so I don't think this is critical anymore. Let's just keep an eye out.

rlmumford’s picture

rlmumford
Primary language English
Location Manchester
CreditAttribution: rlmumford commented
Issue summary: View changes

Added test link