Filter format assertion randomly failing

Since this pattern is used in many tests, but only a specific one fails, I suspect the problem to be located in that filter test only.

This is interesting.

I just ran this test locally and it failed with the same error as in those qa.d.o pages.

Some debugging info:
- On line 453, the test creates the filter formats. It adds both 'tkhmIfWS' and 'xH5IhkNK' successfully (in this test run, those are the random names), in that order.
- On line 456, it should have loaded both these by name. This is not checked by an assertion.
- On line 458, it should have assigned $this->allowed_format to the first, and $this->disallowed_format to the second. This is not checked by an assertion.
- On line 499, it goes to the node/add page, and then on line 500 (the failure line) it checks to see if the allowed format is there, and fails. However, when I look at the debug output for that page, that first name is there.

AHA!

Here's the problem. The text it is looking for is coming from:

  function formatSelectorHTML($format) {
    return '<option value="' . $format->format . '">' . $format->name . '</option>';
  }

But the HTML source I am seeing in the debug output is:

<option value="cvx2lfsf" selected="selected">tkhmIfWS</option>

i.e. the new format is the default, whereas the HTML it is looking for doesn't have selected="selected" in there.

So that's the problem, and that's why it is only happening sometimes -- I guess sometimes the new format is not the default selected option. I'm guessing here...

I don't think the idea in #1 is the problem at all.

Here's a patch that I think resolves the problem, by not assuming one way or the other that when it looks for a filter format on the page, it's either selected or not selected. I'm just running the test locally, but thought I'd might as well let the test bot at it as well...

It passed my tests. I'm going to run the test locally a few more times for the random aspect...

Subscribe. No time to review now, but +1 to getting this fixed. It's been very annoying this last week especially, with bot failing good patches with an annoyingly high frequency.

Thanks, jhodgdon! Your awesome analysis allowed me to work out a proper fix for this random failure.

That helper function is the bug in the first place. The test has to use a xpath to retrieve the available formats.

We do not care for the "selected" property in this test, since this test is solely about text format access, nothing else.

Attached patch is ready to be committed.

Ok, let's try this. It'd be great to get rid of these random test failures because they've been sucking time from core devs the past week or so.

I asked sun about that helper function, but he confirmed it's only used by this test.

Committed #7 to HEAD.

Agreed, sun's is a better test. :)