Currently, all the Chrome tests are failing with this error message:

Could not open connection: An unknown server-side error occurred while processing the command. (Behat\Mink\Exception\DriverException)

This even effects re-running old builds that passed at the time, so it's not due to a code change on our end, but some dynamic part of our build.

I've tried updating to Selenium 2.53.1, but that caused all the Firefox tests to fail with :

Element is not clickable at point (282.6333312988281, 9.066665649414062). Other element would receive the click

I experimented with running the Chrome tests in Firefox instead, but that failed on the Media tests with switching to an iframe:

No frame element found by name or id mediaStyleSelector

I suspect that's just due to the iframe taking a little bit longer to load (we have a 2 second delay, but maybe that's not sufficient).

Anyway, this needs to be solved soon, or all development will stall!

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dsnopek created an issue. See original summary.

cboyden’s picture

The error with not being able to switch to the "mediaStyleSelector" frame is not a timing issue.

For some reason, the browser/driver is not able to click the img tag in the WYSIWYG to select it. Because the image is not selected, when the browser/driver clicks the Add Media button in the WYSIWYG, what opens up is not the style selector but the media browser. The iframes have different names. I discovered this by changing the line in the test to try to open the "mediaBrowser" iframe, and that step passed.

Screenshot of original test (with a delay added):
Behat test failing on step that switches to media style selector frame

Screenshot of altered test:
Behat test passing on step that switches to media browser frame

cboyden’s picture

Working on this at the DrupalCon sprint with @dsnopek. Updating to a later ubuntu version on Travis might cut down on some of the errors.

dsnopek’s picture

Here's panopoly_test patch to take a screenshot to try and help debug

dsnopek’s picture

FileSize
2.84 KB

Bah! That patch was broken. Here's a new version. Also, untested

cboyden’s picture

I believe part of the reason this is failing is that the files path in behat.template.yml was not updated when the paths were renamed recently. I'll try updating the above patch to include this change.

@dsnopek, in order to get this far I think you had to update Travis to the latest trusty build and make some changes to the MySQL setup. Can you post that patch here as well?

cboyden’s picture

Status: Active » Needs review
FileSize
3.41 KB

Patch to change files path attached.

cboyden’s picture

Actually the Travis files path is set differently. The change in the updated patch is still necessary for local testing, but shouldn't affect Travis.

Review of the screenshots generated by the test run for patch 4 indicate that the image is not being successfully uploaded, or not being rendered in either the WYSIWYG or in the node when you save it.

cboyden’s picture

Status: Needs review » Needs work

After doing some manual intervention in local testing (pausing the test and deleting the image entity after it's been inserted in the WYSIWYG the first time) I am able to cause the test to fail locally, even though it passes locally when run without interference.

Going through the test step by step, it appears that the browser/driver can select the broken image tag in the WYSIWYG. I'm working on a local child distribution that adds a class to images in the WYSIWYG editor when they are selected, to visually highlight them, and this is happening in the test even after the image entity has been deleted and the broken image is replaced in the WYSIWYG by the browser's default broken-image icon. So the custom step that clicks the img tag in the WYSIWYG is working the same way whether or not the image is loaded or rendered successfully. Doesn't seem like any change to this custom step would solve the problem, as it's not what's broken.

So it seems like the problem lies in the handoff from the selected image tag to the Media browser. If the selected image isn't recognized, the media browser opens instead of the media style selector.

We know from #2428097: Image styles won't generate on Travis-CI (breaking tests that need valid images) that image style generation doesn't work on Travis. My guess is that something changed recently in the browser or driver so that info about the broken-image icon is being passed to Media, instead of info about the expected image. If this can't be fixed in Media and/or the WYSIWYG editor, we'll have to rewrite the test to avoid the problem, or run the build on a webserver that can handle image style generation.

dsnopek’s picture

To untangle this issue a little bit, here's some background:

  • The currently 7.x-1.x branch is failing on all Chrome tests
  • Updating to the experimental Travis-CI 'trusty' distro will make all the tests pass except two (those are mentioned in #2). I've just attached the patch to do this.
  • The two tests that are failing, are failing because they can't click on the image in the WYSIWYG editor in such a way that clicking the "Media" button in the WYSIWYG will edit that image
  • The patch in #5 takes some screenshots to show that the image isn't rendering, so the theory (furthered in #9) is that this is the reason we can't click the image in a way that works

However, using the patch from #2428097-5: Image styles won't generate on Travis-CI (breaking tests that need valid images) fixes the image rendering, but the tests don't pass. See this Travis build:

https://travis-ci.org/dsnopek/panopoly/builds/227647641

So... I'm not exactly sure where that leaves us?

dsnopek’s picture

Heh, ok. So, putting my "untangling" above led me to try something that maybe I should have tried first: pinning to Chrome 57, which was used in the last successful builds (Chrome 58 is the latest). Adding that to the branch I've been hacking on fixes the broken tests! :-) I have yet to test it on it's own, so maybe it fixes the problem on it's own without the trusty update and all the rest. I'll update when that build is done!

However... We don't necessarily want to stay on Chrome 57 forever. The click behavior we've been depending on is totally non-standard and undefined, so the fact that it broke isn't super surprising. I'm not even sure the Chrome team would consider it a bug!

So, pinning to 57 may only be delaying the issue for a little bit longer...

EDIT: here's the Travis build for just the pinning change; https://travis-ci.org/dsnopek/panopoly/builds/227674494

dsnopek’s picture

Assuming the above build passes, I think I'm going to make that patch be the main patch here and then make new issues for the rest of the changes we've hacked on here:

After getting this issue closed, we should be able to get those closed in relatively short order!

dsnopek’s picture

Bah! Unfortunately, pinning Chrome on it's own didn't work. It appears to need some of the other changes too. I'll try combining some of the patches and see what combination is necessary.

dsnopek’s picture

FileSize
1.67 KB

Hrm. Something is wrong with my trusty patch - this one applies better for me.

dsnopek’s picture

Status: Needs work » Needs review
FileSize
2.05 KB

Ok! Here's a patch that does the 'trusty' update and Chrome 57 pin, which gets the tests passing for me. Here's a combined patch.

I'll commit that and make issues for the remaining stuff!

  • dsnopek committed e986607 on 7.x-1.x
    Issue #2869464 by dsnopek, cboyden: All Chrome tests failing with "Could...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.