Running local tests on an unchanged copy of the 7.x-2.x branch, I get:
"411 passes, 230 fails, 35 exceptions, and 146 debug messages".

I then uploaded a patch that does not touch any of the code (just the README.txt), and the patch failed testing, indicating that testing is broken.

CommentFileSizeAuthor
#14 juicebox-trivial_patch_to_readme-2811273-14.patch333 bytesgisle
#10 2811273-10.patch7.87 KBrjacobs
#7 2811273-7.patch7.84 KBrjacobs
#2 juicebox-trivial_patch_to_readme-2811273-2.patch636 bytesgisle

Comments

gisle created an issue. See original summary.

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented
Status: Active » Needs review
StatusFileSize
new juicebox-trivial_patch_to_readme-2811273-2.patch636 bytes

Trivial patch uploaded - just to see how it is treated by automated tests.

Status: Needs review » Needs work

The last submitted patch, 2: juicebox-trivial_patch_to_readme-2811273-2.patch, failed testing.

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented
Title: Local tests of 7.x-2.1 produces 230 fails, 35 exceptions » Automated tests of the 7.x-2.x branch looks briken
Version: 7.x-2.1 » 7.x-2.x-dev
Category: Support request » Task
Issue summary: View changes
Status: Needs work » Active

OK, the patch failed testing.

So It looks like testing is broken.

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented
Title: Automated tests of the 7.x-2.x branch looks briken » Automated tests of the 7.x-2.x branch looks broken

Fixed typo in summary.

rjacobs’s picture

rjacobs commented

Running local tests on an unchanged copy of the 7.x-2.1 release, I get:
"411 passes, 230 fails, 35 exceptions, and 146 debug messages".

Is this normal?

No, test failures are never normal :) ... and there are no outstanding known issues that are blocking tests.

The automated testing for 7.x-2.x on this module is only set to run on patch submissions and commits (there is no daily check run currently), and there have not been many of those in recent months on that specific branch. So core changes that might have introduced test case problems have simply gone unnoticed. Anyway, it certainly warrants some investigation.

rjacobs’s picture

rjacobs commented
Title: Automated tests of the 7.x-2.x branch looks broken » 7.x-2.x test need update for compatibility with new permissions in core 7.50
Status: Active » Needs review
StatusFileSize
new 2811273-7.patch7.84 KB

The test errors appear to be related to the new "administer fields" permission that was added in core 7.50:

https://www.drupal.org/node/2483307

The setup routines for most of the tests depend on the ability to set and alter field settings which was not being allowed because that new permission was not set during the test setup.

The following patch should address it.

Status: Needs review » Needs work

The last submitted patch, 7: 2811273-7.patch, failed testing.

rjacobs’s picture

rjacobs commented

Well... it addressed most of the errors at least. The lingering one seems to have something to do with the file entity soft dependency, so I'll need to see what's going on there (I can't replicate that one locally just yet).

rjacobs’s picture

rjacobs commented
Status: Needs work » Needs review
StatusFileSize
new 2811273-10.patch7.87 KB

Looks like file_entity also had some permission changes that needed to be accounted for in the test case that leverages it. Here's another go at it.

  • rjacobs committed ec1cb9c on 7.x-2.x 
    Issue #2811273 by rjacobs, gisle: Updates for compatibility with new...
rjacobs’s picture

rjacobs commented
Status: Needs review » Fixed

Looks good. Test fixes committed.

@gisle, can you submit your documentation updates in a separate new issue? I just want to keep this patch dedicated to the test fixes.

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented

@gisle, can you submit your documentation updates in a separate new issue?

I have no (real) documentation updates. Please ignore the patch in #2 - it was only created by me to see whether the automated testing framework was able to tell the difference between errors triggered by the patch being submitted for testing, and errors triggered by code already in the repo (it is not – and I gripe about it here: #2811297: Automated testing should not always blame the last submitted patch.)

I am preparing a real patch for #2725279: Add support for using the Scald Atom Reference field as image source (which I will upload soon).

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented
Status: Fixed » Needs review
StatusFileSize
new juicebox-trivial_patch_to_readme-2811273-14.patch333 bytes

Repeating the exercise with just a patch to README.txt.

When running local tests against the last commit (ec1cb9cb9a67ccc87d7e1df3fc661ca371d1089d) , I still get a lot of failures from 4 of the 6 of the test suites:

Juicebox gallery configuration tests 129 passes,  0 fails,  0 exceptions
Juicebox global configuration tests   72 passes,  9 fails,  0 exceptions
Juicebox field formatter tests       129 passes,  0 fails,  0 exceptions
Juicebox file handling tests          86 passes,  1 fail,   0 exceptions
Juicebox sub-request tests             7 passes, 14 fails,  6 exceptions
Juicebox Views integration tests      33 passes, 45 fails, 14 exceptions

456 passes, 69 fails, 20 exceptions, and 175 debug messages

This is a clean install on Drupal 7.50 and all projects required by Juicebox is updated to their most recent stable release:

Image (core) 7.50
File (core) 7.50
Field (core) 7.50
Field SQL storage (core) 7.50
Libraries 7.x-2.3
System (core) 7.50

Juicebox-Lite 1.5.0

The module is working as expected (I use the Field Formatter). There are no error messages.

gisle’s picture

gisle
he/him
Primary language Norwegian Bokmål
Location Norway
commented
Status: Needs review » Fixed

Local tests fail (4 out of 6 test suites). When I trigger the automated test at Drupal.org (by submitting a bogus patch), it passes.

Setting back to "fixed", as this seems to be a local problem at my site - but any clues you may provide about what the problem is and how to fix it will be most welcome.

I thought that automated tests run in a "clean environment" so that they should not be affected by local configurations ...

rjacobs’s picture

rjacobs commented

It looks like the points from #13-#15 are being addressed in #2811579: Automated test produces false positives if library is Juicebox Lite. Thanks for highlighting that.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.