Apparently the Mime Mail tests have not been working on the testbot for years - the testbot logs go back to 17 Dec 2015 and they show the tests have always been failing since then. And always failing for the same reason!

But in particular, the 7.x-1.x branch tests haven't been run since 14 May 2017, so we can't see what the failures were (old testbot runs are archived and not viewable), and we can't make the tests work on the testbot unless we have current testbot results.

My local testing suggests that these 7.x-1.x should be working. At least, they work for me locally. I suspect there was a problem with the testbot last year which caused the tests to fail, and they haven't been run since.

Anyway, please trigger the 7.x-1.x tests to run again. Also, please set up a new 7.x-1.x test that runs daily, so we get immediate notice if something breaks.

I suggest PHP 7.2 & MySQL 5.5 for the daily test, just like the 8.x-1.x branch.

Drupal core recently became compatible with PHP 7.2, so this will be a good test to see if MimeMail is compatible or if there are some things that need to be fixed for 7.2 compatibility.

CommentFileSizeAuthor
#10 composer.patch460 bytestr
#8 composer.patch426 bytestr
#7 test-dependency.patch403 bytestr
#6 test-test-3.patch2.26 KBtr
#5 test-test-2.patch2.25 KBtr
#3 test-test.patch480 bytestr

Comments

TR created an issue. See original summary.

sgabe’s picture

I can confirm that testing is successful locally, however, they are failing remotely. I restarted the old test and added the new one as requested. Need to find out why the tests are failing...

tr’s picture

StatusFileSize
new480 bytes

Experimenting to see what's wrong ... it looks like parent::setUp() is failing for the MimeMailRulesTestCase.

tr’s picture

Tests aren't starting because testbots are currently down - see https://twitter.com/drupal_infra/status/1019616733957644288

tr’s picture

StatusFileSize
new2.25 KB
tr’s picture

StatusFileSize
new2.26 KB

Sigh.

tr’s picture

Status: Active » Needs review
StatusFileSize
new403 bytes

OK, it's been so long since I've seen this problem that I almost forgot all about it. I guess the Mime Mail tests haven't run green on the drupal.org testbot for years ...

Anyway, I believe the problem is that the testbot only knows about dependent modules declared in the .info file, and when it builds the test environment it only loads the modules that are declared in the .info. So when the Mime Mail test tries to enable rules, entity, and entity_token in the setUp(), the testbot doesn't have these modules available and setUp() fails. (The tests work locally for me because I have those modules on my Drupal site.)

The solution is to name these modules as test_dependencies[] in the .info file. The testbot will build the test environment to include any modules that are named in test_dependencies[].

The problem is, these test_dependencies[] needs to be committed to the repository BEFORE the tests will work - the test_dependencies[] will have no effect if they are added as part of a patch.

So here's a patch that just adds the test_dependencies[]. The tests won't run green until AFTER this patch is committed.

An explanation of this can be found at #2181107: test fails when adding another contrib module to enable in setUp() ...

tr’s picture

StatusFileSize
new426 bytes

I think we can also trigger the testbot to load the test dependencies this way:

Status: Needs review » Needs work

The last submitted patch, 8: composer.patch, failed testing. View results

tr’s picture

StatusFileSize
new460 bytes

composer.json needs regular dependencies as well as test dependencies. Trying again...

tr’s picture

Priority: Normal » Major
Status: Needs work » Needs review

OK, to summarize:

The D7 tests are currently failing because the testbot environment doesn't include Rules or Entity. This is because the testsbot parses the mimemail.info file to figure out what modules are needed in the testbot environment. Mime Mail does NOT depend on Rules or Entity, but it can use them if they are available. And in particular, Mime Mail tests DO use these modules. But since they're not required, they're not declared as dependencies in mimemail.info so the testbot doesn't load them and the tests that need these modules fail.

So the solution is to tell the testbot that these modules need to be downloaded into the testbot environment.

There are two ways to do this:

  1. The patch in #7 adds these modules as test_dependencies[] so that the testbot knows they are needed for tests (and only for tests - they won't be enabled when Mime Mail is installed normally). This patch needs to be committed FIRST, and then any test runs after that should be green.
  2. OR, the patch in #10 adds a composer.json to the Mime Mail project. The testbot will use this composer.json file to resolve dependencies, thus it will know it needs Rules, and Entity

I prefer method 1), as it is entirely Drupal without the need for composer.

Regardless, one of these patches should be committed so we can use the testbot again for Mime Mail.

tr’s picture

Issue summary: View changes

  • sgabe committed 08e6eaa on 7.x-1.x authored by TR
    Issue #2986204 by TR: Missing test dependencies
    
sgabe’s picture

Title: Restart 7.x-1.x tests! » Missing test dependencies
Status: Needs review » Fixed

Changed title to better reflect the issue. The patch attached in #7 has been committed, thanks!

tr’s picture

And the D7 tests are now running and passing!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.