Use drupalci.yml for composer dependency min/max testing - DO NOT COMMIT

I discovered that once we run composer update --prefer-lowest, we then have to run composer run-script drupal-phpunit-upgrade. And since that script only updates phpunit/phpunit, it disallows re-updating to PHPUnit 6+. Therefore we have to explicitly update phpspec/prophecy as well.

I suppose it belongs in core/ doesn't it?

Misnamed composer.max. I guess we need to work on this: #2956525: Warn users that their build tasks have clashing names

Click through to the test results and you see a ton of deprecation errors: https://www.drupal.org/pift-ci-job/977869

You can click through to the artifacts to see more details: https://dispatcher.drupalci.org/job/drupal_patches/59892/artifact/jenkin...

For instance, in the run_tests.min folder, there is a run_testsoutput.txt file which shows you the fails resulting from installing the dependencies with --prefer-minimum.

There's also a phpunit-xml folder which contains all the detailed junit reports for each test run.

Added deprecation notices to the deprecation listener whitelist. They're the same as existing ones, except replacing 'Symfony' with 'version', and no period at the end. ::shrug::

This should give us an indication of what's actually 'broken,' as opposed to what's deprecated.

Reverted the deprecation whitelist and just set it to suppress deprecations.

What can we learn from this?

1) Symfony changed their deprecation notices from saying 'since version x' to 'since Symfony x'. This change occurs in Symfony 3.4.3. https://github.com/symfony/symfony/commit/74383b6e5982fce66119b8e8a0e0a4...

This breaks a few of our tests, which have @expectedDeprecations against the newer messages.

We should change our minimum from "symfony/http-foundation": "~3.4.0", to ~3.4.3. This is probably a de-facto change to all the Symfony components, as well.

2) When we perform the upgrade we sometimes break ComposerIntegrationTest:

1) Drupal\Tests\ComposerIntegrationTest::testMinPHPVersion
doctrine/cache has a PHP dependency requirement of "~7.1"

That test exists to make sure there's no dependency in composer.lock that requires a PHP version higher than 5.5. When this test says composer update as part of this test, it re-writes the lock file for the current version of PHP.

So either we have to remove this test or we have to figure out a way to signal it to skip. We could set an environmental variable like DRUPAL_MIN_MAX_TEST=1 and then have the test skip on this value.

3) It might be that kernel tests break --suppress-deprecations.

4) Add your discovery... More to come no doubt. :-)

Followed the advice from #10.1. That Symfony change in https://github.com/symfony/symfony/commit/74383b6e5982fce66119b8e8a0e0a4... touches all our declared dependencies except one, so I just changed them all to ~3.4.3.

Also marked ComposerIntegrationTest::testMinPHPVersion() as skipped for the purposes of learning more stuff. Note that it also has this todo:

   * @todo This can be removed when DrupalCI supports dependency regression
   *   testing in https://www.drupal.org/node/2874198

That's basically the testbot issue that corresponds to this issue.

Making the dependencies of #2976417: Update symfony (security release) and #2965887: Require egulias/email-validator >= 1.2.14 explicit here, and also a few others that are now in line with composer.lock which seems to make things happy locally. Let's see what the testbot does.

Based on #14, it turns out that if we want guzzlehttp/guzzle to work under PHP 7.2, we have to specify ^6.3. That's fine because it's what's in the current lock file, and only requires PHP 5.5.

So now that we've got passing tests for unit and kernel tests, let's add back functional and simpletest. This will make the test runs much longer but hopefully will give us some more nice fails to look at. :-)

I can't repro the errors in #15 because some errors cause redirects to be sent to the wrong port within functional tests. I'm using MAMP which uses port 8888, and when guzzle gets a redirect from mink, it's to port 80.

However, I found a ton of tests which use t(), and many of the test fail errors you'll see in that report look like this:

1) Drupal\Tests\action\Functional\ConfigurationTest::testActionConfiguration
Object of class Drupal\Core\StringTranslation\TranslatableMarkup could not be converted to int

I'm unable to determine which subsystem allows us to say things like $this->assertRaw(t('something')); based on versions in composer.lock, but not the min/max.

So here we sit. :-)

Comment made in error. Disregard. :-)

Thanks, @vaplas.

Pinned behat/mink to "behat/mink": "1.7.x-dev#9ea1cebe3dc529ba3861d87c818f045362c40484" to lock in #2808085: Pipe char in locators break Mink and Symfony element search That's the commit that's in composer.lock, and you can see what it does here: https://github.com/minkphp/Mink/pull/720/commits

This has (or should have) the side-effect of mitigating #2948700: Casting $text value to string in responseContains/responseNotContains methods. If we end up committing the constraints here, we should be sure and modify core/composer.json in that other issue to move away from requiring specific commits. Todo here would be to figure out which mink releases perform the casting we need and use that as an upper version constraint.

Reroll.

+++ b/core/composer.json
@@ -18,26 +18,26 @@
-        "symfony/http-foundation": "~3.4.14",
...
+        "symfony/http-foundation": "~3.4.11",

Missed that one...

Fixed #26 which it turns out is left over from https://www.drupal.org/SA-CORE-2018-005, so yah. We want that.

If you check the CI console log for the last couple of patches, you see that the max update does this:

18:44:05   - Updating doctrine/common (v2.5.0 => v2.10.0): Downloading (100%)

That is, updates to doctrine/common 2.10. That accounts for about 90% of the fails, because: #2986725: doctrine common 2.9 has moved reflection

I re-opened that issue and added a patch.

Rerolled after #2986725: doctrine common 2.9 has moved reflection

behat/mink has had a release since #20, so it's at 1.7.1.

It turns out that's not the only culprit for #20, because we also have to give a boost to symfony/browser-kit to ^3. I included it here because I can see it in my stack trace, but we get the same effect if we boost symfony/dom-crawler the same way. Maintainers might have an opinion about that one way or another.

This patch reverts to 8.7.x's lock file, so the interdiff is weird.

Here's an informative issue: #2551309: Increase opcache.max_accelerated_files and an old issue from drupalci_environments: #2446719: Need test runners without bytecode caching

And a patch with an opcode cache clearing script. It might not be something we should actually add to core, though maybe we should.

Adding back functional tests to the max portion of the build as well.

PEBKAC.

Opcache clearing ripped off from rebuild.php this time.

More PEBKAC.

In terms of prioritizing, there's also this issue on the testbot: #2951375: Allow for build targets

The idea there was that you could set up your drupalci.yml so it would do, for instance, a 'release' target versus an 'issue' target and so forth, with different DrupalCI builds.

We might care about min/max for some circumstances but not for others.

Otherwise, if we add the build steps from this issue to drupalci.yml, then it will happen for all builds in all circumstances. That's only an undesirable outcome from the perspective of run time.

The work in this issue is based on some Travis-CI based tests, mainly https://github.com/paul-m/drupal_dependency_bounds_test but also some
dependency tweaks gleaned from https://github.com/paul-m/drupal_component_tester

This will probably be one of the use cases for #3031379: Add a new test type to do real update testing

OK, so the blocker here is that #58 needs a way to clear the opcache so that it uses the upgraded/downgraded PHP.

#3031379: Add a new test type to do real update testing makes a new PHP-based HTTP server for every test class specifically because this blocker exists in this issue.

If there is a testbot-level change someone wants to suggest, we can do that, too.