Problem/Motivation

Drupal 9 will release with Symfony 4.4, see #3009219: Update Symfony to 4.4 in Drupal 9.0. In order to be able to do that, we should make Drupal 8 compatible with Symfony 4.4 first as much as possible. We cannot allow Symfony 4.4 to be installed from a simple composer update command, as current ways to install Drupal with composer would not protect production sites from switching to Symfony 4.4 which is unintended. That also means that until there is a Drupal 9 branch, there will be no easy way to test contrib with Symfony 4.4.

Proposed resolution

  1. Resolve and commit the remaining issues needed to make Symfony 4 pass tests.
  2. Provide a script or instructions on how to adjust the various composer.json files for those adventurous souls who wish to try it with contrib, but it should not happen by default on a simple update.

Focus on Symfony 4.4 here. All Symfony 5 releated work is in #3055180: [META] Symfony 5 compatibility.

Remaining tasks

The following outstanding issues must be resolved to make the tests pass:

  • A test fail related to apcu caching on the testbot whose solution is still being determined.

User interface changes

None.

API changes

Symfony 4.4 API will be supported.

Data model changes

None

Release notes snippet

Drupal now supports Symfony 4.4.

CommentFileSizeAuthor
#270 interdiff.2976394.268-270.txt18.74 KBmikelutz
#270 2976394-270.drupal.Plus3074585.patch207.98 KBmikelutz
#270 2976394-270.drupal.RevertAPCU change.patch192.4 KBmikelutz
#268 interdiff.2976394.260-268.txt252.16 KBmikelutz
#268 2976394-268.drupal.Allow-Symfony-44-to-be-installed-in-Drupal-8.patch192.95 KBmikelutz
#260 2976394-260.drupal.Allow-Symfony-44-to-be-installed-in-Drupal-8.patch177.86 KBmikelutz
#252 2976394-252.patch177.55 KBalexpott
#252 250-252-interdiff.txt1.15 KBalexpott
#250 2976394-250.patch177 KBalexpott
#250 249-250-interdiff.txt1.85 KBalexpott
#249 2976394-249.patch176.56 KBalexpott
#247 2976394-247.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch177.31 KBmikelutz
#245 interdiff.2976394.244-245.txt2.05 KBmikelutz
#245 2976394-245.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch178.3 KBmikelutz
#244 2976394-fc-disabling.txt643 bytesmikelutz
#244 2976394-244.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch179.29 KBmikelutz
#244 2976394-244.drupal.fc_disabled.patch179.36 KBmikelutz
#238 interdiff.2976394.236-238.txt2.37 KBmikelutz
#238 2976394-238.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch178.34 KBmikelutz
#236 interdiff.2976394.235-236.txt2.17 KBmikelutz
#236 2976394-236.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch176.89 KBmikelutz
#235 interdiff.2976394.234-235.txt1.42 KBmikelutz
#235 2976394-235.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch174.96 KBmikelutz
#234 interdiff.2976394.231-234.txt1.26 KBmikelutz
#234 2976394-234.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch174.81 KBmikelutz
#231 2976394-231.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch174.8 KBmikelutz
#231 interdiff.2976394.227-231.txt2.78 KBmikelutz
#227 interdiff.2976394.225-227.txt4.21 KBmikelutz
#227 2976394-227.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch174.1 KBmikelutz
#225 interdiff.2976394.222-225.txt166.42 KBmikelutz
#225 2976394-225.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch173.61 KBmikelutz
#223 interdiff.txt2.22 KBgábor hojtsy
#223 2976394-223.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch11.88 KBgábor hojtsy
#221 2976394-221.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch11.88 KBmikelutz
#218 2976394-218.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch11.79 KBmikelutz
#213 2976394-213.drupal.Investigate-problems-with-Symfony-4-now-.patch11.33 KBmikelutz
#213 interdiff.2976394.210-213.txt4.57 KBmikelutz
#210 interdiff.2976394.207-210.txt1.89 KBmikelutz
#210 2976394-210.drupal.Investigate-problems-with-Symfony-4-now-.patch11.33 KBmikelutz
#207 2976394-207.drupal.Investigate-problems-with-Symfony-4-now-.patch11.32 KBmikelutz
#207 interdiff.2976394.206-207.txt7.24 KBmikelutz
#206 interdiff.2976394.199-206.txt115.22 KBmikelutz
#206 2976394-206.drupal.Investigate-problems-with-Symfony-4-now-.patch14.92 KBmikelutz
#199 interdiff.2976394.195-199.txt17.64 KBmikelutz
#199 2976394-199.drupal.Investigate-problems-with-Symfony-4-now-.patch198.12 KBmikelutz
#195 interdiff.2976394.192-195.txt4.72 KBmikelutz
#195 2976394-195.drupal.Investigate-problems-with-Symfony-4-now-.patch180.48 KBmikelutz
#192 interdiff.2976394.189-192.txt8.5 KBmikelutz
#192 2976394-192.drupal.Investigate-problems-with-Symfony-4-now-.patch177.19 KBmikelutz
#189 2976394-189.drupal.Investigate-problems-with-Symfony-4-now-.patch168.69 KBmikelutz
#189 interdiff.2976394.186-189.txt2.54 KBmikelutz
#186 2976394-186.drupal.Investigate-problems-with-Symfony-4-now-.patch167.5 KBmikelutz
#181 2976394-181.drupal.COMBINED_PATCH_FOR_TESTING.patch17.27 KBmikelutz
#181 2976394-181.drupal.WHEN_THIS_ONE_IS_GREEN.patch9.81 KBmikelutz
#181 2976394-181.drupal.COMMIT_THIS_ONE.patch9.25 KBmikelutz
#180 2976394.179-180.interdiff.txt511 bytesmikelutz
#180 2976394-180.drupal.COMMIT_THIS_ONE.patch9.37 KBmikelutz
#179 2976394-179.drupal.WHEN_THIS_ONE_IS_GREEN.patch159.28 KBmikelutz
#179 2976394-179.drupal.COMMIT_THIS_ONE.patch9.68 KBmikelutz
#179 2976394-179.drupal.COMBINED_PATCH_WITH_OUTSTANDING_ISSUES_INCLUDED.patch265.37 KBmikelutz
#174 interdiff.2976394.172-174.txt5.09 KBmikelutz
#174 2976394-174.drupal.Investigate-problems-with-Symfony-4-now-.patch275.11 KBmikelutz
#172 interdiff.2976394.170-172.txt9.61 KBmikelutz
#172 2976394-172.drupal.Investigate-problems-with-Symfony-4-now-.patch270.02 KBmikelutz
#170 interdiff.2976394.168-170.txt7.54 KBmikelutz
#170 2976394-170.drupal.Investigate-problems-with-Symfony-4-now-.patch260.41 KBmikelutz
#168 interdiff.2976394.166-168.txt3.78 KBmikelutz
#168 2976394-168.drupal.Investigate-problems-with-Symfony-4-now-.patch252.87 KBmikelutz
#166 interdiff.2976394.164-166.txt1021 bytesmikelutz
#166 2976394-166.drupal.Investigate-problems-with-Symfony-4-now-.patch249.09 KBmikelutz
#164 interdiff.2976394.163-164.txt85.42 KBmikelutz
#164 2976394-164.drupal.Investigate-problems-with-Symfony-4-now-.patch248.09 KBmikelutz
#163 2976394-163.drupal.Investigate-problems-with-Symfony-4-now-.patch162.67 KBmikelutz
#162 2976394-162.drupal.Investigate-problems-with-Symfony-4-now-.patch162.89 KBmikelutz
#158 2976394-158.drupal.Investigate-problems-with-Symfony-4-now-.patch162.86 KBmikelutz
#158 interdiff.2976394.156-158.txt2.8 KBmikelutz
#156 2976394-156.drupal.Investigate-problems-with-Symfony-4-now-.patch163.71 KBmikelutz
#156 interdiff.2976394.154-156.txt560 bytesmikelutz
#155 interdiff.2976394.148-155.txt24.46 KBmikelutz
#155 2976394-155.drupal.Investigate-problems-with-Symfony-4-now-.patch163.17 KBmikelutz
#153 interdiff.2976394.148-153.txt39.71 KBmikelutz
#153 2976394-153.drupal.Investigate-problems-with-Symfony-4-now-.patch171.03 KBmikelutz
#149 2976394-147-with-3029540-2.patch160.42 KBgábor hojtsy
#147 2976394-147.patch153.76 KBgábor hojtsy
#144 2976394-144.patch172.06 KBgábor hojtsy
#141 2976394-141.patch160.42 KBgábor hojtsy
#134 2976394-134.patch161.43 KBgábor hojtsy
#129 2976394-129.patch162.9 KBcatch
#127 2976394-127.drupal.combo-3029197-3029540-2937542-3029500.DO_NOT_COMMIT.patch174.85 KBgábor hojtsy
#126 2976394-126.drupal.combo-3029197-3029540-2937542.DO_NOT_COMMIT.patch173.38 KBmikelutz
#121 interdiff.2976394.117-121.txt9.14 KBmikelutz
#121 2976394-121.drupal.Investigate-problems-DO_NOT_TEST.patch162.98 KBmikelutz
#118 2976394-118.patch153.84 KBjibran
#118 interdiff-5231c9.txt6.09 KBjibran
#114 2976394-combined-114.patch153.38 KBjibran
#114 2976394-114.patch152.18 KBjibran
#108 2976394-108.patch155.78 KBcatch
#107 2976394-105.patch152.18 KBcatch
#105 2976394-105.patch152.71 KBcatch
#101 2976394-101.patch145.75 KBcatch
#99 2976394-99.patch146.86 KBcatch
#98 2976394-99.patch163.35 KBcatch
#96 2976394-96.patch157.47 KBcatch
#94 2976394-94.patch156.45 KBcatch
#90 2976394-90.patch115.94 KBcatch
#87 2976394-87.patch115.2 KBcatch
#86 2976394-86.patch110.77 KBcatch
#84 2976394-83.patch107.53 KBcatch
#65 2976394-64.patch119.9 KBcatch
#65 2976394-64-interdiff.txt547 bytescatch
#61 2976394-62-interdiff.txt3.89 KBcatch
#61 2976394-62.patch119.9 KBcatch
#61 2976394-61.patch118.94 KBcatch
#61 2976394-61-interdiff.txt2.93 KBcatch
#57 2976394-57.patch299.26 KBwim leers
#57 interdiff.txt968 byteswim leers
#55 2976394-55.patch298.51 KBmartin107
#52 2976394-52.patch295.97 KBmartin107
#49 interdiff-2976394-46-49.txt2.23 KBmartin107
#49 2976394-49.patch319.56 KBmartin107
#46 2976394-46.patch317.32 KBmartin107
#39 interdiff-2976394-37-39.txt4.06 KBmartin107
#39 2976394-39.patch336.96 KBmartin107
#37 interdiff-2976394-32-36.txt1.66 KBmartin107
#37 2976394-36.patch332.9 KBmartin107
#32 interdiff-2976394-28-32.txt4.11 KBmartin107
#32 2976394-32.patch331.24 KBmartin107
#28 interdiff-2976394-18-28.txt4.47 KBmartin107
#28 2976394-28.patch327.13 KBmartin107
#24 interdiff-2976394-18-24.txt1.77 KBmartin107
#24 2976394-24.patch324.43 KBmartin107
#18 2976394-18.patch322.66 KBmartin107
#18 interdiff-2976394-16-17.txt136.1 KBmartin107
#17 2976394-17.patch186.56 KBmartin107
#14 2976394-14.patch155.98 KBmartin107
#9 interdiff-2976394-7-9.patch117.34 KBmartin107
#9 2976394-9.patch118.24 KBmartin107
#7 interdiff-2976394-5-7.txt174.81 KBmartin107
#7 2976394-7.patch177.18 KBmartin107
#5 interdiff-2976394-3-5.txt1.47 KBmartin107
#5 2976394-5.patch2.37 KBmartin107
#3 CHAINSAW-2976394-2-DO-NOT_COMMIT.patch1.35 KBmartin107

Comments

martin107 created an issue. See original summary.

martin107’s picture

Issue tags: -Identify Symfony4 issues NOW
martin107’s picture

Assigned: martin107 » Unassigned
Status: Active » Needs review
StatusFileSize
new1.35 KB

The is far from the last word in Drupal8's construction of a response to Symphony4

It really is a mad slash with a chain saw ..

but hey "drush site install" works! which was unexpected.

here is

composer install

Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 85 installs, 0 updates, 0 removals
  - Installing symfony/css-selector (v3.4.11): Loading from cache
  - Installing behat/mink (dev-master 04ab7af): Loading from cache
  - Installing instaclick/php-webdriver (1.4.5): Loading from cache
  - Installing behat/mink-selenium2-driver (dev-master 93474c6): Loading from cache
  - Installing symfony/polyfill-ctype (v1.8.0): Loading from cache
  - Installing symfony/yaml (v4.1.0): Loading from cache
  - Installing squizlabs/php_codesniffer (2.9.1): Loading from cache
  - Installing drupal/coder (8.2.12): Cloning 984c54a7b1 from cache
  - Installing symfony/class-loader (v3.4.11): Loading from cache
  - Installing symfony/polyfill-mbstring (v1.8.0): Loading from cache
  - Installing symfony/console (v4.1.0): Loading from cache
  - Installing psr/container (1.0.0): Loading from cache
  - Installing symfony/dependency-injection (v4.1.0): Loading from cache
  - Installing symfony/event-dispatcher (v4.1.0): Loading from cache
  - Installing symfony/serializer (v4.1.0): Loading from cache
  - Installing symfony/translation (v4.1.0): Loading from cache
  - Installing symfony/validator (v4.1.0): Loading from cache
  - Installing symfony/process (v4.1.0): Loading from cache
  - Installing symfony/polyfill-iconv (v1.8.0): Loading from cache
  - Installing doctrine/lexer (v1.0.1): Loading from cache
  - Installing doctrine/inflector (v1.3.0): Loading from cache
  - Installing doctrine/collections (v1.5.0): Loading from cache
  - Installing doctrine/cache (v1.7.1): Loading from cache
  - Installing doctrine/annotations (v1.6.0): Loading from cache
  - Installing doctrine/common (v2.8.1): Loading from cache
  - Installing psr/log (1.0.2): Loading from cache
  - Installing symfony/http-foundation (v4.1.0): Loading from cache
  - Installing paragonie/random_compat (v2.0.12): Loading from cache
  - Installing symfony/debug (v3.4.11): Loading from cache
  - Installing symfony/http-kernel (v3.4.11): Loading from cache
  - Installing symfony/routing (v3.4.11): Loading from cache
  - Installing symfony-cmf/routing (1.4.1): Loading from cache
  - Installing easyrdf/easyrdf (0.9.1): Loading from cache
  - Installing zendframework/zend-stdlib (3.2.0): Loading from cache
  - Installing zendframework/zend-escaper (2.6.0): Loading from cache
  - Installing zendframework/zend-feed (2.10.0): Loading from cache
  - Installing stack/builder (v1.0.5): Loading from cache
  - Installing egulias/email-validator (1.2.14): Loading from cache
  - Installing masterminds/html5 (2.3.0): Loading from cache
  - Installing psr/http-message (1.0.1): Loading from cache
  - Installing symfony/psr-http-message-bridge (v1.0.2): Loading from cache
  - Installing zendframework/zend-diactoros (1.7.2): Loading from cache
  - Installing composer/semver (1.4.2): Loading from cache
  - Installing asm89/stack-cors (1.2.0): Loading from cache
  - Installing guzzlehttp/psr7 (1.4.2): Loading from cache
  - Installing guzzlehttp/promises (v1.3.1): Loading from cache
  - Installing guzzlehttp/guzzle (6.3.3): Loading from cache
  - Installing symfony/dom-crawler (v4.1.0): Loading from cache
  - Installing symfony/browser-kit (v4.1.0): Loading from cache
  - Installing fabpot/goutte (v3.2.2): Loading from cache
  - Installing behat/mink-browserkit-driver (1.3.3): Loading from cache
  - Installing behat/mink-goutte-driver (v1.2.1): Loading from cache
  - Installing jcalderonzumba/gastonjs (v1.2.0): Loading from cache
  - Installing twig/twig (v1.35.3): Loading from cache
  - Installing jcalderonzumba/mink-phantomjs-driver (v0.3.3): Loading from cache
  - Installing mikey179/vfsstream (v1.6.5): Loading from cache
  - Installing sebastian/version (2.0.1): Loading from cache
  - Installing sebastian/resource-operations (1.0.0): Loading from cache
  - Installing sebastian/object-reflector (1.1.1): Loading from cache
  - Installing sebastian/recursion-context (3.0.0): Loading from cache
  - Installing sebastian/object-enumerator (3.0.3): Loading from cache
  - Installing sebastian/global-state (2.0.0): Loading from cache
  - Installing sebastian/exporter (3.1.0): Loading from cache
  - Installing sebastian/environment (3.1.0): Loading from cache
  - Installing sebastian/diff (2.0.1): Loading from cache
  - Installing sebastian/comparator (2.1.3): Loading from cache
  - Installing doctrine/instantiator (1.1.0): Loading from cache
  - Installing phpunit/php-text-template (1.2.1): Loading from cache
  - Installing phpunit/phpunit-mock-objects (5.0.7): Loading from cache
  - Installing phpunit/php-timer (1.0.9): Loading from cache
  - Installing phpunit/php-file-iterator (1.4.5): Loading from cache
  - Installing theseer/tokenizer (1.1.0): Loading from cache
  - Installing sebastian/code-unit-reverse-lookup (1.0.1): Loading from cache
  - Installing phpunit/php-token-stream (2.0.2): Loading from cache
  - Installing phpunit/php-code-coverage (5.3.2): Loading from cache
  - Installing webmozart/assert (1.3.0): Loading from cache
  - Installing phpdocumentor/reflection-common (1.0.1): Loading from cache
  - Installing phpdocumentor/type-resolver (0.4.0): Loading from cache
  - Installing phpdocumentor/reflection-docblock (4.3.0): Loading from cache
  - Installing phpspec/prophecy (1.7.6): Loading from cache
  - Installing phar-io/version (1.0.1): Loading from cache
  - Installing phar-io/manifest (1.0.1): Loading from cache
  - Installing myclabs/deep-copy (1.8.0): Loading from cache
  - Installing phpunit/phpunit (6.5.8): Loading from cache
  - Installing symfony/phpunit-bridge (v3.4.11): Loading from cache
martin107’s picture

That was unexpected, so looking for problem I did some digging

Taking one symfony component as a signal that things have gone wrong

locally when I reported the updates above I see

- Installing symfony/event-dispatcher (v4.1.0): Loading from cache

but in the build artifact

https://dispatcher.drupalci.org/job/drupal_patches/59880/consoleText

I see

- Installing symfony/event-dispatcher (v3.4.9): Loading from cache

Which means that I did not trigger the change I wanted ... oh rats

Just compiling changes here for the next iteration

php -- we should force the version to be 7.x as that is a given for Drupal9
twig -- needs to be adjusted to be updated .. as it may affect the decision tree for some symfony components

Also but not sure -- does that rule apply for doctrine as well ?

back to the drawing board

martin107’s picture

Status: Needs review » Needs work
StatusFileSize
new2.37 KB
new1.47 KB

This may require a special testbot run for the changes here to be picked up ... but in this comment I am going to focus on changes to the repo only.

A) Looking at the link below I have driven the php version up to >=7.1.3

Symfony 4.0 requires PHP 7.1.3 or higher to run,

https://symfony.com/doc/current/reference/requirements.html

B) I have relaxed more symfony component to *

I am also going to hold back on changes to twig and doctrine .. because I want to identify response to small changes and to slowly build up to a more complete solution.

catch’s picture

The reason the testbot isn't picking up changes is because the patch is only to composer.json - composer.lock is also in version control, so needs to be updated too here, then it should explode as expected.

Also, thanks for picking this up!

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new177.18 KB
new174.81 KB

Thanks, that was clumsy of me.

composer update

Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 6 updates, 0 removals
- Updating symfony/debug (v3.4.11 => v4.1.0): Downloading (100%)
- Updating symfony/http-kernel (v3.4.11 => v4.1.0): Downloading (100%)
- Updating symfony/routing (v3.4.11 => v4.1.0): Downloading (100%)
- Updating symfony-cmf/routing (1.4.1 => 2.1.0-RC1): Downloading (100%)
- Updating symfony/css-selector (v3.4.11 => v4.1.0): Downloading (100%)
- Updating symfony/phpunit-bridge (v3.4.11 => v4.1.0): Downloading (100%)

symfony-cmf/routing -> 2.1.0-RC1 ... Release candidate one ... that is funny. ( well makes me smile )

Status: Needs review » Needs work

The last submitted patch, 7: 2976394-7.patch, failed testing. View results

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new118.24 KB
new117.34 KB

err this is what I mean.

Status: Needs review » Needs work

The last submitted patch, 9: interdiff-2976394-7-9.patch, failed testing. View results

martin107’s picture

The php7.2 test output is the only thing worth looking at

Here is a potted summary of errors I can see

A) Most are of the form :-

Object of class Drupal\Core\StringTranslation\TranslatableMarkup could not be converted to int

B) I can see about 30 errors which seems related to Symfony4 routing changes.

Drupal\Tests\system\Functional\Module\DependencyTest::testProjectNamespaceForDependencies
LogicException: Cannot use UTF-8 route patterns without setting the "utf8" option for route "/system-test/Ȅchȏ/meφΩ/{text}"

Here is one test in particular which fails hard.

Drupal\KernelTests\Core\Routing\RouteProviderTest with 17 fails.

martin107’s picture

Regarding #11B

As symfony jumpes from 3.2 to 4.0 auto handling of utf8 routes was deprecated ( made explicit )

We can sweep all auto handling out of D8 core code in preparation for D9 today

Here is a issue that reflects that thought.

martin107’s picture

A common pain point to many errors is AssertLegacyTrait.php

#2970831: Fix "AssertLegacyTrait::getRawContent() is scheduled for removal in Drupal 9.0.0. Use $this->getSession()->getPage()->getContent() instead."
#2970052: Fix "assertNoPattern() is deprecated and scheduled for removal in Drupal 9.0.0. Use $this->assertSession()->responseNotMatches($pattern) instead"

So my cunning plan is to rerun testbot on this patch after getting those two issues home..

I don't think future analysis is productive until then.

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new155.98 KB

So my cunning plan is to rerun testbot on this patch after getting those two issues home..

The latest patches associated with those issue are now passing... so I am applying those ontop of that from #9

Let us see what shakes.

Status: Needs review » Needs work

The last submitted patch, 14: 2976394-14.patch, failed testing. View results

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new186.56 KB

a) #2970831: Fix "AssertLegacyTrait::getRawContent() is scheduled for removal in Drupal 9.0.0. Use $this->getSession()->getPage()->getContent() instead." has landed.
b) Drupal D.7.x is now the development target.
c) Since I last posted, to my mind a lot of work has going into resolving technical debt.

so I am starting over with a new patch :-

Setting all symfony sub components to * in composer.json
"composer clear-cache" because annoyingly I have discovered this can sometimes affect the dependency calculations
"composer install" to update composer.lock

and then pushing the patch

I expected the responseNotMatches() issue to linger ( see #14) ... but the resulting catalog of errors might usefully expose something new.

I can say that Drupal install works and the standard landing page appears ..

I can login and see the admin menu bar... the status report page, but that is about the limit of my manual testing

martin107’s picture

StatusFileSize
new136.1 KB
new322.66 KB

My bad ....

now with composer update

martin107’s picture

A partial summary of what has been updated

symfony/class-loader v3.4.15
symfony/http-foundation v3.4.15
symfony/console: v4.1.4
symfony/http-kernel v4.0.14

Status: Needs review » Needs work

The last submitted patch, 18: 2976394-18.patch, failed testing. View results

martin107’s picture

I have been able to relate many, if not all, of the errors back to a known issue.

#2948700: Casting $text value to string in responseContains/responseNotContains methods

assertRaw() containing t() calls -- is what I see in many failures.

quoting from the issue

Example: all cases with AssertLegacyTrait::assertRaw(t(...)); because assertRaw() works via assertSession()->responseContains()

which will cause

Object of class Drupal\Core\StringTranslation\TranslatableMarkup could not be converted to int

Anyway I hope this is useful insight.

jibran’s picture

It would be great if we can try the combined patch from #2948700: Casting $text value to string in responseContains/responseNotContains methods and this issue.

martin107’s picture

I think my bias is clear ... I want D9 as soon as possible.

but I listen to and find rational the broader community consensus that D9 will take more time...

IF we could find a way partial way of adopting symfony/http-kernel v4.0.14 -- while holding other symfony component at 3.x ... then that would fit the aim of doing as much as possible in D8

spread the risk out until it is no risk at all.

But my voice is just one and my analysis is not too deep... and others might kick in reasonable objections.

In short I will help out on the other issue where possible.

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new324.43 KB
new1.77 KB

As per jibran suggestion:

This next patch is just a test of my thinking that most of the errors can be solved if the patch from

#2948700: Casting $text value to string in responseContains/responseNotContains methods

is applied after the patch from this issue ( see #18 )

Status: Needs review » Needs work

The last submitted patch, 24: 2976394-24.patch, failed testing. View results

martin107’s picture

This issue is looking for the unexpected...

So the good news is that the 64 fails can be mapped to this existing issue.

#2961688: Fix "Using UTF-8 route patterns without setting the "utf8" option" deprecation

It is an issue .. which I think has been pushed to completion...it does now need a little review.

If anyone has a pet issue which is in need of attention....I am up for trading reviews :)

jibran’s picture

Let's create another combine patch with the utf8 issue also upload the review the only patch which I assume is uploaded in #18.

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new327.13 KB
new4.47 KB

The patch from #24 is now outdated ( this was committed #2948700: Casting $text value to string in responseContains/responseNotContains methods )

So regarding #27 I am folding in the utf-8 issue ( 2961688-9.patch )

onto the patch from our last known good patch 2976394-18.patch

Status: Needs review » Needs work

The last submitted patch, 28: 2976394-28.patch, failed testing. View results

martin107’s picture

There are several errors in the test results of this form

Drupal\Tests\Core\Controller\ControllerResolverTest::testGetArguments
Error: Call to undefined method Drupal\Core\Controller\ControllerResolver::getArguments()

I am just pooling information here, so others do not have to ... in Drupal land this is a known issue
here is a change record.

https://www.drupal.org/node/2959408

Here is the Symfony4 upgrade documentation

The ControllerResolver::getArguments() method has been removed. If you have your own ControllerResolverInterface implementation, you should inject an ArgumentResolverInterface instance.

https://github.com/symfony/symfony/blob/4.1/UPGRADE-4.0.md

I am still forming an opinion as to the best way forward

As always I want to map problems back to existing D8.7 issues -- this seems to be the closest.

#2916310: Deprecate Drupal core ArgumentsResolver class?

martin107’s picture

1)

After looking I don't think there much doing in regard to resolving the getArguments in D8 for D9.
Every way forward involves breaking changes.

thinking about the drush submodule

Drush\Drupal\DrupalKernel extends Drupal\Core\DrupalKernel

It would be appropriate to signal to contrib that the arguments of the DrupaKernel constructor are deprecated/scheduled for change in D9... but there a hundred of D9 design decisions to be made I can't predict the final shape of the constructor to say anything meaningful.

2)

There are several errors of this form

Drupal\Tests\Core\Controller\FormControllerTest::testControllerResolverDeprecation
TypeError: Argument 1 passed to Drupal\Core\Controller\HtmlFormController::__construct() must be an instance of Symfony\Component\HttpKernel\Controller\ArgumentResolverInterface, instance of Mock_ControllerResolver_adf0402a given, called in /var/www/html/core/tests/Drupal/Tests/Core/Controller/FormControllerTest.php on line 27

/var/www/html/core/lib/Drupal/Core/Controller/HtmlFormController.php:32
/var/www/html/core/tests/Drupal/Tests/Core/Controller/FormControllerTest.php:27

This annotation below is apposite ... does anyone know if there is a quick code trick to prevent testbot from running through legacy code ?

 /**
   * @expectedDeprecation Using the 'controller_resolver' service as the first argument is deprecated, use the 'http_kernel.controller.argument_resolver' instead. If your subclass requires the 'controller_resolver' service add it as an additional argument. See https://www.drupal.org/node/2959408.
   * @group legacy
   */
  public function testControllerResolverDeprecation() {
martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new331.24 KB
new4.11 KB

The interdiff is unreviewable

I am just including phpunit.xml file which differs from phpunit.xml.dist in only one regard

<env name="SYMFONY_DEPRECATIONS_HELPER" value="disabled"/>

‘Messrs Moony, Wormtail, Padfoot and Prongs’ solemnly swear they are up to no good - when they turn off legacy tests.

Status: Needs review » Needs work

The last submitted patch, 32: 2976394-32.patch, failed testing. View results

catch’s picture

It would be appropriate to signal to contrib that the arguments of the DrupalKernel constructor are deprecated/scheduled for change in D9...

It's fine to deprecate constructor arguments and trigger the deprecation in a minor, and a change record is enough. It doesn't matter if the constructor changes again in another minor.

martin107’s picture

When I read the Symfony blogs -- suggesting that for large complex applications, Symfony4 is twice as fast.
There seem to be three key area for the gains.

1) A refactoring of routing, a better understanding based on simultaneous parsing of a single mega regular expression

            .'|/(en|fr|de|es)/admin/post/?(*:82)'
            .'|/(en|fr|de|es)/admin/post/new(*:166)'
            .'|/(en|fr|de|es)/admin/post/(\\d+)(*:253)'
            .'|/(en|fr|de|es)/admin/post/(\\d+)/edit(*:345)'
            .'|/(en|fr|de|es)/admin/post/([^/]++)/delete(*:442)'
            .'|/(en|fr|de|es)/blog/?(*:519)'

https://symfony.com/blog/new-in-symfony-4-1-fastest-php-router

2) A side effect of autowiring, which means that orphaned services, which is not utilised/called get automatically left out of the compiled services definition.

3) Just the fact of dropping all their legacy code.
a) less parsing given a reduced code footprint,
b) less branching to contend with.
c) the new way was designed specifically 'cos it was faster.

I don't claim to have encyclopedic knowledge of all the architecture and performance reasons for departing from the Symfony kernel...

But to me, given 1 and 2 and maybe 3 means that as a group we need to profile/benchmark again for D9 ... I don't want to speak heresy.. but maybe the DrupalKernel simply goes away after we pick up symfony4
or at least looks so different that all the pre optimisation and deprecation that I want to do here is just meaningless.

catch’s picture

1) A refactoring of routing, a better understanding based on simultaneous parsing of a single mega regular expression

We don't actually use Symfony's routing logic at all at this point, just some interfaces and the Route class.

This was done to be scalable with large number of routes (i.e. the problem they claim to have solved). So any comparison would have to be between our database-based router and the new version of Symfony's router - i.e. it will require a separate issue to undo our customisations and then test.

martin107’s picture

Status: Needs work » Needs review
Related issues: +#2997771: GetResponseForExceptionEvent: Prepare for Drupal9
StatusFileSize
new332.9 KB
new1.66 KB

We don't actually use Symfony's routing logic at all at this point, just some interfaces and the Route class.

Thank you for fixing my broken thinking..

My attempts to turn off legacy tests failed miserably....Many of the failing tests are legacy test but I have found and corrected a non legacy test...

Drupal\Tests\Core\EventSubscriber\CustomPageExceptionHtmlSubscriberTest::testHandleWithPostRequest
TypeError: Argument 3 passed to Symfony\Component\HttpKernel\Event\GetResponseForExceptionEvent::__construct() must be of the type integer, string given, called in /var/www/html/core/tests/Drupal/Tests/Core/EventSubscriber/CustomPageExceptionHtmlSubscriberTest.php on line 141

I have spun off a new issue.

Status: Needs review » Needs work

The last submitted patch, 37: 2976394-36.patch, failed testing. View results

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new336.96 KB
new4.06 KB

I have broadened the scope of

#2997771: GetResponseForExceptionEvent: Prepare for Drupal9

that fixes more issues here... now I just feeding back the results. Here is a list of the files affected.

core/modules/serialization/tests/src/Unit/EventSubscriber/DefaultExceptionSubscriberTest.php
core/tests/Drupal/Tests/Core/EventSubscriber/ExceptionJsonSubscriberTest.php
core/tests/Drupal/Tests/Core/EventSubscriber/FinalExceptionSubscriberTest.php

Status: Needs review » Needs work

The last submitted patch, 39: 2976394-39.patch, failed testing. View results

jibran’s picture

Some of the fails in the last patch have been fixed in #2992113: Update core dependencies before 8.6.2

martin107’s picture

Cool thanks for the info....

just for the record .. it is the "Content-Disposition" stuff.

andypost’s picture

What about rewrite \Drupal\Core\DrupalKernel::initializeSettings() which using deprecated \Symfony\Component\ClassLoader\ApcClassLoader ref https://symfony.com/doc/3.3/components/class_loader/cache_class_loader.html

It means whole symfony/class-loader dependency must be removed

andypost’s picture

Looks related issue could be fixed in minor release but not sure about removal of dependency

tim.plunkett’s picture

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new317.32 KB

andypost, tim.plunkett thank you for broadening my perspective.

In previous patches, I had folded in issues that are now fixed. Plus I had some dead code that did not do what I wanted.

So I am starting again - with only changes to composer.lock core/composer.json and core/composer.lock

I am still expecting lots of failures - but with so many recent changes I am curious, again, to know the state of play.

Status: Needs review » Needs work

The last submitted patch, 46: 2976394-46.patch, failed testing. View results

catch’s picture

Nice. Remaining fails are variations on this:

1) Drupal\Tests\file\Functional\Hal\FileUploadHalJsonCookieTest::testPostFileUploadInvalidHeaders
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'{"message":"\u0022Content-Disposition\u0022 header is required. A file name in the format \u0022filename=FILENAME\u0022 must be provided"}'
+'{"message":"No filename found in \u0022Content-Disposition\u0022 header. A file name in the format \u0022filename=FILENAME\u0022 must be provided"}'

/var/www/html/core/modules/rest/tests/src/Functional/ResourceTestBase.php:393
/var/www/html/core/modules/rest/tests/src/Functional/ResourceTestBase.php:455
/var/www/html/core/modules/rest/tests/src/Functional/FileUploadResourceTestBase.php:258

If we can make the assertion a bit more agnostic that might be enough?

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new319.56 KB
new2.23 KB

a)

If we can make the assertion a bit more agnostic that might be enough?

This relates to #41-#42 so I am just going to wait for

#2992113: Update core dependencies before 8.6.2

which is RTBC

b)

The changes here are a response to #43 which pointed out that the tail of

\Drupal\Core\DrupalKernel::initializeSettings()

contains calls to three deprecated classloaders ... which are dropped in favour of an entry in a composer config section.

Status: Needs review » Needs work

The last submitted patch, 49: 2976394-49.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

voleger’s picture

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new295.97 KB

Reroll.

Status: Needs review » Needs work

The last submitted patch, 52: 2976394-52.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

andypost’s picture

Looks only classloader & #48 tests are need fix now

martin107’s picture

Status: Needs work » Needs review
StatusFileSize
new298.51 KB

reroll.

Status: Needs review » Needs work

The last submitted patch, 55: 2976394-55.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new968 bytes
new299.26 KB

#48: interesting failure! I agree with If we can make the assertion a bit more agnostic that might be enough?.

… at least I agreed based on the assumption that a Symfony change caused the exception message to change. But both exceptions (and their messages) are thrown by Drupal code (\Drupal\file\Plugin\rest\resource\FileUploadResource::validateAndParseContentDispositionHeader()). Which means that Symfony changed something.

This is the relevant code:

  protected function validateAndParseContentDispositionHeader(Request $request) {
    // Firstly, check the header exists.
    if (!$request->headers->has('content-disposition')) {
      throw new BadRequestHttpException('"Content-Disposition" header is required. A file name in the format "filename=FILENAME" must be provided');
    }

    $content_disposition = $request->headers->get('content-disposition');

    // Parse the header value. This regex does not allow an empty filename.
    // i.e. 'filename=""'. This also matches on a word boundary so other keys
    // like 'not_a_filename' don't work.
    if (!preg_match(static::REQUEST_HEADER_FILENAME_REGEX, $content_disposition, $matches)) {
      throw new BadRequestHttpException('No filename found in "Content-Disposition" header. A file name in the format "filename=FILENAME" must be provided');
    }

…

So the test failures are occurring because a request with Foo: had $request->headers->has('foo') return FALSE in Symfony 3 but TRUE in Symfony 4.

Status: Needs review » Needs work

The last submitted patch, 57: 2976394-57.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

wim leers’s picture

From 8 to 4 failures. #57 worked :) The remaining 4 failures are in the class loader, composer integration and kernel.

andypost’s picture

So the test failures are occurring because a request with Foo: had $request->headers->has('foo') return FALSE in Symfony 3 but TRUE in Symfony 4.

Both versions mostly identical, the only change in SF not related to has() implementation

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new2.93 KB
new118.94 KB
new119.9 KB
new3.89 KB

The DrupalKernel test failure is testing for the apcu classloader which we no longer attempt to register, so we probably just need to remove that test entirely.

Also

1) Drupal\Tests\ComposerIntegrationTest::testMinPHPVersion
doctrine/cache has a PHP dependency requirement of "~7.1"
Failed asserting that false is true.

/var/www/html/core/tests/Drupal/Tests/ComposerIntegrationTest.php:208

For the purposes of this we need to bump our PHP requirements to 7.1, another option would be to remove that test.

Theoretically gets us down to two fails. Also had to re-roll a composer.json change.

catch’s picture

Re #57 we already had to deal with a similar issue in #2992113-17: Update core dependencies before 8.6.2.

The last submitted patch, 61: 2976394-61.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 61: 2976394-62.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new547 bytes
new119.9 KB

Actual PHP requirement needs to be 7.1.3 not 7.1.

wim leers’s picture

Aha, my bad! It's introduced by Guzzle 6.3.0 -> 6.3.3 then.

Status: Needs review » Needs work

The last submitted patch, 65: 2976394-64.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

berdir’s picture

+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -1067,40 +1067,6 @@ protected function initializeSettings(Request $request) {
-
-    // If the class loader is still the same, possibly
-    // upgrade to an optimized class loader.
-    if ($class_loader_class == get_class($this->classLoader)
-        && Settings::get('class_loader_auto_detect', TRUE)) {
-      $prefix = Settings::getApcuPrefix('class_loader', $this->root);
-      $loader = NULL;
-
-      // We autodetect one of the following three optimized classloaders, if
-      // their underlying extension exists.
-      if (function_exists('apcu_fetch')) {
-        $loader = new ApcClassLoader($prefix, $this->classLoader);
-      }

Do we need to test the impact of this somehow? Or find an alternative?

I imagine a lot of users aren't going to be using --optimize-classloader. At the same time, wondering if it might actually be faster without this for those that *do*, or did we already skip it then, not sure I fully understand the check here righ tnow.

One thing though, if we remove this, we can also remove "$class_loader_class = get_class($this->classLoader);" a few lines above.

catch’s picture

@Berdir: "Do we need to test the impact of this somehow? Or find an alternative?"

+++ b/composer.json
@@ -13,6 +13,7 @@
+        "apcu-autoloader": true,

The patch currently does this, but yeah we need to see what's going on and might have to make our own apcu classloader if this is no good.

Nothing stops us adding our own apcu classloader to 8.x if we need to.

catch’s picture

This is what's left and a real problem:

1) Drupal\Tests\system\Functional\Module\ClassLoaderTest::testClassLoadingNotInstalledModules
"Drupal\module_autoload_test\SomeClass::testMethod() was invoked." not found
Failed asserting that false is true.

/var/www/html/core/tests/Drupal/KernelTests/AssertLegacyTrait.php:31
/var/www/html/core/tests/Drupal/FunctionalTests/AssertLegacyTrait.php:140
/var/www/html/core/tests/Drupal/FunctionalTests/AssertLegacyTrait.php:111
/var/www/html/core/modules/system/tests/src/Functional/Module/ClassLoaderTest.php:56

2) Drupal\Tests\system\Functional\Module\ClassLoaderTest::testClassLoadingDisabledModules
"Drupal\module_autoload_test\SomeClass::testMethod() was invoked." not found
Failed asserting that false is true.

/var/www/html/core/tests/Drupal/KernelTests/AssertLegacyTrait.php:31
/var/www/html/core/tests/Drupal/FunctionalTests/AssertLegacyTrait.php:140
/var/www/html/core/tests/Drupal/FunctionalTests/AssertLegacyTrait.php:111
/var/www/html/core/modules/system/tests/src/Functional/Module/ClassLoaderTest.php:76
andypost’s picture

@catch does it mean that 8.x should use some shim/bridge to current classloader interface while 9.x should be composer-based #1818628: Use Composer's optimized ClassLoader for Core/Component classes where in #95 I filed that apcu is not the best now

Probably better to deprecate class_loader_auto_detect settings?

catch’s picture

@andypost not sure yet, we should open an issue to discuss this separately from this one, but just a note #2704571: Add an APCu classloader with a single entry has been open for two years and needs reviews.

catch’s picture

I'm unable to reproduce the fail in #70 locally, would be useful if someone else could try running that with the latest patch.

dawehner’s picture

+++ b/core/modules/file/src/Plugin/rest/resource/FileUploadResource.php
@@ -349,7 +349,7 @@ protected function streamUploadData() {
-    if (!$request->headers->has('content-disposition')) {
+    if (!$request->headers->has('content-disposition') || empty($request->headers->get('content-disposition'))) {
       throw new BadRequestHttpException('"Content-Disposition" header is required. A file name in the format "filename=FILENAME" must be provided');
     }

This change looks like something which could be a problem in more spaces, we just don't have the needed test coverage.

catch’s picture

Issue tags: +Drupal 9
gábor hojtsy’s picture

jibran’s picture

martin107’s picture

.. then xyz is a duplicate.

When I created this issue I wanted this to be a simple test - a pipe cleaning exercise exercise - which we need to periodically update and recheck.

The PHP bump occurs here only out of necessity- to get the thing to work.

Bumping the PHP version as a candidate for committing ( Which I support ) should be dealt with as a separate isolated issue.

martin107’s picture

Title: Investigate problems with Symfony4 now » Investigate problems with Symfony5 now

Sorry my time for open source is limited at the moment

but given Symfony 5 is the thing we really want!

From https://dri.es/plan-for-drupal-9

Our plan is to release Drupal 9 with Symfony 4 or 5. Symfony 5's release is less than one year away, while Symfony 4 was released a year ago. Ideally Drupal 9 would ship with Symfony 5,

When I find the time I want to re-jig our composer files and pull the latest version of Symfony5 dev -- not yet released.

Maybe someone want to beat me to it?

gábor hojtsy’s picture

Title: Investigate problems with Symfony5 now » Investigate problems with Symfony4 now

@martin107: there is not even a branch yet for Symfony 5 at https://github.com/symfony/symfony! As per the same blog post, let's focus on supporting Symfony 4 as a stepping stone to Symfony 5, so we don't need to do all the work at closer to the finish line and frontload some effort as much as we can. The Symfony 5 issue is at #3009255: [META] Symfony 5.0 compatibility by the way.

catch’s picture

Title: Investigate problems with Symfony4 now » Investigate problems with Symfony5 now

For now I re-rolled over at #3020303: Consider using Symfony flex to allow switching between major Symfony versions and discovered new test failures due to newly introduced deprecations in Symfony 4.2.

catch’s picture

Title: Investigate problems with Symfony5 now » Investigate problems with Symfony4 now

Cross-post - and agreed with Gabor. Symfony 5 will probably branch around the same time as Symfony 4.4 does, until then we need to track Symfony 4 minor updates which we may also have to adapt for.

catch’s picture

Title: Investigate problems with Symfony4 now » Investigate problems with Symfony 4 now
Status: Needs work » Needs review

OK there are multiple new issues introduced with Symfony 4.2, so moving back here to flush them out, but will keep opening sub-issues.

Current state of the patch:

1. Keeps symfony-classloader pinned at 3.4, that's all we need to do to enable Symfony 4, although we'll need to replace it in time for 9.x in #3020296: Remove Symfony's classloader as it does not exist in Symfony 4

2. Incorporates the patch from #3020385: Symfony 3 and Symfony ^4.2 JsonEncode constructors are incompatible.

3. Incorporates #3020301: Remove composer integration PHP version test (rely on DrupalCI PHP version runs instead).

I'm adding these as children under #2937984: [META] Symfony 4.0 compatibility.

catch’s picture

StatusFileSize
new107.53 KB

Status: Needs review » Needs work

The last submitted patch, 84: 2976394-83.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new110.77 KB

New combi patch - I'm splitting out sub-issues as I go, so see the child issues of #2937984: [META] Symfony 4.0 compatibility for specifics.

catch’s picture

The last submitted patch, 86: 2976394-86.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 87: 2976394-87.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

catch’s picture

Status: Needs review » Needs work

The last submitted patch, 90: 2976394-90.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

jibran’s picture

Issue summary: View changes
Issue tags: +Needs issue summary update

Are planning to commit this patch to D8 at some point? Can we please make the expectation clear in IS? Because as it stands composer install is failing on PHP 7.0 which means we can't commit this patch to Drupal 8.8 unless we fix it.

catch’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Only the spin-off issues should be committed, this issue is mostly for testing. I updated the issue summary.

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new156.45 KB

Re-rolled.

Status: Needs review » Needs work

The last submitted patch, 94: 2976394-94.patch, failed testing. View results

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new157.47 KB

Doctrine needs to be at 2.10.

Status: Needs review » Needs work

The last submitted patch, 96: 2976394-96.patch, failed testing. View results

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new163.35 KB
catch’s picture

StatusFileSize
new146.86 KB

Status: Needs review » Needs work

The last submitted patch, 99: 2976394-99.patch, failed testing. View results

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new145.75 KB

Status: Needs review » Needs work

The last submitted patch, 101: 2976394-101.patch, failed testing. View results

effulgentsia’s picture

Status: Needs work » Needs review
effulgentsia’s picture

Status: Needs review » Needs work

Doesn't look like every component got updated to 4.x for the test run. E.g.,

+++ b/composer.lock
@@ -2331,20 +2692,20 @@
             "name": "symfony/validator",
-            "version": "v3.4.15",
+            "version": "v3.4.21",

And others.

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new152.71 KB

A few components are still specifying <4.0.0 so we need to update those too. Good spot. New patch.

Status: Needs review » Needs work

The last submitted patch, 105: 2976394-105.patch, failed testing. View results

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new152.18 KB

Cruft.

catch’s picture

StatusFileSize
new155.78 KB

The last submitted patch, 107: 2976394-105.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 108: 2976394-108.patch, failed testing. View results

gábor hojtsy’s picture

Looking at various fails, this pattern is repeating:

Drupal\Tests\field\Functional\Hal\FieldConfigHalJsonCookieTest::testGet
Exception: TypeError: Argument 1 passed to Symfony\Component\HttpFoundation\Response::setCharset() must be of the type string, null given, called in /var/www/html/core/modules/rest/src/EventSubscriber/ResourceResponseSubscriber.php on line 210
Symfony\Component\HttpFoundation\Response->setCharset()() (Line: 496)

Drupal\Tests\action\Functional\ActionListTest::testEmptyActionList
Exception: TypeError: Argument 1 passed to Symfony\Component\HttpFoundation\Response::setCharset() must be of the type string, null given, called in /var/www/html/core/lib/Drupal/Component/HttpFoundation/SecuredRedirectResponse.php on line 43
Symfony\Component\HttpFoundation\Response->setCharset()() (Line: 496)

There are also other fails, some of which look like could be side effects of this (ie. when a webtest checks site output and the page fails with the exception).

jibran’s picture

Status: Needs work » Needs review
StatusFileSize
new152.18 KB
new153.38 KB
jibran’s picture

Should we upgrade these libraries as well?

diff --git a/core/composer.json b/core/composer.json
index 9252ba4d6a..6f8a836b80 100644
--- a/core/composer.json
+++ b/core/composer.json
@@ -61,9 +61,9 @@
         "mikey179/vfsStream": "^1.2",
         "phpunit/phpunit": "^4.8.35 || ^6.5",
         "phpspec/prophecy": "^1.7",
-        "symfony/css-selector": "^3.4.0",
-        "symfony/phpunit-bridge": "^3.4.3",
-        "symfony/debug": "^3.4.0"
+        "symfony/css-selector": "^3.4.0|~4.0",
+        "symfony/phpunit-bridge": "^3.4.3|~4.0",
+        "symfony/debug": "^3.4.0|~4.0"
catch’s picture

There's a lot of errors like

2) Drupal\Tests\aggregator\Functional\AddFeedTest::testFeedLabelEscaping
"The feed Test feed title &amp;lt;script&amp;gt;alert(123);&amp;lt;/script&amp;gt; has been added." found
Failed asserting that false is true.

No time to spin an issue out right now though, but that seems like it'd clear a lot of the remaining 200 maybe.

gábor hojtsy’s picture

Status: Needs review » Needs work

@jibran: re #115 I think so yeah. Now that #3027871: [symfony 5] Symfony\Component\Validator\Constraints\Email::$strict property is deprecated since Symfony 4.1 landed, this will need a reroll. That in itself will not raise new issues, but #115 may.

@catch: interesting, most fails are in messages, but I don't see the pattern, so not sure what to open an issue about. This message fail from a JS test may help find the reason:

1) Drupal\Tests\block_content\Functional\Hal\BlockContentHalJsonAnonTest::testPost
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'{"message":"Unprocessable Entity: validation failed.\ninfo: Block description: this field cannot hold more than 1 values.\n"}'
+'{"message":"Unprocessable Entity: validation failed.\nstatus.0: The value you selected is not a valid choice.\ninfo: Block description: this field cannot hold more than 1 values.\nreusable.0: The value you selected is not a valid choice.\ndefault_langcode.0: The value you selected is not a valid choice.\n"}'
jibran’s picture

Status: Needs work » Needs review
Issue tags: +Symfony 4
StatusFileSize
new6.09 KB
new153.84 KB

Addressed #115.

Status: Needs review » Needs work

The last submitted patch, 118: 2976394-118.patch, failed testing. View results

mikelutz’s picture

There's a lot of errors like

2) Drupal\Tests\aggregator\Functional\AddFeedTest::testFeedLabelEscaping
"The feed Test feed title &amp;lt;script&amp;gt;alert(123);&amp;lt;/script&amp;gt; has been added." found
Failed asserting that false is true.

No time to spin an issue out right now though, but that seems like it'd clear a lot of the remaining 200 maybe.

given

1) Drupal\Tests\block_content\Functional\Hal\BlockContentHalJsonAnonTest::testPost
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'{"message":"Unprocessable Entity: validation failed.\ninfo: Block description: this field cannot hold more than 1 values.\n"}'
+'{"message":"Unprocessable Entity: validation failed.\nstatus.0: The value you selected is not a valid choice.\ninfo: Block description: this field cannot hold more than 1 values.\nreusable.0: The value you selected is not a valid choice.\ndefault_langcode.0: The value you selected is not a valid choice.\n"}'

Those look to be related to #2937542: Not setting the strict option of the Choice constraint to true is deprecated since Symfony 3.4 and will throw an exception in 4.0

I'm a bit curious. This patch adds #3029197: [symfony 5] Class implements "Symfony\Component\DependencyInjection\ResettableContainerInterface" that is deprecated since Symfony 4.2, use "Symfony\Contracts\Service\ResetInterface" instead and #3029199: [symfony 5] Calling "Symfony\Component\HttpFoundation\Request::getSession()" when no session has been set is deprecated since Symfony 4.1 to the deprecation ignore list and includes the patch from #2937542: Not setting the strict option of the Choice constraint to true is deprecated since Symfony 3.4 and will throw an exception in 4.0 . I'm curious how much that cleans up.

mikelutz’s picture

A good chunk of the remaining functional test failures seem to be due to escaping Constraint Violation messages. The test is looking for A feed named <em class="placeholder">iD5qjReymv</em> already exists. Enter a unique title. and the response contains A feed named &lt;em class=&quot;placeholder&quot;&gt;iD5qjReymv&lt;/em&gt; already exists. Enter a unique title.

I'm only starting to dig into why that might be happening, but I thought I would mention it in case it triggered any ideas with anyone who has been working with this more than I have. It's definitly coming through as A feed named <em class="placeholder">iD5qjReymv</em> already exists. Enter a unique title. on the output page without the upgraded dependencies.

alexpott’s picture

#123 is being caused by type coercion. \Symfony\Component\Validator\ConstraintViolation::__construct() causes the TranslatableMarkup object to be casted into a string and therefore Twig no longer considers this as a safe string and will escape the HTML. The only thing we can do here is sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation()... but the Symfony class uses privates... not going to be pretty.

gábor hojtsy’s picture

Status: Needs review » Needs work
catch’s picture

Status: Needs work » Needs review
StatusFileSize
new162.9 KB

Re-rolling, and also using the most recent patch from #3029197: [symfony 5] Class implements "Symfony\Component\DependencyInjection\ResettableContainerInterface" that is deprecated since Symfony 4.2, use "Symfony\Contracts\Service\ResetInterface" instead because the alternative approach there is not yet covering all of those deprecations.

catch’s picture

Status: Needs review » Needs work

The last submitted patch, 129: 2976394-129.patch, failed testing. View results

catch’s picture

catch’s picture

gábor hojtsy’s picture

Status: Needs work » Needs review
StatusFileSize
new161.43 KB

Rerolled. The only patch that was in this one was #3029500: Drupal\Core\Database\Query\Select::hasAllTags() and hasAnyTag() will require a new "$tags" argument in the next major version of its parent class. Other things that landed since then should improve the passing rate.

Status: Needs review » Needs work

The last submitted patch, 134: 2976394-134.patch, failed testing. View results

berdir’s picture

> Error: Call to undefined method Symfony\Component\HttpFoundation\Request::setTrustedHeaderName()

https://symfony.com/blog/fixing-the-trusted-proxies-configuration-for-sy...

Looks like they forgot to mark that as deprecated in 3.3?

gábor hojtsy’s picture

@berdir: ReverseProxyMiddlewareTest has an @expectedDeprecation for setTrustedHeaderName() so we hid it for ourselves.

I went through all the fails and I think this covers it all:

This combined patch currently contains:

- #3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation()
- #3029197: [symfony 5] Class implements "Symfony\Component\DependencyInjection\ResettableContainerInterface" that is deprecated since Symfony 4.2, use "Symfony\Contracts\Service\ResetInterface" instead

The fails that it produces are covered by:

- #3029979: [Symfony 4] ClassFinder test is broken by Symfony's debug classloader
- #3030474: Drupal\Core\TypedData\Validation\ExecutionContext uses internal methods from Symfony\Component\Validator\Context\ExecutionContextInterface (new)
- #3030485: [Symfony 5] The "Drupal\Core\Validation\TranslatorInterface" interface extends "Symfony\Component\Translation\TranslatorInterface" that is deprecated since Symfony 4.2, use Symfony\Contracts\Translation\TranslatorInterface instead. (new)
- #3030494: [Twig 3] The "Drupal\Core\Template\Loader\StringLoader" class implements "Twig_ExistsLoaderInterface" and "Twig_SourceContextLoaderInterface" that are deprecated since 1.12 and 1.27 (to be removed in 3.0). (new)
- #3030501: [Symfony 4] Drupal\Core\StackMiddleware\ReverseProxyMiddleware calls Symfony\Component\HttpFoundation\Request::setTrustedHeaderName() which does not exist (new)

I think the combination of those cover ALL the fails we are seeing in #134.

gábor hojtsy’s picture

Additionally to the above, the following two Symfony 4 deprecation messages are made to disappear in Drupal\Tests\Listeners\DeprecationListenerTrait:

      'The Symfony\Component\ClassLoader\ApcClassLoader class is deprecated since Symfony 3.3 and will be removed in 4.0. Use `composer install --apcu-autoloader` instead.',
      'The Symfony\Component\ClassLoader\WinCacheClassLoader class is deprecated since Symfony 3.3 and will be removed in 4.0. Use `composer install --apcu-autoloader` instead.',

I also looked at @expectedDeprecation instances in core based on finding out that those were hiding an issue in #3030501. There are these in Drupal\Tests\Component\EventDispatcher\ContainerAwareEventDispatcherTest:

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. U
se EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testAddAListenerService() {
      parent::testAddAListenerService();
    }

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. U
se EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testPreventDuplicateListenerService() {
      parent::testPreventDuplicateListenerService();
    }

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. Use EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testAddASubscriberService() {
      parent::testAddASubscriberService();
    }

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. Use EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testHasListenersOnLazyLoad() {
      parent::testHasListenersOnLazyLoad();
    }

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. Use EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testGetListenersOnLazyLoad() {
      parent::testGetListenersOnLazyLoad();
    }

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. Use EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testRemoveAfterDispatch() {
      parent::testRemoveAfterDispatch();
    }

    /**
     * @expectedDeprecation The Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher class is deprecated since Symfony 3.3 and will be removed in 4.0. Use EventDispatcher with closure factories instead.
     * @group legacy
     */
    public function testRemoveBeforeDispatch() {
      parent::testRemoveBeforeDispatch();
    }
gábor hojtsy’s picture

#3030501: [Symfony 4] Drupal\Core\StackMiddleware\ReverseProxyMiddleware calls Symfony\Component\HttpFoundation\Request::setTrustedHeaderName() which does not exist just landed, yay!

As for the classloader skipped deprecation messages uncovered in #138 I believe #3020296: Remove Symfony's classloader as it does not exist in Symfony 4 will deal with them, but that seems to be possible to do independent of Symfony 4 compatibility as we can independently stick to the old class loader versions at least alongside Symfony 4.

I don't know what to do about the @expectedDeprecation's around ContainerAwareEventDispatcher.

gábor hojtsy’s picture

Status: Needs review » Needs work

The last submitted patch, 141: 2976394-141.patch, failed testing. View results

gábor hojtsy’s picture

The fails from #2937542: Not setting the strict option of the Choice constraint to true is deprecated since Symfony 3.4 and will throw an exception in 4.0 are back (it was rolled back in the meantime), will roll a compound patch that has that too.

gábor hojtsy’s picture

Status: Needs work » Needs review
StatusFileSize
new172.06 KB

Rolled with that. Let's see testbot.

Status: Needs review » Needs work

The last submitted patch, 144: 2976394-144.patch, failed testing. View results

gábor hojtsy’s picture

@alexpott created #3038062: [Symfony 4] Skip ReverseProxyMiddlewareTest::testReverseProxyEnabledLegacy test to properly skip testing \Symfony\Component\HttpFoundation\Request::setTrustedHeaderName() when it does not exist. Running a compound test there to see how it helps.

gábor hojtsy’s picture

Status: Needs work » Needs review
StatusFileSize
new153.76 KB

Status: Needs review » Needs work

The last submitted patch, 147: 2976394-147.patch, failed testing. View results

gábor hojtsy’s picture

Status: Needs work » Needs review
StatusFileSize
new160.42 KB

Status: Needs review » Needs work

The last submitted patch, 149: 2976394-147-with-3029540-2.patch, failed testing. View results

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mikelutz’s picture

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new171.03 KB
new39.71 KB

This updates Symfony 4.2.2 to 4.2.4, which should fix at least one of the 10 remaining failures.

Status: Needs review » Needs work
mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new163.17 KB
new24.46 KB

Let's try that one more time...

mikelutz’s picture

StatusFileSize
new560 bytes
new163.71 KB

Alright then... I thought I screwed up the update, but that wasn't me.. We require symfony/psr-http-message-bridge: ^1.0 which is now 1.2, which deprecates a core service class....

mikelutz’s picture

Last attempt tonight, putting psr-http-message-bridge back down to 1.1.1 for now. 1.2 requires php 7.1, so we aren't ready for that just yet.

Status: Needs review » Needs work
mikelutz’s picture

Well, all that work, and we traded one fixed test for one new fail, lol.

mikelutz’s picture

I found an issue for the eventDispatcher issue #2895487: [Symfony 4] ContainerAwareEventDispatcherTest depends on Symfony tests, and tagged it.

mikelutz’s picture

StatusFileSize
new162.89 KB

reroll

mikelutz’s picture

StatusFileSize
new162.67 KB

Rerolling here, the plain upgrade. I also added in swapping out the message factory for a psr17 message factory, so that we can use the symfony/psr-http-message-bridge 1.2 which has come out recently.

Status: Needs review » Needs work

Status: Needs review » Needs work
mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new252.87 KB
new3.78 KB

Rolling in more JSON:api fixes from a reroll of #3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation() fixing a couple new instances of ->getMessage() introduced by JSON:api

Status: Needs review » Needs work
mikelutz’s picture

Status: Needs review » Needs work
mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new270.02 KB
new9.61 KB

Ignoring the deprecations..

Status: Needs review » Needs work
mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new275.11 KB
new5.09 KB

This should fix the last 4 tests. The two new twig failures look to be due to new fun twig changes between 1.38.2, and 1.38.4 and more conflicts with the changes we made in the hotfixes last week. Environment::resolveTemplate return type keeps changing, it seems. using ->load in the test seems to force a TemplateWrapper, and seems to be the right thing to call, but I'd love someone who knows more about twig to verify that is right.

The other two failures were due to stricter typecasting in the validation system again. Symfony is converting TranslateableMarkup to strings, but we didn't inject the TranslationManager into the test container in those tests. I think the conversion is fine here, we just need to add the needed service to the container. I need to open issues for both of those and put in the actual fixes, and come up with a list where if these 4 or 5 issues get committed, then Symfony 4 works. :-)

mikelutz’s picture

I opened #3042847: [Symfony 4] Inject string_translation service into ContextDefinitionIsSatisfiedTest and EntityContextDefinitionIsSatisfiedTest and #3042848: use ->load() instead of ->resolveTemplate in Twig Resolver tests for the last two pieces, and to test those changes against symfony 3.

Assuming everything comes back green, I think that everything needed for symfony 4 to work now has an issue and patch in needs review, and this issue has a combination of all of them showing green tests.

mikelutz’s picture

So, once the individual issues are resolved and committed, we will have to rollback the psr17 message factory I put into this patch for the actual commit to 8.8 that allows for symfony 4 without requiring it, because it requires diactros factory at version 2, which is php7.1, I think we will have to pin symfony/psr-http-message-factory to 1.1.1 or add the deprecations that 1.2 generates to the ignore list. I was going to suggest pinning it to 1.1.1, but I'm starting to lean towards just suppressing the deprecations. Would love FM opinions on #3039584: The "Symfony\Bridge\PsrHttpMessage\Factory\DiactorosFactory" class is deprecated since symfony/psr-http-message-bridge 1.2, use PsrHttpFactory instead. because that really does need to be resolved, and probably backported to at least 8.7 and probably 8.6 too.

I'm done rolling patches tonight, but once we are ready to roll the actual patch for 8.8 to allow symfony 4 we should have a resolution for that particular dependency one way or another.

mikelutz’s picture

mikelutz’s picture

Title: Investigate problems with Symfony 4 now » Allow Symfony 4 to be installed in Drupal 8
Priority: Normal » Major
Issue summary: View changes
Status: Needs review » Postponed
StatusFileSize
new265.37 KB
new9.68 KB
new159.28 KB

Here's a re-roll to keep this up to date. I'm including 3 patches.

  1. A potentially committable patch which allows Symfony4 installation through composer without changing the lock file. (This should be green)
  2. A test patch which does include a lock file. That patch should come back green once all the outstanding issues are resolved.
  3. A test patch which includes the lock file and all outstanding issue patches merged in, which should be green. This is purely a sanity check, should show us if any new issues pop up.

I'm postponing this issue on:
#3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation()
#3031577: \Drupal\Tests\Listeners\DeprecationListenerTrait::getSkippedDeprecations() does not work in unit tests
#3042694: [Symfony 4] JSON:API ResourceResponseSubscriber can pass NULL to Symfony\Component\HttpFoundation\Response::setCharset()
#3042847: [Symfony 4] Inject string_translation service into ContextDefinitionIsSatisfiedTest and EntityContextDefinitionIsSatisfiedTest

Until those are resolved,I don't believe there is anything further to do here except keep up with rerolls.

mikelutz’s picture

StatusFileSize
new9.37 KB
new511 bytes

Always one tiny thing... :-(

mikelutz’s picture

Rerolling all three patches.
#3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation() is the last remaining issue as the other three in #179 have been committed to 8.8.x

Also, I am trying having drupalci just run composer update for testing, rather than committing a large lock file. I'm not sure if it will work.

Also, I've switched the combined patch to use the simpler version of #3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation() from comment #2, which should work just fine with Symfony4 (though likely not Symfony5). Whatever route we go in that issue should still make us symfony4 compatible.

catch’s picture

Status: Postponed » Needs review

Un-postponing to start tracking Symfony 4.3

mile23’s picture

Note that symfony/browserkit 4.2.0+ supports meta refresh tags, thanks to @jhedstrom: https://github.com/symfony/symfony/pull/27118

This means our tests will have to change: https://git.drupalcode.org/project/drupal/blob/8.8.x/core/tests/Drupal/T...

alexpott’s picture

@Mile23/ re #183 - are you sure that something needs to change because of that? The meta refresh following looks optional and disabled by default so I'm not sure what'll need to change.

catch’s picture

Status: Needs review » Needs work

4.3.0-beta1 is out, so it should be possible to do a new patch here.

We'll still need to combine this with one of the patches from #3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation() to remove those test failures.

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new167.5 KB

Let's just see where we are at. I have to go back to including the lock file again because we won't pick up the beta with 'prefer-stable: true' and I don't want to set that to false and pick up other dev dependencies

Status: Needs review » Needs work
gábor hojtsy’s picture

Just sampled the first 150 fails of the 2.7k and those are all these:

  • Support for mapping keys in multi-line blocks is deprecated since Symfony 4.3 and will throw a ParseException in 5.0. in SimpletestUiTest::testTestingThroughUI from Drupal\Tests\simpletest\Functional
  • The "Symfony\Component\HttpFoundation\File\MimeType\MimeTypeGuesser" class is deprecated since Symfony 4.3, use "Symfony\Component\Mime\MimeTypes" instead. in SimpletestUiTest::testTestingThroughUI from Drupal\Tests\simpletest\Functional
  • The "Symfony\Component\HttpFoundation\File\MimeType\FileBinaryMimeTypeGuesser" class is deprecated since Symfony 4.3, use "Symfony\Component\Mime\FileBinaryMimeTypeGuesser" instead. in SimpletestUiTest::testTestingThroughUI from Drupal\Tests\simpletest\Functional
  • The "Symfony\Component\HttpFoundation\File\MimeType\FileinfoMimeTypeGuesser" class is deprecated since Symfony 4.3, use "Symfony\Component\Mime\FileinfoMimeTypeGuesser" instead. in SimpletestUiTest::testTestingThroughUI from Drupal\Tests\simpletest\Functional
  • The signature of the "Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher::dispatch()" method should be updated to "dispatch($event, string $eventName = null)", not doing so is deprecated since Symfony 4.3. in SimpletestUiTest::testTestingThroughUI from Drupal\Tests\simpletest\Functional
  • The "Symfony\Component\BrowserKit\Response::getStatus()" method is deprecated since Symfony 4.3, use getStatusCode() instead. in InstallUninstallTest::testInstallUninstall from Drupal\Tests\system\Functional\Module

So it looks like solving those would kill most (if not all) new deprecation errors.

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new2.54 KB
new168.69 KB

Lets suppress them for now, and break them out into separate issues, to see if there is any way to fix them for SF5 while keeping support for SF3.

Status: Needs review » Needs work
catch’s picture

Most (all?) the remaining ones are like this:

1) Drupal\Tests\book\Functional\BookContentModerationTest::testBookWithPendingRevisions
Behat\Mink\Exception\ResponseTextException: The text "You can only change the book outline for the published version of this content." was not found anywhere in the text of the current page.

All validation messages.

mikelutz’s picture

Status: Needs review » Needs work
gábor hojtsy’s picture

Summary of those results:

This happens with various event classes, this is just an example:

The "Drupal\Core\Routing\RouteBuildEvent" class extends "Symfony\Component\EventDispatcher\Event" that is deprecated since Symfony 4.3, use "Symfony\Contracts\EventDispatcher\Event" instead. in RouteSubscriberTest::testOnAlterRoutes from Drupal\Tests\views\Unit\EventSubscriber

Happens once:

The "Goutte\Client" class extends "Symfony\Component\BrowserKit\Client" that is deprecated since Symfony 4.3, use "\Symfony\Component\BrowserKit\AbstractBrowser" instead. in BrowserTestBaseTest::testGetHttpClient from Drupal\Tests\Core\Test

Appears 4 times:

The "Symfony\Component\Routing\CompiledRoute::serialize()" method is considered internal since Symfony 4.3, will be removed in Symfony 5 as the class won't implement Serializable anymore. It may change without further notice. You should not extend it from "Drupal\Core\Routing\CompiledRoute". in RouterTest::testMatchesWithDifferentFitOrder from Drupal\Tests\Core\Routing

Happens once:

The "Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher::dispatch()" method will require a new "string|null $eventName" argument in the next major version of its parent class "Symfony\Contracts\EventDispatcher\EventDispatcherInterface", not defining it is deprecated. in SectionRenderTest::setUp from Drupal\Tests\layout_builder\Unit

Still happens once:

The "Drupal\Core\File\MimeType\MimeTypeGuesser" class implements "Symfony\Component\HttpFoundation\File\MimeType\MimeTypeGuesserInterface" that is deprecated since Symfony 4.3, use {@link MimeTypesInterface} instead. in MimeTypeGuesserTest::testSymfonyGuesserRegistration from Drupal\Tests\Core\File

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new180.48 KB
new4.72 KB

Skipping the remaining deprecations to get back to green, then we can start picking them apart.

Status: Needs review » Needs work

The last submitted patch, 195: 2976394-195.drupal.Investigate-problems-with-Symfony-4-now-.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikelutz’s picture

Of course they moved AbstractEventDispatcherTest into EventDispatcherTest right after we switched over to it.. :-/

mikelutz’s picture

mikelutz’s picture

Issue summary: View changes
Status: Needs work » Needs review
Related issues: +#3055180: [META] Symfony 5 compatibility
StatusFileSize
new198.12 KB
new17.64 KB

Rolling in #3055040: Roll AbstractEventDispatcherTest into ContainerAwareEventDispatcherTest patch to get a green base to begin trying to pick off the individual deprecations into their own issues.

This leaves #3029540: [Symfony 4] Sub class \Symfony\Component\Validator\ConstraintViolation and use that in \Drupal\Core\TypedData\Validation\ExecutionContext::addViolation() and #3055040: Roll AbstractEventDispatcherTest into ContainerAwareEventDispatcherTest as requirements for Symfony 4 support. The remaining failures from #186 are deprecations in Symfony 4, and are technically tasks required for Symfony 5 support. I've opened #3055180: [META] Symfony 5 compatibility to begin tracking issues related to Symfony 5.

gábor hojtsy’s picture

Both sub-issues landed now. How do we ensure Symfony 4 keeps passing? Should we open an RTBC issue to keep retesting?

catch’s picture

Do we want a change record for this issue? This also unblocks #3020303: Consider using Symfony flex to allow switching between major Symfony versions which definitely would need a change record, maybe we should defer a change record until there?

I think an RTBC testing issue is a good idea so we can close this as fixed once that's opened with an updated patch.

mikelutz’s picture

I'm happy to re-roll a new patch we can use as an RTBC testing patch.

I'm not sure we can do #3020303: Consider using Symfony flex to allow switching between major Symfony versions. We discussed it in the Symfony 4/5 meeting in Seattle, and realized that it's going to cause problems. We had to do a lot of work to make core compatible with SF4, and it's very likely that there are contrib modules that will not work with SF4 right now, but we have no system to provide a way for contrib to test against SF4.

In composerland reality, if a contrib module is not compatible with SF4 for whatever reason, that should be declared in their composer file, and any site using that module would not be updated to SF4. In actual reality, no contrib module has been able to easily test against SF4 or given any thought to setting version constraints in composer against symfony. If we allowed SF4 in composer.json, contrib would begin getting bug reports that they wouldn't be able to reproduce on testbot.

I feel like we need a way to allow SF4 that is not going to auto install it on a simple composer update I'm not super familiar with composer scripting, I know the old drupal-phpunit-upgrade script forced the update to the 'already allowed but not in composer.lock' phpunit 6. Is it possible to have a composer script that also changes the requirements in composer.json to allow SF4 before trying to run the update?

gábor hojtsy’s picture

@mikelutz, @catch I think optional Symfony 4 support if it only needs patching composer files (or for people using https://github.com/drupal-composer/drupal-project not even that I think?) is valuable by itself. That said, we would need to commit the deprecation test silencing parts of the patch for it to be feasible. Otherwise patching too many files IMHO. We cannot claim Symfony 4 compatibility then. What do you two think?

gábor hojtsy’s picture

Ok I was enlightened just now. I was thinking we can augment core with updated composer dependencies just like https://github.com/webflo/drupal-core-strict does, but https://github.com/drupal-composer/drupal-project does not / cannot do away with core's existing composer dependency limitations, so we could only go more strict and not less strict.

So it would only be possible with a composer package independent of drupal/core (eg. drupal/core-sf4) and somewhere that composer package would need to be provided. In that case however, any package that sets an explicit dependency on drupal/core some version will be impossible to use with it. Or @berdir said we may be able to have a new branch 8.8-sf4 and branch alias so it appears like 8.8, see https://getcomposer.org/doc/articles/aliases.md#require-inline-alias -- that sounds like quite a bit of magic, especially given I don't know how d.o would deal with custom Drupal core branch names like that (update status, testbot, issue queue, etc).

gábor hojtsy’s picture

Whoops, sorry, let's discuss in #3020303: Consider using Symfony flex to allow switching between major Symfony versions where that discussion should be.

So leaving this open for this one from above from @catch:

I think an RTBC testing issue is a good idea so we can close this as fixed once that's opened with an updated patch.

Let's only do a change record when there is an actual change we can deal with. Since this requires composer changes to be finished, we should do them first before a change record.

mikelutz’s picture

Fresh patch to install symfony 4.3.1, interdiff may not be useful, but I removed the lock file now that we are back to a stable version of SF4.

mikelutz’s picture

Fixing the deprecation messages around route serialization, which changed in 4.3.1 (see https://github.com/symfony/symfony/pull/31866)

The last submitted patch, 206: 2976394-206.drupal.Investigate-problems-with-Symfony-4-now-.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 207: 2976394-207.drupal.Investigate-problems-with-Symfony-4-now-.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikelutz’s picture

mikelutz’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work
mikelutz’s picture

It's been a little while since we've done this, symfony 4.3.3 is now out. Let's see where we are.

Status: Needs review » Needs work
andypost’s picture

one more failed test `Drupal\Tests\Core\EventSubscriber\RedirectResponseSubscriberTest`

martin107’s picture

Ok so two things jump out at me.

1) EventDispatcher changes.
2) Process changes

Just making notes for myself might at well do it in public ... to save others from digging through the test artifacts.

1)

22x: Calling the "Symfony\Component\EventDispatcher\EventDispatcherInterface::dispatch()
" method with the event name as the first argument is deprecated since Symfony 4.3, pass it as the second argument and provide the event object as the first argument instead.

The order of the arguments is reversed so class for named services is optional

https://symfony.com/blog/new-in-symfony-4-3-simpler-event-dispatching

// Before
$dispatcher->dispatch(OrderEvents::NEW_ORDER, $newOrderEvent);

// After
$dispatcher->dispatch($newOrderEvent, OrderEvents::NEW_ORDER);

2) command processing

8x: Passing a command as string when creating a "Symfony\Component\Process\Process" instance is deprecated since Symfony 4.2,
pass it as an array of its arguments instead, or use the "Process::fromShellCommandline()" constructor if you need features provided by the shell.

in short put the arguments in an array -- escaping is an issue.

https://symfony.com/doc/current/components/process.html

mikelutz’s picture

Yes, and both of these are just deprecations in symfony 4. We can suppress the messages in Drupal 8 and still be compatible. We do need to address the event dispatcher changes, we should do some deprecations around our own dispatcher to prepare, and there are a couple issues out for that.

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new11.79 KB

new patch file to adjust for recent composer changes, no interdiff. Added one deprecation to the skip list to leave the actual errors relating to traversing xpaths in functional tests.

Status: Needs review » Needs work

The last submitted patch, 218: 2976394-218.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikelutz’s picture

Status: Needs work » Needs review

#3072752: Invalid markup in ImageItem (causes test failures with symfony/dom-crawler:4.3+) was committed, which should cover the remaining test failures since the last time the test was ran.

mikelutz’s picture

StatusFileSize
new11.88 KB

Here's a re-roll to test against 4.3.3

Status: Needs review » Needs work

The last submitted patch, 221: 2976394-221.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

gábor hojtsy’s picture

Status: Needs work » Needs review
StatusFileSize
new11.88 KB
new2.22 KB

Literally one word changed in the deprecation message text for dispatch :D This:

'Calling the "Symfony\Component\EventDispatcher\EventDispatcherInterface::dispatch()" method with the event name as {+the+} first argument is deprecated since Symfony 4.3, pass it as the second argument and provide the event object as the first argument instead.',

xjm’s picture

Issue tags: +mwds2019
mikelutz’s picture

StatusFileSize
new173.61 KB
new166.42 KB

Well, might as well push the envelope then. Let's try 4.4-dev :-)

Status: Needs review » Needs work

The last submitted patch, 225: 2976394-225.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new174.1 KB
new4.21 KB

Suppressing a few new deprecation messages. I opened #3074585: [Symfony 5] Replace app.root and site.path string services with container parameters for the one I think we could fix in D8, the other (Passing arguments to "Symfony\Component\HttpFoundation\Request::isMethodSafe()" has been deprecated since Symfony 4.4; use "Symfony\Component\HttpFoundation\Request::isMethodCacheable()" to check if the method is cacheable instead.) we could fix, but SF3 throws an error if that method is called WITHOUT arguments, so we would have to bypass the method altogether and decide safe methods on our own, which we could do, or just suppress the message until we are on SF4 and fix the call once D9 opens, which is my preference.

Status: Needs review » Needs work

The last submitted patch, 227: 2976394-227.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new2.78 KB
new174.8 KB

Additional deprecation suppression that can't easily be fixed in SF3:

mikelutz’s picture

Status: Needs review » Needs work

The last submitted patch, 231: 2976394-231.drupal.Allow-Symfony-4-to-be-installed-in-Drupal-8.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mikelutz’s picture

Fix typo in one of the deprecation suppressions. That leaves the following issues:

#3074627: Fix passing invalid second argument to $this->container->get() in Drupal\Tests\node\Kernel\NodeConditionTest::testConditions()
#3074632: The second argument to $container->register should be a class name string, not an object in AccessManagerTest
#3074645: Account for changing violation message in symfony 'range' constraint between SF3 and SF4.4

and

Error Message
exception: [Other] Line 0 of sites/default/files/simpletest/phpunit-51.xml:
PHPunit Test failed to complete; Error: PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

Testing Drupal\FunctionalJavascriptTests\Core\Installer\Form\SelectProfileFormTest
E                                                                   1 / 1 (100%)

Time: 2.94 seconds, Memory: 4.00MB

There was 1 error:

1) Drupal\FunctionalJavascriptTests\Core\Installer\Form\SelectProfileFormTest::testUmamiProfileWarningMessage
Error: Call to a member function isVisible() on null

/var/www/html/core/tests/Drupal/FunctionalJavascriptTests/Core/Installer/Form/SelectProfileFormTest.php:127

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Which regrettably is passing for me locally at the moment with chromedriver 75.

mikelutz’s picture

mikelutz’s picture

Still unable to duplicate the installer test fail locally.. Trying to get a little bit more information here.

Status: Needs review » Needs work
mikelutz’s picture

gábor hojtsy’s picture

Title: Allow Symfony 4 to be installed in Drupal 8 » Allow Symfony 4.4 to be installed in Drupal 8
Issue summary: View changes

Updating title and issue summary.

gábor hojtsy’s picture

gábor hojtsy’s picture

mikelutz’s picture

Still unable to replicate the failure in #235 in my development environment. I set up DrupalCI locally and got it to fail, but I can't see what's going on as easily.

I isolated the test in drupalci, and downgraded each of the symfony components one by one to narrow it down to the dependency-injection component. I bisected that one back and isolated the problem down to a single commit: https://github.com/symfony/symfony/commit/7c01c4c80c69159b2b39ea8bc53431196d7b29fb

Now I need to follow the code path and see what that commit could possibly have done to break one test, and only on drupalci...

mikelutz’s picture

My best guess so far is that previous to this patch calling $container->get('service_container') would return NULL and now it returns $this.

I see this being called in the installer during the attempt to set up the file system.

Maybe there is a difference between the way my local dev environment file system is set up when I run tests through PHPstorm vs the way the file system is setup in drupalCI, causing an error on drupalci that my system can handle somehow. Not sure yet, but I'm done for today, figured I would leave this here in case anybody had any idea.

Edit: I misread the commit on Friday, this isn't correct. I'm still hunting.

mikelutz’s picture

Narrowed it down to a crash inside of an APCU File Cache read, I believe. Here's some relevant tests:

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new178.3 KB
new2.05 KB

Rolling in not using the file cache on profile discovery in the installer to test against the full test suite

catch’s picture

fwiw I'd be fine with disabling
Drupal\FunctionalJavascriptTests\Core\Installer\Form\SelectProfileFormTest and opening a critical issue to re-enable it to unblock the actual Symfony 4 commit. However we should only do that as a last resort in Drupal 9 itself if we're unable to fix it in Drupal 8 before October when 8.9.x is opened (which is the trigger for opening 9.0.x too per #2608062: [META] Requirements for tagging Drupal 9.0.0-alpha1).

mikelutz’s picture

StatusFileSize
new177.31 KB

Rebasing my local branch.

@catch I'm not sure what the 'correct' fix is. It really feels like a php bug, although the same error consistently appears in 7.1, 7.2, and 7.3, and I can't seem to boil it down to easily reproducible steps either. My tests just show the script dying during an apcu_fetch call, which shouldn't happen. Disabling that cache seems to work, but I just don't yet know what the 'correct' fix should be.

mikelutz’s picture

Issue summary: View changes
alexpott’s picture

StatusFileSize
new176.56 KB

Rerolled... as this conflicted with #3078058: Update composer.lock with scaffold metadata from drupal/core composer.json - should still fail.

alexpott’s picture

StatusFileSize
new1.85 KB
new177 KB

Let's see what happens if we revert the change to install_begin_request() and write a proper test settings.php in \Drupal\FunctionalJavascriptTests\Core\Installer\Form\SelectProfileFormTest::setUp(). This test is a bit special in that it is very similar to an InstallerTestBase test but built on WebDriverTestBase and not BrowserTestBase (and so supports JS). The test passes locally.

Status: Needs review » Needs work

The last submitted patch, 250: 2976394-250.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new1.15 KB
new177.55 KB

Here's a way to continue running the test whilst avoiding the error by disabling the use of APCu for this test. I've added a big @todo. One cause for concern is that this is most real test we have of visiting the installer since it is visiting the installer using a real browser. I'm not sure any other test does this.

mikelutz’s picture

As part of my investigation, I went to go check the other installer JavaScript tests, to see if they had similar problems, and was surprised to see there weren't any other JavaScript tests of the installer.

gábor hojtsy’s picture

Directly parenting this to the branch / alpha1 issue as one of the 2 issues left :)

Status: Needs review » Needs work

The last submitted patch, 252: 2976394-252.patch, failed testing. View results

pasqualle’s picture

I am a bit confused about D8.9 release.
Is there a plan to release D8.9 with SF4.4? Or D8 will have only "unofficial" compatibility with SF4?

voleger’s picture

As I understand, the same optional support as we have with Twig2 now.

catch’s picture

@Pasqualle the issue for that is #3020303: Consider using Symfony flex to allow switching between major Symfony versions where there is a fair bit of discussion.

mikelutz’s picture

mikelutz’s picture

Status: Needs work » Needs review
StatusFileSize
new177.86 KB

Time to dust this one back off and get caught up in preparation for the D9 branch. Straight re-roll to start.

Status: Needs review » Needs work
wim leers’s picture

Nice work, @mikelutz! 🙏👍

voleger’s picture

wim leers’s picture

And so did #3085673: [Symfony 4] EntityResourceValidationTraitTest should not get a mock for an internal trait.. I see @catch already triggered a re-test after he committed both. 👍 #260 should come back green! 🤞

catch’s picture

Status: Needs work » Needs review

Nice green test run now :)

mikelutz’s picture

Green yes, but we still need to decide and implement the final solution to #246/#252.

mikelutz’s picture

reroll again

Status: Needs review » Needs work
mikelutz’s picture

Here's a test reverting the change in #252 and adding in the patch from #3074585: [Symfony 5] Replace app.root and site.path string services with container parameters to see if removing those deprecations fixes the apcu issue at all.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

alexpott’s picture

Updating credit to credit review comments.

gábor hojtsy’s picture

Status: Needs work » Postponed (maintainer needs more info)

Given #3088754: Update to Drupal 9 to Symfony 4.4.0 what is left to do here?

mikelutz’s picture

Status: Postponed (maintainer needs more info) » Fixed

As far as I'm concerned, the purpose of this issue was to do everything possible in Drupal 8 to prepare for updating the Drupal 9 brnach to Symfony 4 once it opened. Some of the most recent changes to be compatible with SF4 were not backported to Drupal 8, and they don't need to be. The biggest reason to try to support Drupal 8 on Symfony 4 was for testing purposes, and we now have the D9 branch for that. With the opening of D9, now on Symfony 4.4 stable, I think this issue served its purpose and can be closed.

I'm closing it as fixed to credit all the people that have contributed towards the very successful effort, but feel free to re-open or switch to outdated if you prefer.

mikelutz’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.