Problem/Motivation

Since the transition of Zend Framework to the Laminas project, composer has been warning about the deprecation of the zendframework/* components.

Our composer dependencies should be updated to reflect this.

  • zendframework/zend-diactoros → laminas/laminas-diactoros
  • zendframework/zend-escaper → laminas/laminas-escaper
  • zendframework/zend-feed → laminas/laminas-feed
  • zendframework/zend-stdlib → laminas/laminas-stdlib

Proposed resolution

Replace the Zend components with Laminas components

Remaining tasks

Clarify support status of Zend components and just how abandoned they are.

User interface changes

None

API changes

New Laminas components. The Zend components removed and a BC layer provided by laminas/laminas-zendframework-bridge

Data model changes

None

Release notes snippet

Laminas components replace Zend components. Code should be updated to use the new Laminas classes. A backwards compatibility layer is provided by laminas/laminas-zendframework-bridge. See https://www.drupal.org/node/3104895

CommentFileSizeAuthor
#88 3104015-9.x-88.patch558 bytesalexpott
#88 3104015-8.9.x-88.patch39.54 KBalexpott
#88 87-interdiff.txt1.65 KBalexpott
#81 77-81-interdiff.txt2.48 KBalexpott
#81 3104015-8.9.x-81.patch37.89 KBalexpott
#77 interdiff-77.txt8.32 KBxjm
#77 laminas-3104015-77.patch39.85 KBxjm
#75 laminas-3104015-75.patch33.03 KBxjm
#74 laminas-3104015-74.patch14.34 KBxjm
#64 3104015-8.9.x-64.patch40 KBalexpott
#61 3104015-8.9.x-2-61.patch26.3 KBalexpott
#7 3104015-7.patch24.18 KBlongwave
#14 3104015-9.0.x-14.patch38.3 KBalexpott
#18 3104015-8.9.x-18.patch40 KBalexpott
#38 3104015-9.0.x-38-rebase-of-14.patch37.61 KBTravisCarden
#40 3104015-9.0.x-40.patch13.35 KBTravisCarden
#41 3104015-9.0.x-41.patch49.85 KBTravisCarden
#43 3104015-9.0.x-43.patch51.53 KBSpokje
#45 3104015-9.0.x-45.patch53.14 KBTravisCarden
#52 3104015-9.0.x-52.patch53.1 KBTravisCarden
#55 patches.diff1.14 KBTravisCarden
#55 3104015-55-rebase-of-14.patch38.3 KBTravisCarden
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

fgm created an issue. See original summary.

Bruno Vincent’s picture

Ok, I'm getting the same errors, how can I resolve this?

For this code for example:
zendframework/zend-diactoros → laminas/laminas-diactoros

What would be the exact syntax to input into composer?

fgm’s picture

Until core moves on this, at the project level, what you can do is add the requirements in your root composer.json file. Since the laminas versions have a "replaces" clause targeting their equivalent Zend Framework clauses, you'll get them instead of the legacy zendframework versions. Diactoros needs to be kept to 1.8.x due to core requirements, but the others may use the latest versions. So just do:

composer require laminas/laminas-diactoros:^1.8 laminas/laminas-escaper 
composer require laminas/laminas-feed laminas/laminas-stdlib

And the warnings will disappear.

Note that this won't work for the similar warning about container-interop/container-interop (if you have Drush) because the suggested replacement, psr/container, does not have such a "replaces" clause in its composer.json.

Berdir’s picture

> And the warnings will disappear.

Maybe, but are you sure that the code using this library will actually still work? The new packages use a different namespace.

Berdir’s picture

alexpott’s picture

Version: 8.9.x-dev » 9.0.x-dev
Priority: Normal » Critical

This is a critical task for Drupal 9 as it is sub-optimal to have a Drupal 9 alpha using abandoned packages.

longwave’s picture

Status: Active » Needs review
FileSize
24.18 KB

This will conflict with #3104354: Update Laminas components including Diactoros but best to get this in first I guess.

fgm’s picture

This is the goal of https://github.com/laminas/laminas-zendframework-bridge which is brought as a dependency by these packages.

The package mentions that it needs a composer plugin to be added: indeed when you just add these dependencies, they do replace the ZF ones, but when testing, at least with Drush, it appears to work:

$ drush core-cli
Psy Shell v0.9.12 (PHP 7.2.22 — cli) by Justin Hileman
>>> $df = Zend\Diactoros\ServerRequestFactory::fromGlobals()
=> Laminas\Diactoros\ServerRequest {#4019}
>>>

I haven't examined in detail how it woks, especially considering the fact that the Zend classes no longer appears in the vendor/composer/autoload* files, but it does work at this basic level. Now, of course, if some code checks namespaces manually, things could break, but I haven't noticed it yet.

It would still be nice to have this without having to wait for an API change in D9: I've already seen this question on Slack, and it's likely to multiply over time.

Status: Needs review » Needs work

The last submitted patch, 7: 3104015-7.patch, failed testing. View results

Berdir’s picture

I suppose they are doing more or less dynamic class aliases then or so, didn't see that part.

The test fail is a good example of where that BC doesn't work, the thrown exception doesn't match what we except, so we don't catch it and it fails.

fgm’s picture

Hmm, I guess the error is not about an expectation, but a behavior "DOMDocument cannot parse XML: Extra content at the end of the document"

Berdir’s picture

That tests specifically tests for that, it is called 1) Drupal\Tests\aggregator\Functional\AddFeedTest::testAddLongFeed :)

It is about the try catch there that doesn't work anymore.

alexpott’s picture

This change is also causing a problem with running verbose tests locally - I guess because the autoloader is not providing the BC layer in the test runner.

I'm getting

1) Drupal\Tests\aggregator\Functional\AddFeedTest::testAddFeed
Exception: Warning: require(/Users/alex/dev/sites/drupal8alt.dev/vendor/composer/../zendframework/zend-diactoros/src/ServerRequestFactory.php): failed to open stream: No such file or directory
require()() (Line: 112)
alexpott’s picture

Status: Needs work » Needs review
FileSize
38.3 KB

Here's an approach that fixes #7. I've replaced all the usages of Zend classes in core with Laminas classes. This is a bit more forward compatible. The BC layer for the autoloader is still there. RE #13 this was happening because the autoloader was using APC and it hadn't been cleared.

james.williams’s picture

Will this be a candidate for back-porting to D8 once the D9 fix has been completed?

alexpott’s picture

@james.williams I think so because of the BC layer implemented by the laminas/laminas-zendframework-bridge - but we need a release manager to +1 this.

alexpott’s picture

Actually I think we need to backport this to 8.9.0 because otherwise code for 8.9 and 9.0 won't be able to be the same.

alexpott’s picture

Here's a patch for 8.9

alexpott’s picture

Issue summary: View changes
Issue tags: -Needs change record

Added the change record - https://www.drupal.org/node/3104895

catch’s picture

  1. +++ b/core/core.services.yml
    @@ -1430,78 +1430,78 @@ services:
    +# Laminas Feed reader plugins. Plugin instances should not be shared.
       feed.reader.dublincoreentry:
    -    class: Zend\Feed\Reader\Extension\DublinCore\Entry
    +    class: Laminas\Feed\Reader\Extension\DublinCore\Entry
         shared: false
    

    We need to deprecate/remove all of these per #2979588: Deprecate Laminas\Feed reader and writer services.

    Since they're unused, I think we could probably get away with doing that in 8.9/9.0 still. Doesn't need to happen here, just bringing it up because it would have been a smaller patch if that issue had landed first.

  2. +++ b/core/lib/Drupal/Component/Bridge/ZfExtensionManagerSfContainer.php
    @@ -5,26 +5,27 @@
     use Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException;
    -use Zend\Feed\Reader\ExtensionManagerInterface as ReaderManagerInterface;
    -use Zend\Feed\Writer\ExtensionManagerInterface as WriterManagerInterface;
    +use Laminas\Feed\Reader\ExtensionManagerInterface as ReaderManagerInterface;
    +use Laminas\Feed\Writer\ExtensionManagerInterface as WriterManagerInterface;
     
    

    So we're doing this because the bc layer isn't kicking in when we're running tests, is that definitely specific to tests?

alexpott’s picture

#20.1 @catch so you think we can still remove these services in Drupal 9?

#20.2 @catch the BC layer is present in tests it just is insufficient in some cases - see #10. We're doing that to not depend on the BC layer - to make upgrading to version 4 simpler when it comes (I'm assuming version 3 still has the BC layer because it existed before the Laminas release).

alexpott’s picture

Issue summary: View changes

I've reached out on the Laminas slack to clarify the support status for the current Zend components now that Laminas has packages and the Zend packages are marked as abandoned. If the Zend components will not receive any updates that potentially we have to backport this change to 8.8.x too and ask ourselves about 8.7.x as that still has security support.

TravisCarden’s picture

@alexpott, is there anything we can do in the meantime, or do we just have to wait for word from Laminas on the matter?

alexpott’s picture

@TravisCarden well we can review and get this committed to Drupal 9 because I think regardless of any answer from Laminas as to security support we're going to need to do this.

catch’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +9.0.0 release notes

OK I'm happy with #14 for Drupal 9. It's a straightforward update for us, and there's the bc layer (courtesy of Zend/Laminas) for contrib which should hopefully be enough for the feeds module to be compatible with both 8 and 9.

I am not sure either way about backporting this to Drupal 8 still but that's something we can decide after it's been committed to 9.x, so marking RTBC for that.

longwave’s picture

If 8.9.x is going to be LTS, what happens if we don't backport this but there is a security issue next year in one of these components? (which presumably won't be fixed in the ZendFramework versions?)

catch’s picture

@longwave we have three choices I think - this is assuming Zend definitely doesn't do 18 months of support on the old Zend components.

1. Backport this to 8.9 (and/or 8.8) - risks some disruption in the 8.8-8.9 upgrade path but ensures we can apply security fixes if/when they come up.

2. Backport this to 8.9 only if and when there's a security release on Laminas. Means potentially never having to do this, but makes the security release potentially disruptive if we do.

3. If there's a security release on Laminas, and only if it affects us, fork the Zend library and backport it ourselves.

All three have their own benefits and drawbacks.

fgm’s picture

It seems the first choice has the smallest set of inconvenients: these days most Drupal devs are working on upgrading regularly 8.8 with an eye on 9.0, so the possible disruption on the upgrade path will have more eyes/hands on it than if the change has to be done after the actual active devs have moved on to the next 9.x release as would be the case in choice 2 or 3.

xjm’s picture

Issue tags: +Needs dependency evaluation

Any thoughts about whether we should land this first or #3104354: Update Laminas components including Diactoros? The disadvantage of doing that before the backport is that the patches diverge a bit more, but that's already going to be happening anyway.

We'll also need to update the dependency evaluation information for the new packages, including their security coverage policies if they differ from ZF's LTS policy (especially for Diactoros):
https://www.drupal.org/core/dependencies#criteria

Let's put a dependency evaluation in the issue summary. I know discussion of their security and release policies is underway but let's get it spelled out in the IS. Thanks!

xjm’s picture

I have mixed feelings about backporting this to 8.9.x. If the BC layer is imperfect, that means new BC breaks from 8.8, which we don't want. And we do tell people that one of the things that happens in the major branch is a dependency update to a new version. (Pity we didn't know about all this before the 8.8 deadline.)

On the other hand I agree that we want 8.9 and 9.0 to be as similar as possible.

xjm’s picture

Status: Reviewed & tested by the community » Needs work

NW for the dependency evaluation.

xjm’s picture

+++ b/core/modules/system/tests/modules/router_test_directory/src/TestControllers.php
@@ -8,7 +8,7 @@
-use Zend\Diactoros\Response\HtmlResponse;
+use Laminas\Diactoros\Response\HtmlResponse;

Will contrib also have to change all their use statements like this if we backport it to 8.9? Def a BC break even if it's simple to fix.

alexpott’s picture

Re #32 - nope they won't have to change all the use statements. The BC layer seems to have one issue as per #7 but by in-large it appears to work. We should investigate why #7 failed and see if it is important and thry to fix upstream if it is. Also we should try to get the BC layer to issue silenced deprecations as this will help the the Drupal and rest of PHP ecosystem upgrade.

catch’s picture

Status: Needs work » Needs review

Laminas' security policy is here: https://getlaminas.org/security/

I couldn't find documentation for LTS releases or anything else. Given they're immediately dropping support for Zend Framework (unless that changes) I'd be inclined not to believe anything about LTS anyway.

andypost’s picture

ThirstySix’s picture

#18 is working fine for 8.8 version.

TravisCarden’s picture

Has anyone seen the Zend Framework to Laminas Migration Guide? They provide a migration script. Was that used to help generate any of the patches here? I might give it a try.

TravisCarden’s picture

The Laminas migration script yielded no changes over the patch in #14 other than the addition of an unnecessary dependency (a Composer plugin that intercepts new composer require calls for Zend Framework packages and rewrites them to Laminas equivalents). Here's a reroll of #14.

Status: Needs review » Needs work

The last submitted patch, 38: 3104015-9.0.x-38-rebase-of-14.patch, failed testing. View results

TravisCarden’s picture

Status: Needs work » Needs review
FileSize
13.35 KB

The previous patch failed due to symfony/psr-http-message-bridge using the Diactoros library we're removing, so I updated to the latest version of that library, which removes the dependency on it. So the updated library diff looks like this:

$ composer-lock-diff --no-links
+--------------------------------------+--------+---------+
| Production Changes                   | From   | To      |
+--------------------------------------+--------+---------+
| symfony/psr-http-message-bridge      | v1.3.0 | v2.0.0  |
| zendframework/zend-diactoros         | 1.8.7  | REMOVED |
| zendframework/zend-escaper           | 2.6.1  | REMOVED |
| zendframework/zend-feed              | 2.12.0 | REMOVED |
| zendframework/zend-stdlib            | 3.2.1  | REMOVED |
| laminas/laminas-diactoros            | NEW    | 1.8.7   |
| laminas/laminas-escaper              | NEW    | 2.6.1   |
| laminas/laminas-feed                 | NEW    | 2.12.0  |
| laminas/laminas-stdlib               | NEW    | 3.2.1   |
| laminas/laminas-zendframework-bridge | NEW    | 1.0.1   |
| nyholm/psr7                          | NEW    | 1.2.1   |
| php-http/message-factory             | NEW    | v1.0.2  |
| psr/http-factory                     | NEW    | 1.0.1   |
+--------------------------------------+--------+---------+

Note: nyholm/psr7 was suggested by the new version of symfony/psr-http-message-bridge to replace the missing DiactorosFactory that caused the previous patch to fail. The addition of php-http/message-factory and psr/http-factory was already discussed at #3104354-9: Update Laminas components including Diactoros and documented on the core dependency page.

TravisCarden’s picture

Status: Needs review » Needs work

The last submitted patch, 41: 3104015-9.0.x-41.patch, failed testing. View results

Spokje’s picture

catch’s picture

Issue tags: -Needs dependency evaluation

I've added an entry for Laminas on https://www.drupal.org/core/dependencies - just above the Zend entries for now. I think this is as close to a proper dependency evaluation as we can get while they're migrating. The important thing here is they're completely retiring Zend/* while transferring all the code and issues wholesale to Laminas, so while things might be different with Laminas if they also change the project structure alongside the rename, Zend/* just does not exist and is unsupported at this point.

TravisCarden’s picture

Title: Replace ZendFramework/* dependencies by their Laminas equivalents » Replace ZendFramework/* dependencies with their Laminas equivalents
Status: Needs work » Needs review
FileSize
53.14 KB

Fixed test failures.

longwave’s picture

+++ b/composer.lock
@@ -492,16 +495,14 @@
+                "symfony/psr-http-message-bridge": "^2.0.0",

Why are we making this change here, isn't this out of scope? I thought the Zend and Laminas versions were supposed to be identical except for the compatibility bridge?

TravisCarden’s picture

@longwave, it's because symfony/psr-http-message-bridge:v1.3.0 (unfortunately) has an implicit dependency on zendframework:

$ ag -Q 'use Zend'
Factory/DiactorosFactory.php
23:use Zend\Diactoros\Response as DiactorosResponse;
24:use Zend\Diactoros\ServerRequest;
25:use Zend\Diactoros\ServerRequestFactory as DiactorosRequestFactory;
26:use Zend\Diactoros\Stream as DiactorosStream;
27:use Zend\Diactoros\UploadedFile as DiactorosUploadedFile;

The v2.0.0 release changes literally nothing from v1.3.0 but removing that dependency (changelog), so aside from the constraint change, there should be no impact.

xjm’s picture

+++ b/core/tests/Drupal/BuildTests/QuickStart/QuickStartTestBase.php
@@ -36,6 +36,7 @@ abstract class QuickStartTestBase extends BuildTestBase {
   public function installQuickStart($profile, $working_dir = NULL) {
     $php_finder = new PhpExecutableFinder();
     $install_process = $this->executeCommand($php_finder->find() . ' ./core/scripts/drupal install ' . $profile, $working_dir);
+    $this->assertErrorOutputContains('Congratulations, you installed Drupal!');
     $this->assertCommandOutputContains('Username:');

This is presumably a good assertion to add, but how did it end up in the scope of this patch?

longwave’s picture

> @longwave, it's because symfony/psr-http-message-bridge:v1.3.0 (unfortunately) has an implicit dependency on zendframework:

I thought this is what the compatibility bridge was designed to handle, so we shouldn't strictly need to upgrade here. #14 seemed to pass without this which is why I am confused.

Spokje’s picture

@longwave So it looks like symfony/psr-http-message-bridge:v1.3.0 can do the job (#14), but it would still leave us with an implicit dependency on zendframework.

Since this issue is about getting rid of dependencies on zendframework, it makes sense to me to bump symfony/psr-http-message-bridge to ^2.0.0 so we indeed get rid of all dependencies to zendframework?

longwave’s picture

OK, thanks for the clarification, I understand now.

  1. +++ b/core/tests/Drupal/Tests/Core/Controller/ControllerResolverTest.php
    @@ -14,13 +14,16 @@
    +use Nyholm\Psr7\ServerRequest;
    +use Nyholm\Psr7\Uri;
    

    Unused use statements.

  2. +++ b/core/tests/Drupal/Tests/Core/Controller/ControllerResolverTest.php
    @@ -43,24 +46,18 @@ class ControllerResolverTest extends UnitTestCase {
    +    $server_request_factory = new Psr17Factory();
    +    $http_message_factory = new PsrHttpFactory($server_request_factory, new Psr17Factory(), new Psr17Factory(), new Psr17Factory());
    

    This looks a bit odd, why have the variable for only one argument? https://github.com/symfony/psr-http-message-bridge/blob/master/Tests/Fac... uses the same object for all four parameters so should we do that here?

TravisCarden’s picture

FileSize
53.1 KB
  1. Re @xjm in #48: Ah, of course. I added that assertion because that test failed due to the afore-mentioned implicit dependency symfony/psr-http-message-bridge:v1.3.0 has on zendframework but provided no useful information about why. (Just something like "String '' does not contain 'Username:'".) When I added that assertion the failure output showed the exact problem, leading me to the fix.
  2. Re @longwave in #51: Good catch on both counts, thanks. Fixed.
longwave’s picture

This looks RTBC as far as the code is concerned, but do we need a dependency evaluation for nyholm/psr7?

edit: just spotted this:

+++ b/core/core.services.yml
@@ -778,10 +778,13 @@ services:
+    arguments: [ '@psr17.http_factory', '@psr17.http_factory', '@psr17.http_factory', '@psr17.http_factory', ]

We aren't 100% consistent in core.services.yml but there are extra spaces and a trailing comma here.

alexpott’s picture

So I think as #18 shows we don't need to update symfony/psr-http-message-bridge - the laminas components do a composer replace on the zend packages so the dependency is actually met.

I think we should go back to #18 here because that's what we need to 8.x and 9.x and then open yet another update all the dependencies issue for Drupal 9 to update all the dependencies including the psr-http-message-bridge and the laminas components too.

TravisCarden’s picture

FileSize
38.3 KB
1.14 KB

I hope that's the case, @alexpott, but there may be a catch. Tell me if I'm missing something.

The patch in #18 is against 8.9.x, and it still applies cleanly. I assume the tests will still pass. So if that's all you're talking about, I'm on board.

But I don't think the corresponding change against 9.0.x fares the same. The patch that passed tests in #14 no longer applies. That's why I rerolled it in #38. Now looking back at it, I realize I made a mistake in that patch. But I've rerolled #14 again and verified the correctness of it, and I get the same test failure. Here's the patch along with a diff showing that the only difference between my reroll and your original in #14 is context (including the Guzzle version upgrade from #3104473). Let's see what happens!

TravisCarden’s picture

Well... I stand corrected. ¯\_(ツ)_/¯

longwave’s picture

Status: Needs review » Reviewed & tested by the community

#18 and #55 look RTBC to me.

Opened #3107396: Upgrade symfony/psr-http-message-bridge to v2.0.0 to finish the rest of the upgrade discussed above.

  • catch committed dc86e91 on 9.0.x
    Issue #3104015 by TravisCarden, alexpott, longwave, Spokje, fgm, catch,...
catch’s picture

Status: Reviewed & tested by the community » Patch (to be ported)
Issue tags: +Needs release manager review

Committed dc86e91 and pushed to 9.0.x. Thanks!

Not yet committing to 8.9.x/8.8.x and still not sure about the best course of action there.

alexpott’s picture

Version: 9.0.x-dev » 8.9.x-dev

Note there is new version of laminas/laminas-zendframework-bridge - 1.0.1 so the Drupal 8.x patch should use that. For Drupal 9 this is being fixed in #3104354: Update Laminas components including Diactoros as that is updating laminas components there.

alexpott’s picture

Status: Patch (to be ported) » Needs review
FileSize
26.3 KB

I've updated the laminas/laminas-zendframework-bridge also the fail #7 doesn't happen. However it didn't happen locally for me even on 1.0.0 so /shrug.

Here's a patch for 8.9.x that uses the latest laminas/laminas-zendframework-bridge and doesn't change all the classes thereby relying on the BC layer.

Status: Needs review » Needs work

The last submitted patch, 61: 3104015-8.9.x-2-61.patch, failed testing. View results

alexpott’s picture

Hmmm... I see. This happens for me on PHP 7.2 and lower but not on PHP 7.3. Interesting. So the BC layer works most of the time but a PHP bug means it doesn't work on PHP 7.2 and lower :(

alexpott’s picture

Add class_exists(ExceptionInterface::class); to \Drupal\aggregator\Plugin\aggregator\parser\DefaultParser::parse() results in a green test on PHP 7.2. But I think we should go for #18 on Drupal 8.x because if we don't then code navigation is broken in IDEs because they don't know anything about the magic in the Laminas autoloader.

So here's a patch that updates #18 to use the latest bridge.

alexpott’s picture

So it's not PHP version :) I've finally been able to trigger the error locally at will. If I have apcu enabled I get the bug. If I remove apcu then I get the bug. So the Zend BC layer is fine - it's our very strange messing about with the autoloader in \Drupal\Core\DrupalKernel::initializeSettings() that is causing the problem.

It is fixable if we do something like...

diff --git a/core/lib/Drupal/Core/DrupalKernel.php b/core/lib/Drupal/Core/DrupalKernel.php
index b9f7b30613..fa195b8711 100644
--- a/core/lib/Drupal/Core/DrupalKernel.php
+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -1086,7 +1086,11 @@ protected function initializeSettings(Request $request) {
         $loader = new XcacheClassLoader($prefix, $this->classLoader);
       }
       if (!empty($loader)) {
-        $this->classLoader->unregister();
+        // Remove all the autoloaders.
+        foreach(spl_autoload_functions() as $function) {
+          spl_autoload_unregister($function);
+        }
+        // $this->classLoader->unregister();
         // The optimized classloader might be persistent and store cache misses.
         // For example, once a cache miss is stored in APCu clearing it on a
         // specific web-head will not clear any other web-heads. Therefore
@@ -1098,6 +1102,8 @@ protected function initializeSettings(Request $request) {
         // class loader they are replacing.
         $old_loader->register(TRUE);
         $loader->register(TRUE);
+        // Register the BC layer.
+        \Laminas\ZendFrameworkBridge\Autoloader::load();
       }
     }
   }

But this option looks super scary of someone has done any other autoloader futzing.

TravisCarden’s picture

Secondary to the main point (which I'll look into), does #65 indicate a gap in our test coverage? If something passes without a very common component of production environments (apcu) but fails with it, it seems like we have a defect vector unless the testbot tests both with and without it.

DamienMcKenna’s picture

Tagging as a requirement for Drupal 9.0-beta1.

alexpott’s picture

@DamienMcKenna this has made it into Drupal 9 already. The big question is what do we do about Drupal 8.

xjm’s picture

We should still keep the tag for posterity.

catch’s picture

Just to note I really have no idea what to do here:

Either we:

1. Do this in 8.9.x, risk some disruption, make a potential Laminas security update easier during the 8.9.x support cycle.

or

2. Don't do this, but have to fork-for-backport or make the switch if Laminas makes a security release

One introduces disruption for potentially no gain given the end of life of 8.9.x is December 2021.

One has a higher risk of a disruptive security release some time when 8.9.x is minimally maintained, but it's only disruption that we'd be deferring from here.

Also by not committing this to 8.8.x, we still have a nearly one-year window where we could still have to figure out a security backport process to 8.8.x, so committing it to 8.9.x only actually cuts the window-of-annoyance by six months.

xjm’s picture

Issue tags: +beta deadline

We're also running out of time to do it in Drupal 8 at all at this point.

tedbow’s picture

Given the problem posed by @catch in #70 thought it might be useful to look at past ZendFramework security releases since 2016, longer than we would have to worry about but also I didn't want to go through all the releases since 2009

Sorry, probably to framework managers this stuff is already known but I wasn't sure about the frequency and how many affect use. Hopefully this will be useful to others.

  1. ZF2019-01: Information disclosure in zend-developer-tools
    Released 2019-03-28
    Affects zendframework/zend-developer-tools
    Doesn't look like we use this?
  2. ZF2018-01: URL Rewrite vulnerability
    Released 2018-08-01
    We were affected https://www.drupal.org/SA-CORE-2018-005

    I tried to find the actual fix but the closest I found was something @alexpott made to attempt provide a backport to an LTS version https://github.com/alexpott/zend-diactoros/commit/67f2894cdf645ee3b1cbef...

    But seems like the original fix would have been similar

  3. ZF2016-04: Potential remote code execution in zend-mail via Sendmail adapter
    Released 2016-12-20
    Affects Zend-mail
    Doesn't look like we use this?
  4. ZF2016-03: Potential SQL injection in ORDER and GROUP functions of ZF1
    Released 2016-09-08
    Doesn't look like we were affected
    This was an improvement over the next issue
  5. ZF2016-02: Potential SQL injection in ORDER and GROUP statements of Zend_Db_Select
    Released 2016-07-13
  6. ZF2016-01: Potential Insufficient Entropy Vulnerability in ZF1
    Released 2016-04-13
    Doesn't look like we were affected

In total there have been 44 security releases for ZendFramework since 2009.
Thats 4~5 a year
Looking at recent years
2019: 1
2018: 1
2016: 4 (though 1 was an improvement over another in the year)
2015: 11
2014: 6
2013: 3

Looks like there have been no security advisories under the Laminas Project yet https://getlaminas.org/security/

tedbow’s picture

Retesting #64. I don't think it applies anymore

I looked into reroll this but I am not clear how to do this. I couldn't clear directions here. Can someone update the summary regarding that as this will probably need to be rerolled with any changes to composer.lock.

Or is there another issue I can look at that did something similar?

xjm’s picture

The frequency of security advisories isn't the issue, so much, and not really a predictor of what will happen in the future. All it takes is one security issue with one of the components we do use that we can't update. The closer we are to supported packages, the easier it will be for us to respond to the security issue.

I rerolled this from #64 by removing the hunk that changes composer.lock, applying the remaining patch, and then running composer update --lock.

xjm’s picture

Errr #74 didn't actually add the lockfile back; here's the correct patch.

xjm’s picture

🤔 The output from the fail in #75:

--- Errors ---
> Drupal\Composer\Composer::ensureComposerVersion
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 1 install, 5 updates, 0 removals
  - Removing drupal/core-project-message (8.9.x-dev), source is still present in /var/www/html/vendor/drupal/core-project-message
As there is no 'unzip' command installed zip files are being unpacked using the PHP zip extension.
This may cause invalid reports of corrupted archives. Besides, any UNIX permissions (e.g. executable) defined in the archives will be lost.
Installing 'unzip' may remediate them.
  - Installing drupal/core-project-message (8.9.x-dev ec887a5): Downloading    Failed to download drupal/core-project-message from dist: Could not authenticate against github.com
    Now trying to download from source
  - Installing drupal/core-project-message (8.9.x-dev ec887a5): Cloning ec887a5cd5
  - Removing drupal/core-vendor-hardening (8.9.x-dev), source is still present in /var/www/html/vendor/drupal/core-vendor-hardening
  - Installing drupal/core-vendor-hardening (8.9.x-dev 9c43a12): Downloading    Failed to download drupal/core-vendor-hardening from dist: Could not authenticate against github.com
    Now trying to download from source
  - Installing drupal/core-vendor-hardening (8.9.x-dev 9c43a12): Cloning 9c43a126bd
  - Installing laminas/laminas-zendframework-bridge (1.0.1): Downloading (100%)
  - Updating laminas/laminas-stdlib (3.2.1 => 3.2.1): Downloading (100%)
  - Updating laminas/laminas-escaper (2.6.1 => 2.6.1): Downloading
                                             
  [Composer\Downloader\TransportException]   
  Could not authenticate against github.com  

So it looks like the problem is with github and the metapackages, not the Laminas ones? composer install works fine locally.

xjm’s picture

xjm’s picture

Yep, that got tests to run.

xjm’s picture

+++ b/composer/Metapackage/CoreRecommended/composer.json
@@ -17,6 +17,8 @@
         "doctrine/lexer": "1.0.2",
+        "drupal/core-project-message": "8.9.x-dev",
+        "drupal/core-vendor-hardening": "8.9.x-dev",
         "easyrdf/easyrdf": "0.9.1",

No idea why the composer update drupal/core* is also adding this weird hunk to the metapackage?

xjm’s picture

#77 has a pass once and a "Core composer reinstall failure" the second time for the same patch on the same environment.

alexpott’s picture

Here's a patch without the drupal/core-project-message and drupal/core-vendor-hardening changes. I think composer gets a bit confused sometimes when it has to deal with existing dependencies, new dependencies and our fake path repositories like these libraries. The only way I've found of reliably making these type of changes is to remove vendor before you start, do composer install, use composer require, move stuff to core/composer.json, run composer update drupal/core - and liberal use of the COMPOSER_ROOT_VERSION=8.9.x-dev env var.

selinav’s picture

I have the problem with the 8.8.4. Can I test the patch for this version ?

longwave’s picture

Status: Needs review » Needs work

After applying #81 to 8.9.x there are two remaining references to Zend packages that I think need to be updated:

core/lib/Drupal/Component/Bridge/composer.json:    "zendframework/zend-feed": "^2.12"
core/modules/layout_builder/tests/src/FunctionalJavascript/ContentPreviewToggleTest.php:use Zend\Stdlib\ArrayUtils;

Other than this I don't see anything else that needs changing and so if we are to make this change in 8.9.x this is RTBC.

jungle’s picture

As this issue is NW now, is it possible to get #3121885: Update coder to 8.3.8 reviewed and committed early than this one?

xjm’s picture

Re: #84, this one is still the priority; it is critical to get it into 8.9 prior to the beta deadline which is... well technically it was 14h ago, but we have a little flexibility. :)

Coder is a dev dependency so we can make that change during beta as needed.

Thanks!

xjm’s picture

Oh, @selinav, regarding:

I have the problem with the 8.8.4. Can I test the patch for this version ?

I assume you mean that you're getting messages that the Zend packages are abandoned and should be replaced with Laminas when you do a composer install or update? The current patch for this issue is for 8.9 so it won't work on 8.8, but once this is committed to 8.9 we are are discussing doing an 8.8 backport immediately after the next patch release. (It's a disruptive change normally not allowed in patch releases, but we understand this is adding a lot of noise for composer sites.)

Hope that helps answer your question! In the meantime, you can ignore those warnings; they are just letting you know that the Zend versions of the packages won't have future releases, which is why core is converting them to the Laminas versions in at least 8.9 and potentially 8.8. But the current versions do work fine for now.

xjm’s picture

Mmmm, there's also this (in 9.0.x and 8.9.x):

composer/Plugin/VendorHardening/Config.php:    'zendframework/zend-escaper' => ['doc'],
composer/Plugin/VendorHardening/Config.php:    'zendframework/zend-feed' => ['doc'],
composer/Plugin/VendorHardening/Config.php:    'zendframework/zend-stdlib' => ['doc'],

Which could explain why I was getting the weird dependency in #77? Is retaining the references to ZF instead of Laminas there intended?

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.65 KB
39.54 KB
558 bytes

#87 is a good spot. There's no impact in having this here because those directories don't exist. I've checked the new Laminas deps and they no longer include a docs directory and they still don't have a tests directory. So the right thing here is to remove this from both 8.9.x and 9.x - we missed this on the 9.x patch.

8.9.x patch also addresses #83

longwave’s picture

Status: Needs review » Reviewed & tested by the community

Both patches in #88 look good to me.

catch’s picture

  • catch committed ab0f87d on 9.1.x
    Issue #3104015 by alexpott, TravisCarden, xjm, longwave, Spokje, catch,...

  • catch committed f5bc1a0 on 9.0.x
    Issue #3104015 by alexpott, TravisCarden, xjm, longwave, Spokje, catch,...

  • catch committed 1b68fb8 on 8.9.x
    Issue #3104015 by alexpott, TravisCarden, xjm, longwave, Spokje, catch,...
catch’s picture

Version: 8.9.x-dev » 8.8.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed/pushed the clean-up to 9.1.x and 9.0.x, and also the 8.9.x patch.

Moving back to 8.8.x now. Discussed this with xjm yesterday and we think it might be better to commit this just after the next 8.8 patch release.

catch’s picture

Version: 8.8.x-dev » 8.9.x-dev
Status: Patch (to be ported) » Fixed

Discussed this again with @xjm this morning. We are both a bit borderline, but leaning towards not committing this to 8.8.x. This leaves a window where we might be forced to do a backport for a Zend/Laminas security release (until December this year), but also means not adding a potentially disruptive change to a patch release for something that might not happen at all.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

xjm’s picture

pritam.tiwari’s picture

I did face this issue during site upgrade with drush from 8.7.x to 8.9.0 druing the database upgrade. In my case drush is working fine post upgrade.

snsblvd’s picture

I do have the same error (with the latest drush 8.3.5) and Update Core from 8.7.9 to 8.9.0. The site seems to working fine after the upgrade (with the error). Can this error be ignored or can this be fixed somehow (without migrating to composer)?