Problem/Motivation
Since the transition of Zend Framework to the Laminas project, composer has been warning about the deprecation of the zendframework/* components.
Our composer dependencies should be updated to reflect this.
- zendframework/zend-diactoros → laminas/laminas-diactoros
- zendframework/zend-escaper → laminas/laminas-escaper
- zendframework/zend-feed → laminas/laminas-feed
- zendframework/zend-stdlib → laminas/laminas-stdlib
Proposed resolution
Replace the Zend components with Laminas components
Remaining tasks
Clarify support status of Zend components and just how abandoned they are.
User interface changes
None
API changes
New Laminas components. The Zend components removed and a BC layer provided by laminas/laminas-zendframework-bridge
Data model changes
None
Release notes snippet
Laminas components replace Zend components. Code should be updated to use the new Laminas classes. A backwards compatibility layer is provided by laminas/laminas-zendframework-bridge
. See https://www.drupal.org/node/3104895
Comment | File | Size | Author |
---|---|---|---|
#88 | 3104015-9.x-88.patch | 558 bytes | alexpott |
#88 | 3104015-8.9.x-88.patch | 39.54 KB | alexpott |
#88 | 87-interdiff.txt | 1.65 KB | alexpott |
#81 | 77-81-interdiff.txt | 2.48 KB | alexpott |
#81 | 3104015-8.9.x-81.patch | 37.89 KB | alexpott |
Comments
Comment #2
Bruno Vincent CreditAttribution: Bruno Vincent commentedOk, I'm getting the same errors, how can I resolve this?
For this code for example:
zendframework/zend-diactoros → laminas/laminas-diactoros
What would be the exact syntax to input into composer?
Comment #3
fgmUntil core moves on this, at the project level, what you can do is add the requirements in your root
composer.json
file. Since the laminas versions have a"replaces"
clause targeting their equivalent Zend Framework clauses, you'll get them instead of the legacyzendframework
versions. Diactoros needs to be kept to 1.8.x due to core requirements, but the others may use the latest versions. So just do:And the warnings will disappear.
Note that this won't work for the similar warning about
container-interop/container-interop
(if you have Drush) because the suggested replacement,psr/container
, does not have such a"replaces"
clause in itscomposer.json
.Comment #4
Berdir> And the warnings will disappear.
Maybe, but are you sure that the code using this library will actually still work? The new packages use a different namespace.
Comment #5
BerdirI guess we're more likely to do #3043471: Replace the DiactorosFactory message factory in symfony/psr-http-message-bridge with a PSR-17 compliant message factory instead, in D9?
Comment #6
alexpottThis is a critical task for Drupal 9 as it is sub-optimal to have a Drupal 9 alpha using abandoned packages.
Comment #7
longwaveThis will conflict with #3104354: Update Laminas components including Diactoros but best to get this in first I guess.
Comment #8
fgmThis is the goal of https://github.com/laminas/laminas-zendframework-bridge which is brought as a dependency by these packages.
The package mentions that it needs a composer plugin to be added: indeed when you just add these dependencies, they do replace the ZF ones, but when testing, at least with Drush, it appears to work:
I haven't examined in detail how it woks, especially considering the fact that the Zend classes no longer appears in the
vendor/composer/autoload*
files, but it does work at this basic level. Now, of course, if some code checks namespaces manually, things could break, but I haven't noticed it yet.It would still be nice to have this without having to wait for an API change in D9: I've already seen this question on Slack, and it's likely to multiply over time.
Comment #10
BerdirI suppose they are doing more or less dynamic class aliases then or so, didn't see that part.
The test fail is a good example of where that BC doesn't work, the thrown exception doesn't match what we except, so we don't catch it and it fails.
Comment #11
fgmHmm, I guess the error is not about an expectation, but a behavior "DOMDocument cannot parse XML: Extra content at the end of the document"
Comment #12
BerdirThat tests specifically tests for that, it is called 1) Drupal\Tests\aggregator\Functional\AddFeedTest::testAddLongFeed :)
It is about the try catch there that doesn't work anymore.
Comment #13
alexpottThis change is also causing a problem with running verbose tests locally - I guess because the autoloader is not providing the BC layer in the test runner.
I'm getting
Comment #14
alexpottHere's an approach that fixes #7. I've replaced all the usages of Zend classes in core with Laminas classes. This is a bit more forward compatible. The BC layer for the autoloader is still there. RE #13 this was happening because the autoloader was using APC and it hadn't been cleared.
Comment #15
james.williams CreditAttribution: james.williams at ComputerMinds commentedWill this be a candidate for back-porting to D8 once the D9 fix has been completed?
Comment #16
alexpott@james.williams I think so because of the BC layer implemented by the laminas/laminas-zendframework-bridge - but we need a release manager to +1 this.
Comment #17
alexpottActually I think we need to backport this to 8.9.0 because otherwise code for 8.9 and 9.0 won't be able to be the same.
Comment #18
alexpottHere's a patch for 8.9
Comment #19
alexpottAdded the change record - https://www.drupal.org/node/3104895
Comment #20
catchWe need to deprecate/remove all of these per #2979588: Deprecate Laminas\Feed reader and writer services.
Since they're unused, I think we could probably get away with doing that in 8.9/9.0 still. Doesn't need to happen here, just bringing it up because it would have been a smaller patch if that issue had landed first.
So we're doing this because the bc layer isn't kicking in when we're running tests, is that definitely specific to tests?
Comment #21
alexpott#20.1 @catch so you think we can still remove these services in Drupal 9?
#20.2 @catch the BC layer is present in tests it just is insufficient in some cases - see #10. We're doing that to not depend on the BC layer - to make upgrading to version 4 simpler when it comes (I'm assuming version 3 still has the BC layer because it existed before the Laminas release).
Comment #22
alexpottI've reached out on the Laminas slack to clarify the support status for the current Zend components now that Laminas has packages and the Zend packages are marked as abandoned. If the Zend components will not receive any updates that potentially we have to backport this change to 8.8.x too and ask ourselves about 8.7.x as that still has security support.
Comment #23
TravisCarden CreditAttribution: TravisCarden at Acquia commented@alexpott, is there anything we can do in the meantime, or do we just have to wait for word from Laminas on the matter?
Comment #24
alexpott@TravisCarden well we can review and get this committed to Drupal 9 because I think regardless of any answer from Laminas as to security support we're going to need to do this.
Comment #25
catchOK I'm happy with #14 for Drupal 9. It's a straightforward update for us, and there's the bc layer (courtesy of Zend/Laminas) for contrib which should hopefully be enough for the feeds module to be compatible with both 8 and 9.
I am not sure either way about backporting this to Drupal 8 still but that's something we can decide after it's been committed to 9.x, so marking RTBC for that.
Comment #26
longwaveIf 8.9.x is going to be LTS, what happens if we don't backport this but there is a security issue next year in one of these components? (which presumably won't be fixed in the ZendFramework versions?)
Comment #27
catch@longwave we have three choices I think - this is assuming Zend definitely doesn't do 18 months of support on the old Zend components.
1. Backport this to 8.9 (and/or 8.8) - risks some disruption in the 8.8-8.9 upgrade path but ensures we can apply security fixes if/when they come up.
2. Backport this to 8.9 only if and when there's a security release on Laminas. Means potentially never having to do this, but makes the security release potentially disruptive if we do.
3. If there's a security release on Laminas, and only if it affects us, fork the Zend library and backport it ourselves.
All three have their own benefits and drawbacks.
Comment #28
fgmIt seems the first choice has the smallest set of inconvenients: these days most Drupal devs are working on upgrading regularly 8.8 with an eye on 9.0, so the possible disruption on the upgrade path will have more eyes/hands on it than if the change has to be done after the actual active devs have moved on to the next 9.x release as would be the case in choice 2 or 3.
Comment #29
xjmAny thoughts about whether we should land this first or #3104354: Update Laminas components including Diactoros? The disadvantage of doing that before the backport is that the patches diverge a bit more, but that's already going to be happening anyway.
We'll also need to update the dependency evaluation information for the new packages, including their security coverage policies if they differ from ZF's LTS policy (especially for Diactoros):
https://www.drupal.org/core/dependencies#criteria
Let's put a dependency evaluation in the issue summary. I know discussion of their security and release policies is underway but let's get it spelled out in the IS. Thanks!
Comment #30
xjmI have mixed feelings about backporting this to 8.9.x. If the BC layer is imperfect, that means new BC breaks from 8.8, which we don't want. And we do tell people that one of the things that happens in the major branch is a dependency update to a new version. (Pity we didn't know about all this before the 8.8 deadline.)
On the other hand I agree that we want 8.9 and 9.0 to be as similar as possible.
Comment #31
xjmNW for the dependency evaluation.
Comment #32
xjmWill contrib also have to change all their use statements like this if we backport it to 8.9? Def a BC break even if it's simple to fix.
Comment #33
alexpottRe #32 - nope they won't have to change all the use statements. The BC layer seems to have one issue as per #7 but by in-large it appears to work. We should investigate why #7 failed and see if it is important and thry to fix upstream if it is. Also we should try to get the BC layer to issue silenced deprecations as this will help the the Drupal and rest of PHP ecosystem upgrade.
Comment #34
catchLaminas' security policy is here: https://getlaminas.org/security/
I couldn't find documentation for LTS releases or anything else. Given they're immediately dropping support for Zend Framework (unless that changes) I'd be inclined not to believe anything about LTS anyway.
Comment #35
andypostThere's guide for upgrade https://getlaminas.org/blog/2019-12-31-out-with-the-old-in-with-the-new....
Comment #36
ThirstySix CreditAttribution: ThirstySix as a volunteer commented#18 is working fine for 8.8 version.
Comment #37
TravisCarden CreditAttribution: TravisCarden at Acquia commentedHas anyone seen the Zend Framework to Laminas Migration Guide? They provide a migration script. Was that used to help generate any of the patches here? I might give it a try.
Comment #38
TravisCarden CreditAttribution: TravisCarden at Acquia commentedThe Laminas migration script yielded no changes over the patch in #14 other than the addition of an unnecessary dependency (a Composer plugin that intercepts new
composer require
calls for Zend Framework packages and rewrites them to Laminas equivalents). Here's a reroll of #14.Comment #40
TravisCarden CreditAttribution: TravisCarden at Acquia commentedThe previous patch failed due to
symfony/psr-http-message-bridge
using the Diactoros library we're removing, so I updated to the latest version of that library, which removes the dependency on it. So the updated library diff looks like this:Note:
nyholm/psr7
was suggested by the new version ofsymfony/psr-http-message-bridge
to replace the missingDiactorosFactory
that caused the previous patch to fail. The addition ofphp-http/message-factory
andpsr/http-factory
was already discussed at #3104354-9: Update Laminas components including Diactoros and documented on the core dependency page.Comment #41
TravisCarden CreditAttribution: TravisCarden at Acquia commentedOops. Missed a commit.
Comment #43
SpokjeRe-roll of #41
Comment #44
catchI've added an entry for Laminas on https://www.drupal.org/core/dependencies - just above the Zend entries for now. I think this is as close to a proper dependency evaluation as we can get while they're migrating. The important thing here is they're completely retiring Zend/* while transferring all the code and issues wholesale to Laminas, so while things might be different with Laminas if they also change the project structure alongside the rename, Zend/* just does not exist and is unsupported at this point.
Comment #45
TravisCarden CreditAttribution: TravisCarden at Acquia commentedFixed test failures.
Comment #46
longwaveWhy are we making this change here, isn't this out of scope? I thought the Zend and Laminas versions were supposed to be identical except for the compatibility bridge?
Comment #47
TravisCarden CreditAttribution: TravisCarden at Acquia commented@longwave, it's because
symfony/psr-http-message-bridge:v1.3.0
(unfortunately) has an implicit dependency onzendframework
:The v2.0.0 release changes literally nothing from v1.3.0 but removing that dependency (changelog), so aside from the constraint change, there should be no impact.
Comment #48
xjmThis is presumably a good assertion to add, but how did it end up in the scope of this patch?
Comment #49
longwave> @longwave, it's because symfony/psr-http-message-bridge:v1.3.0 (unfortunately) has an implicit dependency on zendframework:
I thought this is what the compatibility bridge was designed to handle, so we shouldn't strictly need to upgrade here. #14 seemed to pass without this which is why I am confused.
Comment #50
Spokje@longwave So it looks like
symfony/psr-http-message-bridge:v1.3.0
can do the job (#14), but it would still leave us with an implicit dependency onzendframework
.Since this issue is about getting rid of dependencies on
zendframework
, it makes sense to me to bumpsymfony/psr-http-message-bridge
to^2.0.0
so we indeed get rid of all dependencies tozendframework
?Comment #51
longwaveOK, thanks for the clarification, I understand now.
Unused use statements.
This looks a bit odd, why have the variable for only one argument? https://github.com/symfony/psr-http-message-bridge/blob/master/Tests/Fac... uses the same object for all four parameters so should we do that here?
Comment #52
TravisCarden CreditAttribution: TravisCarden at Acquia commentedsymfony/psr-http-message-bridge:v1.3.0
has on zendframework but provided no useful information about why. (Just something like "String '' does not contain 'Username:'".) When I added that assertion the failure output showed the exact problem, leading me to the fix.Comment #53
longwaveThis looks RTBC as far as the code is concerned, but do we need a dependency evaluation for
nyholm/psr7
?edit: just spotted this:
We aren't 100% consistent in core.services.yml but there are extra spaces and a trailing comma here.
Comment #54
alexpottSo I think as #18 shows we don't need to update symfony/psr-http-message-bridge - the laminas components do a composer replace on the zend packages so the dependency is actually met.
I think we should go back to #18 here because that's what we need to 8.x and 9.x and then open yet another update all the dependencies issue for Drupal 9 to update all the dependencies including the psr-http-message-bridge and the laminas components too.
Comment #55
TravisCarden CreditAttribution: TravisCarden at Acquia commentedI hope that's the case, @alexpott, but there may be a catch. Tell me if I'm missing something.
The patch in #18 is against 8.9.x, and it still applies cleanly. I assume the tests will still pass. So if that's all you're talking about, I'm on board.
But I don't think the corresponding change against 9.0.x fares the same. The patch that passed tests in #14 no longer applies. That's why I rerolled it in #38. Now looking back at it, I realize I made a mistake in that patch. But I've rerolled #14 again and verified the correctness of it, and I get the same test failure. Here's the patch along with a diff showing that the only difference between my reroll and your original in #14 is context (including the Guzzle version upgrade from #3104473). Let's see what happens!
Comment #56
TravisCarden CreditAttribution: TravisCarden at Acquia commentedWell... I stand corrected. ¯\_(ツ)_/¯
Comment #57
longwave#18 and #55 look RTBC to me.
Opened #3107396: Upgrade symfony/psr-http-message-bridge to v2.0.0 to finish the rest of the upgrade discussed above.
Comment #59
catchCommitted dc86e91 and pushed to 9.0.x. Thanks!
Not yet committing to 8.9.x/8.8.x and still not sure about the best course of action there.
Comment #60
alexpottNote there is new version of laminas/laminas-zendframework-bridge - 1.0.1 so the Drupal 8.x patch should use that. For Drupal 9 this is being fixed in #3104354: Update Laminas components including Diactoros as that is updating laminas components there.
Comment #61
alexpottI've updated the laminas/laminas-zendframework-bridge also the fail #7 doesn't happen. However it didn't happen locally for me even on 1.0.0 so /shrug.
Here's a patch for 8.9.x that uses the latest laminas/laminas-zendframework-bridge and doesn't change all the classes thereby relying on the BC layer.
Comment #63
alexpottHmmm... I see. This happens for me on PHP 7.2 and lower but not on PHP 7.3. Interesting. So the BC layer works most of the time but a PHP bug means it doesn't work on PHP 7.2 and lower :(
Comment #64
alexpottAdd
class_exists(ExceptionInterface::class);
to \Drupal\aggregator\Plugin\aggregator\parser\DefaultParser::parse() results in a green test on PHP 7.2. But I think we should go for #18 on Drupal 8.x because if we don't then code navigation is broken in IDEs because they don't know anything about the magic in the Laminas autoloader.So here's a patch that updates #18 to use the latest bridge.
Comment #65
alexpottSo it's not PHP version :) I've finally been able to trigger the error locally at will. If I have apcu enabled I get the bug. If I remove apcu then I get the bug. So the Zend BC layer is fine - it's our very strange messing about with the autoloader in \Drupal\Core\DrupalKernel::initializeSettings() that is causing the problem.
It is fixable if we do something like...
But this option looks super scary of someone has done any other autoloader futzing.
Comment #66
TravisCarden CreditAttribution: TravisCarden at Acquia commentedSecondary to the main point (which I'll look into), does #65 indicate a gap in our test coverage? If something passes without a very common component of production environments (apcu) but fails with it, it seems like we have a defect vector unless the testbot tests both with and without it.
Comment #67
DamienMcKennaTagging as a requirement for Drupal 9.0-beta1.
Comment #68
alexpott@DamienMcKenna this has made it into Drupal 9 already. The big question is what do we do about Drupal 8.
Comment #69
xjmWe should still keep the tag for posterity.
Comment #70
catchJust to note I really have no idea what to do here:
Either we:
1. Do this in 8.9.x, risk some disruption, make a potential Laminas security update easier during the 8.9.x support cycle.
or
2. Don't do this, but have to fork-for-backport or make the switch if Laminas makes a security release
One introduces disruption for potentially no gain given the end of life of 8.9.x is December 2021.
One has a higher risk of a disruptive security release some time when 8.9.x is minimally maintained, but it's only disruption that we'd be deferring from here.
Also by not committing this to 8.8.x, we still have a nearly one-year window where we could still have to figure out a security backport process to 8.8.x, so committing it to 8.9.x only actually cuts the window-of-annoyance by six months.
Comment #71
xjmWe're also running out of time to do it in Drupal 8 at all at this point.
Comment #72
tedbowGiven the problem posed by @catch in #70 thought it might be useful to look at past ZendFramework security releases since 2016, longer than we would have to worry about but also I didn't want to go through all the releases since 2009
Sorry, probably to framework managers this stuff is already known but I wasn't sure about the frequency and how many affect use. Hopefully this will be useful to others.
Released 2019-03-28
Affects zendframework/zend-developer-tools
Doesn't look like we use this?
Released 2018-08-01
We were affected https://www.drupal.org/SA-CORE-2018-005
I tried to find the actual fix but the closest I found was something @alexpott made to attempt provide a backport to an LTS version https://github.com/alexpott/zend-diactoros/commit/67f2894cdf645ee3b1cbef...
But seems like the original fix would have been similar
Released 2016-12-20
Affects Zend-mail
Doesn't look like we use this?
Released 2016-09-08
Doesn't look like we were affected
This was an improvement over the next issue
Released 2016-07-13
Released 2016-04-13
Doesn't look like we were affected
In total there have been 44 security releases for ZendFramework since 2009.
Thats 4~5 a year
Looking at recent years
2019: 1
2018: 1
2016: 4 (though 1 was an improvement over another in the year)
2015: 11
2014: 6
2013: 3
Looks like there have been no security advisories under the Laminas Project yet https://getlaminas.org/security/
Comment #73
tedbowRetesting #64. I don't think it applies anymore
I looked into reroll this but I am not clear how to do this. I couldn't clear directions here. Can someone update the summary regarding that as this will probably need to be rerolled with any changes to composer.lock.
Or is there another issue I can look at that did something similar?
Comment #74
xjmThe frequency of security advisories isn't the issue, so much, and not really a predictor of what will happen in the future. All it takes is one security issue with one of the components we do use that we can't update. The closer we are to supported packages, the easier it will be for us to respond to the security issue.
I rerolled this from #64 by removing the hunk that changes
composer.lock
, applying the remaining patch, and then runningcomposer update --lock
.Comment #75
xjmErrr #74 didn't actually add the lockfile back; here's the correct patch.
Comment #76
xjm🤔 The output from the fail in #75:
So it looks like the problem is with github and the metapackages, not the Laminas ones?
composer install
works fine locally.Comment #77
xjmAh, maybe I need to
composer update drupal/core*
again after updating the lockfile.Comment #78
xjmYep, that got tests to run.
Comment #79
xjmNo idea why the
composer update drupal/core*
is also adding this weird hunk to the metapackage?Comment #80
xjm#77 has a pass once and a "Core composer reinstall failure" the second time for the same patch on the same environment.
Comment #81
alexpottHere's a patch without the
drupal/core-project-message
anddrupal/core-vendor-hardening
changes. I think composer gets a bit confused sometimes when it has to deal with existing dependencies, new dependencies and our fake path repositories like these libraries. The only way I've found of reliably making these type of changes is to remove vendor before you start, do composer install, use composer require, move stuff to core/composer.json, run composer update drupal/core - and liberal use of theCOMPOSER_ROOT_VERSION=8.9.x-dev
env var.Comment #82
selinav CreditAttribution: selinav commentedI have the problem with the 8.8.4. Can I test the patch for this version ?
Comment #83
longwaveAfter applying #81 to 8.9.x there are two remaining references to Zend packages that I think need to be updated:
Other than this I don't see anything else that needs changing and so if we are to make this change in 8.9.x this is RTBC.
Comment #84
jungleAs this issue is NW now, is it possible to get #3121885: Update coder to 8.3.8 reviewed and committed early than this one?
Comment #85
xjmRe: #84, this one is still the priority; it is critical to get it into 8.9 prior to the beta deadline which is... well technically it was 14h ago, but we have a little flexibility. :)
Coder is a dev dependency so we can make that change during beta as needed.
Thanks!
Comment #86
xjmOh, @selinav, regarding:
I assume you mean that you're getting messages that the Zend packages are abandoned and should be replaced with Laminas when you do a composer install or update? The current patch for this issue is for 8.9 so it won't work on 8.8, but once this is committed to 8.9 we are are discussing doing an 8.8 backport immediately after the next patch release. (It's a disruptive change normally not allowed in patch releases, but we understand this is adding a lot of noise for composer sites.)
Hope that helps answer your question! In the meantime, you can ignore those warnings; they are just letting you know that the Zend versions of the packages won't have future releases, which is why core is converting them to the Laminas versions in at least 8.9 and potentially 8.8. But the current versions do work fine for now.
Comment #87
xjmMmmm, there's also this (in 9.0.x and 8.9.x):
Which could explain why I was getting the weird dependency in #77? Is retaining the references to ZF instead of Laminas there intended?
Comment #88
alexpott#87 is a good spot. There's no impact in having this here because those directories don't exist. I've checked the new Laminas deps and they no longer include a docs directory and they still don't have a tests directory. So the right thing here is to remove this from both 8.9.x and 9.x - we missed this on the 9.x patch.
8.9.x patch also addresses #83
Comment #89
longwaveBoth patches in #88 look good to me.
Comment #90
catchComment #94
catchCommitted/pushed the clean-up to 9.1.x and 9.0.x, and also the 8.9.x patch.
Moving back to 8.8.x now. Discussed this with xjm yesterday and we think it might be better to commit this just after the next 8.8 patch release.
Comment #95
catchDiscussed this again with @xjm this morning. We are both a bit borderline, but leaning towards not committing this to 8.8.x. This leaves a window where we might be forced to do a backport for a Zend/Laminas security release (until December this year), but also means not adding a potentially disruptive change to a patch release for something that might not happen at all.
Comment #97
xjmComment #98
pritam.tiwari CreditAttribution: pritam.tiwari as a volunteer commentedI did face this issue during site upgrade with drush from 8.7.x to 8.9.0 druing the database upgrade. In my case drush is working fine post upgrade.
Comment #99
snsblvd CreditAttribution: snsblvd commentedI do have the same error (with the latest drush 8.3.5) and Update Core from 8.7.9 to 8.9.0. The site seems to working fine after the upgrade (with the error). Can this error be ignored or can this be fixed somehow (without migrating to composer)?