Problem/Motivation
Since #2910211: Allow computed exposed properties in ComplexData to support cacheability., implicit cacheability bubbling is deprecated, because there now finally is a way for normalizers to bubble cacheability metadata of the data they're normalizing.
As of #2870194: Ensure that process-isolated tests can use Symfony's PHPunit bridge to catch usages of deprecated code, deprecation errors result in test failures. But any deprecation errors that result in core test failures are ignored, for now.
Proposed resolution
Let's fix all deprecation errors for the deprecation that #2910211 added. This is then also fixing REST/Serialization technical debt. That deprecation is:
Implicit cacheability metadata bubbling (onto the global render context) in normalizers is deprecated since Drupal 8.5.0 and will be removed in Drupal 9.0.0. Use the "cacheability" serialization context instead, for explicit cacheability metadata bubbling. See https://www.drupal.org/node/2918937
Remaining tasks
Find all the places.Fix them.
User interface changes
None.
API changes
None.
Data model changes
None.
Comment | File | Size | Author |
---|---|---|---|
#62 | 2922487-62.patch | 7.56 KB | jofitz |
#62 | interdiff-2922487-58-62.txt | 1.82 KB | jofitz |
#58 | 2922487-58.patch | 8.47 KB | jofitz |
#54 | 2922487-54.patch | 9.21 KB | Wim Leers |
#54 | interdiff.txt | 4.4 KB | Wim Leers |
Comments
Comment #2
Wim LeersComment #3
Wim LeersMost of the failures are in the HAL REST tests. Let's fix those.
Comment #4
Wim LeersAll
Comment
,EntityTestLabel
,Item
,Media
andNode
REST tests are also failing, due to\Drupal\serialization\Normalizer\EntityReferenceFieldItemNormalizer::normalize()
usingEntity::url()
(which is a deprecated API too), which callsUrl::toString()
, which automatically bubbles onto the global render context, hence triggering the deprecated code path.Comment #5
Wim LeersUnused.
Comment #6
Wim LeersComment #7
Wim LeersThis should come back green, and should be ready for final review.
Comment #11
Wim LeersComment #13
Wim LeersI forgot to fix one.
Comment #14
tedbowI am confused because this issue seems to be about just removing deprecation warnings. So not using a deprecated code path.
But here we are actually changing what is expected in a the normalized entity. If are just fixing the use of deprecated code shouldn't the REST responses not change?
Comment #15
dawehnerHow is that working? Right now,
$context
is not passed by reference.\Drupal\serialization\Normalizer\NormalizerBase::addCacheableDependency
manipulates$context
, but then nothing happens? I'm a bit confused ...Comment #16
tedbowre #15 my understanding is you don't actually need to pass the
$context
by reference.$context[ResourceResponseSubscriber::SERIALIZATION_CONTEXT_CACHEABILITY]
contains an instance of\Drupal\Core\Cache\CacheableMetadata
. So since it is object you always have reference to it(unless you explicitly clone it).So in
\Drupal\serialization\Normalizer\NormalizerBase::addCacheableDependency()
$context[ResourceResponseSubscriber::SERIALIZATION_CONTEXT_CACHEABILITY]->addCacheableDependency($data);
does affect the
$context
that\Drupal\rest\EventSubscriber\ResourceResponseSubscriber::renderResponseBody
will have access to.If
\Drupal\serialization\Normalizer\NormalizerBase::addCacheableDependency()
added or removed keys from the$context
array itself then that would not affect the$context
variable that the caller had because arrays are not passed by reference by default.Comment #17
dawehnerOh nevermind, PHP is, well, not the nicest language :)
Comment #19
Wim LeersRight :/
This is a BC break.
This is the root cause.
The old code calls
::url()
, the new code calls::toUrl()
. Why? Because\Drupal\Core\Entity\Entity::url()
callsUrl::toString()
, which bubbles onto the global render context, which is the thing that is deprecated.Unfortunately,
\Drupal\file\Entity\File::url()
is overridden to have completely different behavior.File
entities do not actually have a canonical URL, butFile::url()
was effectively hacked in #2277705: Files don't have URI nor href (kinda "you're not hacking core if core is already hacked").And that hack is now broken.
There is sadly only one work-around AFAICT: special case the
File
entity in the normalizer.Comment #20
Wim LeersAdded a terrible work-around which spreads #2277705: Files don't have URI nor href's bad decision further, but that's the only way to retain BC AFAICT.
Comment #21
Wim LeersComment #23
Wim Leersbecause as of #20, we're passing/storing an absolute external URL to
\Drupal\Core\Url
.Comment #27
Wim LeersNo longer applied. Rebased.
Comment #28
Wim LeersThe changes in
File
with the crazy work-arounds for BC actually caused all these test failures. AFAICT #2277705: Files don't have URI nor href really has put us in an impossible situation… :(This introduces work-arounds so that the REST test coverage is aware of the lying that
\Drupal\file\Entity\File
has to not break BC, but in not breaking BC, it must continue to return invalid values … and hence it breaks the REST test coverage! So we need to make the REST test coverage aware of the brokenness ofFile
after all.AFAICT this is the best possible solution…
Comment #31
Wim LeersRebased #28.
Comment #33
Wim LeersA 10-minute round of debugging didn't reveal the root cause. I'm quite confused how this could cause
entity_test
tests to fail. But I did spot one stupid mistake I made in #28 that surely cannot help…Comment #34
Wim LeersIt'd help if I made #33's change consistently… and now suddenly most tests should pass!
Comment #35
Wim LeersThis just might be green… 💪
Comment #39
Wim LeersSo close! Three failures left, all three about 406s instead of 403s.
Comment #40
Wim Leers✅
Comment #41
borisson_This comment doesn't really provide a lot of value.
This might be better, but I'm not sure.
To provide BC, we provide the old value for canonical links.
Comment #42
Wim LeersFair!
Comment #43
borisson_Sorry for not finding this earlier:
The issue linked to from here is already closed (also, both that issue and this one end on 487 just to make things extra confusing).
Comment #44
Wim LeersGood catch; clarified the comments.
Comment #45
BerdirSeems like this is heavily overlapping with #2402533: Provide File::createFileUrl() as a replacement for the deprecated File:url() implementation, as you also commented there.
I'm not too happy about adding more code that's already deprecated, also without doing any deprecation messages.
We have the already deprecated \Drupal\hal\Normalizer\FileEntityNormalizer, could we override getEntityUri() there and deal with file like that? Then it's all in one place and in 9.x, we can just remove that file. I think that's what we should have done from the beginning instead of overriding url() like that in the file entity class.
An argument could be made that another entity type has a similar special override of url() and this would break it, but that's already kind of broken, you already have the url() call above that you call *despite* not having a canonical link template. The whole thing is a special case for file.
And now we rely that file does lie about implementing the canonical link template so that we skip the condition that we befor e had specifically for file :)
This is still calling urlInfo(). which is deprecated, just doesn't trigger a warning yet, does it make sens to fix that here too?
Comment #46
Wim LeersI know this overlaps with #2402533: Provide File::createFileUrl() as a replacement for the deprecated File:url() implementation — see your comment #2402533-83: Provide File::createFileUrl() as a replacement for the deprecated File:url() implementation and my response #2402533-84: Provide File::createFileUrl() as a replacement for the deprecated File:url() implementation from 6 months ago.
I like that proposal a lot! But AFAICT it'd still result in deprecation warnings. Far fewer, but they'd still be there. Which would mean we won't be able to do this:
But, nonetheless, I did try to implement what I think you mean. I do think it's the better approach. It also means this issue doesn't need to make any changes to
\Drupal\file\Entity\File
anymore, which makes this issue far simpler!Can you please review the changes? :)
Comment #47
Berdirwe should probably improve the return documentation a bit and mention that it can be a string (deprecated?) or a GeneratedUrl object?
can't you do the same here as you did in getEntityUri() ?
Convert that string again to a url object and add the option?
That means the deprecation message that you want to get rid of here won't go away.
We can't make the one from url() go away once we actually add that but we'll have to figure that out when we add that and want to get rid of the deprecation messages then.
Actually... you could just call file_create_url($this->getFileUri()) directly here if we want to avoid the (future) deprecation messages.
I'm actually not 100% sure what the final behavior in 9.x will be now for file in a REST response.
We now have the file URL in a separate computed field, so overriding uri is already deprecated. IMHO, I would also expect that the file then would not have a top-level self link anymore, like I suggested in the issue referenced above. which means...
I'm not quite sure where this part should go, we can possibly figure it out as part of removing/really deprecating url(), but a separate issue for it probably makes sense, again based on comment that it is valid that an entity does not have a self link, so instead of returning url(), we should just return NULL and the caller should then allow that.
Comment #49
Wim Leers#46 failed because it removed work-arounds that were necessary for this:
And we need to make that change to fix the deprecation warning.
#48:
AFAICT we can't because then
file_create_url()
won't be called.That's indeed a way make future removal even easier. Good call :)
Right, but we can only remove that
self
link in Drupal 9, right? Or do you see a way to do that without breaking BC?NULL
today, right? Because that'd be a BC break.Implemented @Berdir's suggestion, but this patch will still fail in the exact same way due to the reason provided at the top of this comment.
Comment #50
Wim LeersHere's a work-around attempt for the problem described in #49 in
EntityReferenceFieldItemNormalizer
.Comment #52
Wim LeersThat worked :) Back to @Berdir for review!
Comment #53
Berdir#47.1
and #47.3 ;)
as far as I see, these changes are not necessary anymore? hasLinkTemplate() is already FALSE, so this isn't doing anymore anymore.
I reverted all changes in this file and at least FileHalJsonCookieTest passed just fine.
Why can't we handle this with a setting, just like bc_file_uri_as_url_normalizer?
I guess we can't do this until we resolve #2925520: Add a computed 'file_url' property to FileItem (for exposing file URL in file field normalization). what about instead of a @todo on 9, make it a @todo pointing to that issue and there we add file_url and deprecate this?
Comment #54
Wim Leersstring
? I must be overlooking something. Sorry :(That's indeed why. Getting rid of that deprecation warning is the goal of this issue. So, I kept this.
Comment #56
Wim LeersPing @Berdir
Comment #57
BerdirHuh, where exactly did all those months go since the last update here :)
1. No I did, I missed the getGeneratedUrl() call. All good.
I think this looks great now, pretty small patch in the end, without any change to the outside and fewer deprecated method calls. Would have set this to RTBC, but apparently the patch needs a reroll for 8.7 according to #54?
Comment #58
jofitz CreditAttribution: jofitz at ComputerMinds commentedRe-rolled.
Comment #59
Wim Leers😃 I know, time flies!
Great!
🎉
Thanks, @Jo Fitzgerald for the reroll! Per @Berdir's feedback, moving to RTBC :)
Comment #61
alexpottThis change is unnecessary - no?
Are we deprecating or removing?
Comment #62
jofitz CreditAttribution: jofitz at ComputerMinds commentedAddressed comments in #61.
Comment #63
Wim LeersRight; that contained logic changes in an iteration from long ago. But now it indeed can be dropped altogether. Great! :) Thanks, Alex and Jo! 👌
Comment #64
alexpottCommitted 68c0fd8 and pushed to 8.7.x. Thanks!
Fixed coding standards on commit.