$entity->_referringItem conflicts with rendering the same entity multiple times on the same page

Problem/Motivation

entity reference formatters set a magic, mostly undocumented property _referringItem on entities that are being rendered.

That's a powerful feature for more advanced, context-sensitive control over the render process. For example we use it in combination with paragraph behaviors that allow users to control how entities look in ways that would be complex to handle with just view modes.

Steps to reproduce

Beside caching issues that are tricky but can be handled if you are careful (either disable caching or add keys), we recently realized that this is a problem if you are rendering the same entity multiple times on the same page with different behaviours, due to lazy rendering and render caching.

Basically, you have multiple formatters that all set that property on the same object, but you can only have one of those. And at an unknown time later in the render process, the rendering actually happens, which is when code might access it.

That means the last formatter that happens to set the property wins and that remains set for the rest of the process as entities are statically cached including all their set properties.

A similar problem also applies to $entity->view, but that's a different issue for a different day.

Proposed resolution

As a workaround in our project, we cloned the entity objects as part of \Drupal\Core\Field\Plugin\Field\FieldFormatter\EntityReferenceFormatterBase::getEntitiesToView(), that helps in our case.

A better solution might be to put it in the prepared render array, then it has a clear scope but it's obviously an API change. But in scope of #3281720: [meta] Deprecate __get/__set() on ContentEntityBase, that would need to move away anyway.

It's still challenging to handle with enabled render caching, see the related issue for that.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet