Value remains even though the reference entity is deleted

Yes this is a big problem. Related issues are #2827177: Delete references to deleted entities Drupal 8 and #1368386: Delete references to deleted entities with the added complexity of what should happen with the value.

See this comment for possible approaches https://www.drupal.org/node/1368386#comment-11834828

For entity references in D7 this deletion issue is handled by some additional contrib modules. Maybe it's too big to handle it here. In my opinion there should be some setting to at least display the value when the entity reference is invalid or empty.

Attached a first go at this: It adds two elements to the field formatter settings:

* Display invalid reference
* Invalid reference label

Depending on what you chose you can set the formatters to display just the value or a custom label like "(None)" or "(Deleted)".

For developers: if you have your own custom formatter and it's based EntityReferenceFormatterBase like the two formatters that come with the module then you can use the new ReferenceValueFormatterBase as a drop-in replacement to get this extra functionality (but also needs some code adjustments as now entity can be NULL - the label is set in the _label property).

This was already merged here: https://git.drupalcode.org/project/reference_value_pair/-/commit/60f23fb... 8 years ago so I'll close the issue.