Term entity type's "parent" field is now a standard Entity Reference field and the taxonomy_term_hierarchy table has been removed

With entity reference fields, you can do:

$entity->some_reference_field->target_id

But not with the Term entity type's parent field:

$term = Term::load(12);
$term->parent->target_id // returns always NULL, even if the term has a parent

This field has always been a special case, because it is marked as hasCustomStorage(), so its values are never loaded.

Not only does the above PHP code not work as expected due to custom storage, there are three other consequences:

1. REST support is missing: it's impossible to access a term's parent term via core's Serialization module, core's REST module, contrib's JSON API module, contrib's GraphQL module or contrib's Default Content module
2. Views integration is custom
3. Migrate D8 source support would need to be custom. See #2935951: Copy migrate source plugin from migrate_drupal_d8 into migrate_drupal

To fix all of them at the root, rather than adding more custom code and work-arounds, we changed Term to not use custom field storage for its parent field. This also allowed custom code to be deleted. It strengthens Drupal 8's Entity API: more things work in a predictable way.

DB schema change

The move from custom storage to non-custom storage means that the Taxonomy module no longer needs to define a custom schema: the taxonomy_term_hierarchy table (manually created by the Taxonomy module) disappears in favor of an auto-created taxonomy_term__parent table (automatically created by the field system).

Per https://www.drupal.org/core/d8-bc-policy#schema, this is not a BC break:

While the database query API and schema API itself are stable, the schema of any particular table is not. Writing queries directly against core tables is not recommended and may result in code that breaks between minor versions.

Code using the Entity Query API does not need to be updated. Custom SQL queries using the taxonomy_term_hierarchy table will need to be updated, and are recommended to use the Entity Query API instead.

Impact on Views

All views continue to work, there is an automated update path for them to use the new auto-created table.

Impact on RESTful Web Services module

Fixes REST API's responses. (As well as JSON API, GraphQL, etc.) The field simply had to be ignored until now by all clients, because there never was a sensible value in there.