Problem/Motivation

If you have an entity type that is revisionable and has a non-revisionable base field with multiple columns \Drupal\Core\Entity\Sql\SqlContentEntityStorage::loadFromSharedTables() will fatal making it impossible to load the entity.

The offending code is:

        // Find revisioned fields that are not entity keys. Exclude the langcode
        // key as the base table holds only the default language.
        $base_fields = array_diff($table_mapping->getFieldNames($this->baseTable), array($this->langcodeKey));
        $fields = array_diff($table_mapping->getFieldNames($this->revisionDataTable), $base_fields);

        // Find fields that are not revisioned or entity keys. Data fields have
        // the same value regardless of entity revision.
        $data_fields = array_diff($table_mapping->getFieldNames($this->dataTable), $fields, $base_fields);
        if ($data_fields) {
          $fields = array_merge($fields, $data_fields);
          $query->leftJoin($this->dataTable, 'data', "(revision.$this->idKey = data.$this->idKey)");
          $query->fields('data', $data_fields);
        }

The important detail here is that $table_mapping->getFieldNames() returns a list of entity field names, not database field (i.e. column) names.

Proposed resolution

$table_mapping->getColumnNames($field_name) should be used to convert the list of field names into a list of column names or $table_mapping->getAllColumns() should be used directly to get a list of column names.

Remaining tasks

User interface changes

API changes

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because a fatal error is triggered from e.g. MyEntityType::load()
Issue priority Major because there is a fatal error and this is an integral part of Drupal 8's API and there is no workaround
Prioritized changes Prioritized because it is a bug fix

Comments

tstoeckler’s picture

Note that this code was last touched in #2431329: Make (content) translation language available as a field definition, but it was just moved around there. I didn't dig any further to find where we introduced this.

tstoeckler’s picture

Title: Base fields with multiple columns break revisionable entity types » Non-revisionable base fields with multiple columns break revisionable entity types
Issue summary: View changes

I just realized fields have to be non-revisionable to reproduce this.

Updating issue summary and title.

jhedstrom’s picture

I looked into writing a test and realized I don't fully understand the issue as described above. The Redirect module has non-revisionable base fields with multiple columns and works as expected.

berdir’s picture

redirect however isn't revisionable itself. This only occurs with entity types that have revision tables and some revisionable fields.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jibran’s picture

jibran’s picture

Status: Active » Needs review
Issue tags: -Needs tests
StatusFileSize
new3.28 KB
new7.52 KB

It was happening with DER so I treid ER with same test case scenario and it passes because it has only one column but when I used ShapeItem as a base field then it's a different story.

jibran’s picture

StatusFileSize
new10.81 KB

Here is a complete patch.

The last submitted patch, 7: non_revisionable_base-2506805-7-pass.patch, failed testing.

The last submitted patch, 7: non_revisionable_base-2506805-7-fail.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 8: non_revisionable_base-2506805-8.patch, failed testing.

jibran’s picture

Status: Needs work » Needs review
StatusFileSize
new11.43 KB
new3.46 KB
new4.49 KB

Here is the green patch with the fix.

jibran’s picture

StatusFileSize
new1.06 KB
new2.39 KB
new3.43 KB

Minor doc update and removed useless changes.

The last submitted patch, 12: non_revisionable_base-2506805-12-fail.patch, failed testing.

The last submitted patch, 13: non_revisionable_base-2506805-13-fail.patch, failed testing.

The last submitted patch, 12: non_revisionable_base-2506805-12-fail.patch, failed testing.

The last submitted patch, 13: non_revisionable_base-2506805-13-fail.patch, failed testing.

chx’s picture

Status: Needs review » Needs work

Let's step back and consider what caused the bug. I nominate lack of comments. The code fixing it is correct but I believe this is a very hard to understand, rarely used piece of code and as such very heavy commenting is in order. The existing amount of very useful comments in loadFromSharedTables concur but we need even more.

The important detail here is that $table_mapping->getFieldNames() returns a list of entity field names, not database field (i.e. column) names.

Well, if that's the case then why not add it to the getFieldNames doxygen?

So loadFromSharedTables has similar sounding, somewhat explained variables -- but $fields is not explained at all:

        $base_fields = array_diff($table_mapping->getFieldNames($this->baseTable), array($this->langcodeKey));
        $fields = array_diff($table_mapping->getFieldNames($this->revisionDataTable), $base_fields);
        $data_fields = array_diff($table_mapping->getFieldNames($this->dataTable), $fields, $base_fields);
        if ($data_fields) {
          $fields = array_merge($fields, $data_fields);

Oh god. So the first are $key_fields. Then it's revisionable fields. Or non_key_fields? I am not sure what goes into the revision data table but $fields is an extremely poor name. Please check and rename. And then $data_fields, did you mean non_revisionable_fields? And then why do we merge these two? While I did delete some comments on copy paste, there's no comment for this. I think this would be a wonderful time to add a number of lines to the SqlContentEntityStorageSchema doxygen about what stores what and refer to it from here.

$column_names = array_merge($column_names, array_values($table_mapping->getColumnNames($data_field))); this needs a comment reminding the dear reader that getColumNames is keyed by property and we want to collect them all like so many pokemon so we want numeric keys instead which array_merge will then renumber...

jibran’s picture

Status: Needs work » Needs review
StatusFileSize
new3.15 KB
new5.63 KB

Thank you the review @chx and I appreciate you being brutally honest about the doxygen issue. :-)

Here is a new patch I hope this addresses your concerns.

jibran’s picture

StatusFileSize
new1.81 KB
new6.43 KB

Updated the var name as per @chx suggestion.

chx’s picture

Status: Needs review » Reviewed & tested by the community

This is lovely. Sooner or later both jibran and I will understand the field mapping system and hopefully others too :)

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed 28356f4e9fbeaf94a40ffeca479c40e2c77906ce to 8.2.x and fc06148 to 8.1.x. Thanks!

Nice comments and yes it is easy to see how we got this wrong before.

  • alexpott committed 28356f4 on 8.2.x
    Issue #2506805 by jibran, tstoeckler, chx: Non-revisionable base fields...

  • alexpott committed fc06148 on 8.1.x
    Issue #2506805 by jibran, tstoeckler, chx: Non-revisionable base fields...
jibran’s picture

Thanks @alexpott. It actually helped me to build this list https://gist.github.com/jibran/7afa1a6d90387e5ff674b6705b71af9b

  • alexpott committed 28356f4 on 8.3.x
    Issue #2506805 by jibran, tstoeckler, chx: Non-revisionable base fields...

  • alexpott committed 28356f4 on 8.3.x
    Issue #2506805 by jibran, tstoeckler, chx: Non-revisionable base fields...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.