Problem/Motivation

\Drupal\Core\Entity\Query\Sql\Tables, which is used for the SQL entity query implementation, has been refactored a lot already, but still contains some assumptions about base fields (which it sometimes still calls entity properties). For example, it does not support base fields with multiple columns, as it has never been updated to support that when support for it was added to SqlContentEntityStorage.

Practically, this means for example that it is right now impossible to query the taxonomy term description format (Not a very useful example, but that might be the only one we have in core...)

Proposed resolution

Refactor the logic in that class so that there are only the absolutely necessary differences between shared tables and dedicated tables, all the other logic should be the same for all fields.

Remaining tasks

User interface changes

API changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir’s picture

Issue summary: View changes
fago’s picture

Priority: Major » Critical

I'd call it critical if you are not able to query for the fields of an entity, thus bumping priority.

Berdir’s picture

Issue tags: +Ghent DA sprint

While not technically blocked on it, so not postponing, but it is touching the same code as #2068655: Entity fields do not support case sensitive queries, I'll start with this when that is in.

catch’s picture

Priority: Critical » Major

Downgrading this to major because it looks like it's:

1. Not a regression from Drupal 7
2. Not having any impact on core
3. Not blocking a contrib module or core patch
4. Possible to fix with zero API implications in a patch release.

If any of these are incorrect we should look at bumping priority back up though.

Berdir’s picture

1. Hm.

taxonomy_term.description:
In 7.x, it was possible to query the format because the description value and format were two different properties. Now that we made it a single field with two properties, it is no longer possible.

that's not a very useful example (why would you do that..) but AFAIK, that is the only example that we have in core right now.

Just wanted to point this out, I don't care if this issue is critical or major, I plan to work on it asap anyway now that the other entity query issue is in.

catch’s picture

Hmm actually I can think of a use case for that..

At the moment it looks like we just prevent text formats from being deleted via the UI.

If we allowed that, but added validation on text format usage, you'd need to be able to query the taxonomy term description (or any base field that has a format) to see if it has a record using that format, as well as configurable fields.

So it blocks any issue that allows that, but I don't think we have such an issue open.

dawehner’s picture

I have a more common usecase: $menu_Link_content->link->uri ...

webchick’s picture

Right, this is referenced by #2406749: Use a link field for custom menu link; however, that issue also seems to have a workaround, so I think this still qualifies as "really nice to have"?

dawehner’s picture

Right, this is referenced by #2406749: Use a link field for custom menu link; however, that issue also seems to have a workaround, so I think this still qualifies as "really nice to have"?

It is not really nice to have ... the workaround breaks the APIs we have. If anyone writes a different storage for sql tables, the queries result in invalid SQL.
If all the abstractions don't actually work, where is the point in starting with them in the first place.

yched’s picture

Agreed, we should really fix this, or EntityQueries are half broken.
The workarounds for writing EQs on base fields with multiple columns are hardcoded on SQL column names, and would break if a different storage is swapped.

Just needs a deep dive into Drupal\Core\Entity\Query\Sql\Tables :/

Berdir’s picture

Yes, I still want to fix this, major or critical. I was hoping that the entity query delta issue would have been done quickly, I was waiting on that one to avoid non-trivial conflicts in Tables.php, working with that class is hard enough without messy conflicts :)

Berdir’s picture

Here's a test as a starting point. Will try to work on this soon.

Berdir’s picture

Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, 12: entity-query-base-field-columns-2391217-1.patch, failed testing.

Berdir’s picture

Status: Needs work » Needs review
FileSize
6.26 KB

Ok, I was hoping to use this issue to refactor \Drupal\Core\Entity\Query\Sql\Tables in this issue, instead of adding even more conditions, but it is probably better to go with a minimal fix here and test coverage so that we can get rid of this cri-major issue and refactor the code in a normal task.

Tables::addField() is now 150 lines of code with an insane amount of if/else/for with nasty $key++ in the middle of loops, and re-using of condition variable assignments 50 lines earlier. @Mile23 will probably suffer an instant heart attack if he will ever find this code. But as I said, that's something for another issue/day I think.

This passes EntityQueryTest and BreadcrumbTest. Go bot.

yched’s picture

Agreed with bringing some sanity back in Tables.php in a separate issue.

Comparing the "it is in a dedicated table" and "it is in a shared table" code branches now, looks like the only remaining difference feature-wise is the "follow relationship" thing (which TBH, confuses me a bit), that is still not supported for base fields. Do we want to open a separate issue for that ?

Other than that, patch looks ready to me

Berdir’s picture

Comparing the "it is in a dedicated table" and "it is in a shared table" code branches now, looks like the only remaining difference feature-wise is the "follow relationship" thing (which TBH, confuses me a bit), that is still not supported for base fields. Do we want to open a separate issue for that ?

It confuses me more than a bit. The relationship code continues below, and it relies on this part. Because here we populate $propertyDefinitions there if it did not already happen here, *and* we run some additional code there. I tried to refactor this already when i changed it to support multi-value base fields (from config vs base field to separate table vs. shared table field). I tried to move all this code out of the if/else, but then the code below didn't work as expected anymore. So I duplicated it with different variable names to not change how that part there works. Fun, isn't it?

AFAIK, relationships for base fields actually work:

$ids = \Drupal::entityQuery('node')
  ->condition('uid.entity.name', 'admin')
  ->execute();

Works as expected. Don't ask me how or why.

So yes, we need a follow-up, but just for "Refactor \Drupal\Core\Entity\Query\Sql\Tables to avoid code duplication and improve readability" or something like that.

yched’s picture

Status: Needs review » Reviewed & tested by the community

Aw. Fun indeed :-/

Ok, so let's fix this for now.

plach’s picture

Issue tags: +entity storage

Looks good to me as well. Would it make sense to add an explicit test for the entity relationship example above?

+1 on creating a separate issue for the Table refactoring.

Berdir’s picture

dawehner’s picture

+1

But yeah continuous work in a new issue to clean things up here, would be nice.

yched’s picture

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

I like the idea of doing a cleanup in a separate issue. This issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed 326db64 and pushed to 8.0.x. Thanks!

  • alexpott committed 326db64 on 8.0.x
    Issue #2391217 by Berdir: Support base fields with multiple columns in...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.