Problem/Motivation

DER has views integration for storage fields after #2321721: Provide a views relationship for each dynamic entity reference field but views integration for basefield is still pending. Core has not provided a proper API for this yet #2337515: Allow @FieldType to customize views data.

Proposed resolution

Until #2337515: Allow @FieldType to customize views data is resolved we'll implement hook_views_data for base fields and add forward and backward relationships. For ER field cores uses protected method EntityViewsData::processViewsDataForEntityReference() DER can't subclass EntityViewsData

Remaining tasks

Write tests
Review
Commit

User interface changes

None

API changes

We'll have relationships for basefields.

CommentFileSizeAuthor
#54 provide_views-2548395-54.patch136.59 KBjibran
#49 provide_views-2548395-49.patch139.45 KBjibran
#49 interdiff.txt11.35 KBjibran
#48 interdiff.txt5.14 KBjibran
#47 provide_views-2548395-47.patch140.4 KBjibran
#45 provide_views-2548395-45.patch139.88 KBjibran
#45 interdiff.txt16.15 KBjibran
#40 provide_views-2548395-40.patch138.48 KBjibran
#40 interdiff.txt12.27 KBjibran
#37 provide_views-2548395-37.patch129.7 KBjibran
#37 provide_views-2548395-37-do-not-test.patch7.24 KBjibran
#37 interdiff.txt14.43 KBjibran
#36 provide_views-2548395-36.patch129.63 KBjibran
#36 interdiff.txt72.81 KBjibran
#34 provide_views-2548395-34.patch59.63 KBjibran
#34 interdiff.txt30.33 KBjibran
#32 provide_views-2548395-32.patch40.95 KBjibran
#32 interdiff.txt18.5 KBjibran
#30 provide_views-2548395-30.patch35.91 KBjibran
#30 interdiff.txt2.62 KBjibran
#28 provide_views-2548395-28.patch35.32 KBjibran
#28 interdiff.txt32.99 KBjibran
#26 2548395-interdiff-25-26-do-not-test.diff7.17 KBdpi
#26 2548395-26-vdc-relationships-base.patch5.74 KBdpi
#25 2548395-interdiff-23-25-do-not-test.diff8.48 KBdpi
#25 2548395-25-vdc-relationships-base.patch5.62 KBdpi
#23 2548395-23-vdc-relationships-base.patch13.8 KBdpi
#23 2548395-interdiff-21-23-do-not-test.diff3.13 KBdpi
#21 2548395-21-vdc-relationships-base.patch13.18 KBdpi
#21 2548395-interdiff-15-21-do-not-test.diff9.39 KBdpi
#15 2548395-interdiff-7-15.diff3.27 KBdpi
#15 2548395-15-vdc-relationships-base.patch11.13 KBdpi
#8 2548395-interdiff-6-to-7.diff14 KBdpi
#8 2548395-7-vdc-relationships-base.patch11.11 KBdpi
#6 2548395-interdiff-1-to-6.diff4.03 KBdpi
#6 2548395-6-vdc-relationships-base.patch6.19 KBdpi
#2 2548395-1-vdc-relationships-base.patch5.06 KBdpi

Comments

dpi created an issue. See original summary.

dpi’s picture

Issue summary: View changes
Issue tags: +Needs tests
StatusFileSize
new5.06 KB
dpi’s picture

Issue summary: View changes
Status: Active » Needs review

Ideally hook_field_views_data should be merged into hook_views_data to reduce duplicate code.

jibran’s picture

For ER core adds data in EntityViewsData::processViewsDataForEntityReference() see #1740492: Implement a default entity views data handler. Use of hook_views_data seems wrong because it adds new data where as we are altering existing data so maybe hook_views_data_alter. @dawehner what do you suggest?

dpi’s picture

I think we should be using hook_views_data, we're creating a whole collection of new fake fields. If it was just one then we could use the existing column field.

dpi’s picture

Issue summary: View changes
StatusFileSize
new6.19 KB
new4.03 KB

This one checks that both entity types are SQL and removes debug junk.

dpi’s picture

Issue summary: View changes
dpi’s picture

Issue summary: View changes
StatusFileSize
new11.11 KB
new14 KB

This patch refactors both hooks into one.

The last submitted patch, 6: 2548395-interdiff-1-to-6.diff, failed testing.

dpi’s picture

Issue summary: View changes

The last submitted patch, 8: 2548395-7-vdc-relationships-base.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 8: 2548395-interdiff-6-to-7.diff, failed testing.

dawehner’s picture

jibran’s picture

dpi’s picture

Status: Needs work » Needs review
StatusFileSize
new11.13 KB
new3.27 KB

Test shouldn't pass until #2553831: getTargetTypes has expectations that are not being met is committed.

  • Fixed field config columns.
  • Change pseudo field IDs back to old ID to satisfy existing test views. Personally I'd prefer a der_ prefix.

The last submitted patch, 15: 2548395-15-vdc-relationships-base.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 15: 2548395-interdiff-7-15.diff, failed testing.

jibran’s picture

Let's not combine both the hooks. We should keep dynamic_entity_reference_field_views_data() because it is the proper way and I think we can use dynamic_entity_reference_views_data() for base fields so that other modules can change that in hook_views_data_alter(). Note that dynamic_entity_reference_views_data() is a temporary fix till we have a proper api after #2337515: Allow @FieldType to customize views data.
I have worked on this over the weekend. I used the patch from #6. I have setup a test module with basefields found some nasty edge cases. I'll write some tests later this week.

jibran’s picture

Re: #15 Patch looks good. Can you move base field relationships to hook_views_data as per #18? I can add my tests in #2542274: Write tests for DER field as basefield on an entity.
Does it work properly when a base field has a dedicated table?

dpi’s picture

I have split them back into separate hooks.

Does it work properly when a base field has a dedicated table?

I have a DER base field with unlimited cardinality (which results in dedicated table) in Courier's template collection entity. It works as expected.

dpi’s picture

Status: Needs work » Needs review

One day I'll remember

dpi’s picture

I didn't properly test back references to base fields with multiple cardinality. This fixes it.

In addition, relationships are no longer created to entities that do not support views.

jibran’s picture

+++ b/dynamic_entity_reference.views.inc
@@ -6,93 +6,236 @@
 function dynamic_entity_reference_field_views_data(FieldStorageConfigInterface $field_storage) {

Let's revert all the changes to this hook.

dpi’s picture

No changes, only revert to fields hook per #24.

Can we do the changes in a different issue (out of scope)? Or is there something wrong with the changes?

dpi’s picture

Oops deleted imports.
Removed $table foreach.

jibran’s picture

Can we do the changes in a different issue (out of scope)? Or is there something wrong with the changes?

Nothing wrong with the changes. I quiet like the changes. Let's move them to follow up and let's also fix them in core for ER field as well.

jibran’s picture

Added relationship data test for single field. Made some changes because of tests. Update docs.
Needs to add actual view to verify the relationship.
Unable to make the base field multivalue but is behaving as a multi field anyway.
Needs to relationship data test for multivalue field.
Needs to add actual view to verify the relationship.

jibran’s picture

+++ b/src/Tests/Views/DynamicEntityReferenceBaseFieldRelationshipTest.php
@@ -0,0 +1,324 @@
+    // Update definitions and schema.
+    $manager = \Drupal::entityDefinitionUpdateManager();
+    $storage_definition = $manager->getFieldStorageDefinition('dynamic_references', 'entity_test');
+    $storage_definition->setCardinality(FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED);
+    $manager->updateFieldStorageDefinition($storage_definition);
+    $manager = \Drupal::entityDefinitionUpdateManager();
+    $storage_definition = $manager->getFieldStorageDefinition('dynamic_references', 'entity_test_mul');
+    $storage_definition->setCardinality(FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED);
+    $manager->updateFieldStorageDefinition($storage_definition);

@plach can I do this in test? Or is there some easy way to update base field definition. dynamic_entity_reference_entity_test adds a base field named dynamic_references with 1 cardinality. For testSingleBaseFieldRelationship test that works fine but I have to make it an FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED for second test testMultiBaseFieldRelationship. I tried above code in the test from https://www.drupal.org/node/2554097 but it gave exception.

+++ b/tests/modules/dynamic_entity_reference_entity_test/dynamic_entity_reference_entity_test.module
@@ -0,0 +1,52 @@
+      ->setCardinality(1)
+++ b/src/Tests/Views/DynamicEntityReferenceBaseFieldRelationshipTest.php
@@ -0,0 +1,324 @@
+    $entity->dynamic_references[] = $referenced_entity;
+    $entity->dynamic_references[] = $referenced_entity_mul;
...
+    $this->assertEqual($entity->dynamic_references[0]->entity->id(), $referenced_entity->id());
+    $this->assertEqual($entity->dynamic_references[1]->entity->id(), $referenced_entity_mul->id());
...
+    $entity->dynamic_references[] = $referenced_entity;
+    $entity->dynamic_references[] = $referenced_entity_mul;
...
+    $this->assertEqual($entity->dynamic_references[0]->entity->id(), $referenced_entity->id());
+    $this->assertEqual($entity->dynamic_references[1]->entity->id(), $referenced_entity_mul->id());

I set cardinality to 1 but how come I can set two values for a field and it passes? Is there something wrong with der?

jibran’s picture

StatusFileSize
new2.62 KB
new35.91 KB

Here is uncommented version of testMultiBaseFieldRelationship This patch will fail with exception.

Status: Needs review » Needs work

The last submitted patch, 30: provide_views-2548395-30.patch, failed testing.

jibran’s picture

Status: Needs work » Needs review
StatusFileSize
new18.5 KB
new40.95 KB

Ok I found the way around #29.1 so unping @plach. But #29.2 is still an issue. Anyways this patch is green. Remaining thing is to check these with actual views.

jibran’s picture

+++ b/tests/modules/dynamic_entity_reference_entity_test/dynamic_entity_reference_entity_test.module
@@ -0,0 +1,61 @@
+    // @todo use \Drupal::state()->get('dynamic_entity_reference_entity_test', 0); here.
+    if (DynamicEntityReferenceBaseFieldRelationshipTest::$testRun) {

Why can't I use state here? Core tests use it all over the place. Any suggestions!

jibran’s picture

StatusFileSize
new30.33 KB
new59.63 KB

Here are the tests for basefield view tests with 1 cardinality, found a bug in views data and fixed it. Remaining tasks are doc fixes and cardinality > 1 testing with view.

jibran’s picture

As it turns out views integration for multi value basefields is broken #2477899: Multiple valued Base fields won't work in Views. We can't really write views tests for multi values der basefield not until we add relationship ourselves which seems hacky and can be moved to follow up. Only thing remaining here is write tests for entity_test_mul der basefield. I'll look at it later tonight.

jibran’s picture

StatusFileSize
new72.81 KB
new129.63 KB
  1. Added views data for multi value basefield.
  2. Added tests for data table entity

@todo

  1. FIx views for multi value basefield.
  2. Add tests for multi value basefield views.
jibran’s picture

StatusFileSize
new14.43 KB
new7.24 KB
new129.7 KB

Yay!!! found another issue in DERBaseFieldViewsData. Tests++

Added a do-not-test patch without any of the test codes.
Fixed views for multi value basefield.
@todo
Add tests for multi value basefield views.

Status: Needs review » Needs work

The last submitted patch, 37: provide_views-2548395-37.patch, failed testing.

The last submitted patch, 37: provide_views-2548395-37.patch, failed testing.

jibran’s picture

Assigned: dpi » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new12.27 KB
new138.48 KB

Everything is done here. This is ready for reviews.

dpi’s picture

Looks ready to go.

I assume this comment is for the intersect operation:

// @todo mention the reason here.

Something like

Filter out non SQL entity types.
or
Filter out non SQL entity types as they are not joinable.

$relationship

Why not just merge the array directly with $data[$target_table][$psuedo_field]['relationship'] instead of creating a variable?

And finally

Do you want to change the relationship ID? ($psuedo_field assignments) to prevent potential collisions with core.

Solution: Add a prefix like "der_". We shouldnt be changing ID's after this is committed as it will break existing views.

If the plan is to remove the code later. Then we must ensure the ID is identical, and core provides identical functionality.

naitik945’s picture

  1. +++ b/src/Tests/Views/DynamicEntityReferenceBaseFieldRelationshipTest.php
    @@ -0,0 +1,662 @@
    +    // Check the backwards reference for test entity - data table using dynamic_references.
    

    In general, all lines of code should not be longer than 80 characters.
    https://www.drupal.org/coding-standards#linelength

  2. +++ b/src/Tests/Views/DynamicEntityReferenceBaseFieldRelationshipTest.php
    @@ -0,0 +1,662 @@
    +    // Check the backwards reference for test entity - data table using dynamic_references.
    

    In general, all lines of code should not be longer than 80 characters.
    https://www.drupal.org/coding-standards#linelength

  3. +++ b/src/Tests/Views/DynamicEntityReferenceBaseFieldRelationshipTest.php
    @@ -0,0 +1,662 @@
    +    // Check the backwards reference for test entity - data table using dynamic_references.
    

    In general, all lines of code should not be longer than 80 characters.
    https://www.drupal.org/coding-standards#linelength

dpi’s picture

WTF

Spam deleted.

jibran’s picture

StatusFileSize
new16.15 KB
new139.88 KB

Fixed #41, #42 and #43.

If the plan is to remove the code later. Then we must ensure the ID is identical, and core provides identical functionality.

The ID is identical that is why I was able to create views from gui.

Do you want to change the relationship ID? ($psuedo_field assignments) to prevent potential collisions with core.

Core is not providing any relationship. Core doesn't even know that relationship exists so no collision possible.

dpi’s picture

Status: Needs review » Reviewed & tested by the community

Yep, all good, I struck the text since I saw $field_name is used in the ID. I wrongly assumed it was concatenating origin and target etids.

Looks good to go.

jibran’s picture

StatusFileSize
new140.4 KB

Updated setup function logic to accommodate #2542274: Write tests for DER field as basefield on an entity.

jibran’s picture

StatusFileSize
new5.14 KB

And interdiff.

jibran’s picture

StatusFileSize
new11.35 KB
new139.45 KB
dawehner’s picture

Issue summary: View changes
dawehner’s picture

This looks really good, I would say, ship it!

  1. +++ b/dynamic_entity_reference.views.inc
    @@ -98,3 +102,150 @@ function dynamic_entity_reference_field_views_data(FieldStorageConfigInterface $
    +      // Unlimited (-1) or > 1 store field data in a dedicated table.
    +      $table = $field->getCardinality() == 1 ? $base_table :  $entity_type->getBaseTable() . '__' . $field_name;
    

    Afaik you could just use \Drupal\Core\Entity\Sql\TableMappingInterface::getFieldTableName here?

  2. +++ b/dynamic_entity_reference.views.inc
    @@ -98,3 +102,150 @@ function dynamic_entity_reference_field_views_data(FieldStorageConfigInterface $
    +        $target_table = $target_entity_type->getDataTable() ?: $target_entity_type->getBaseTable();
    ...
    +          'base' => $target_table,
    

    I know this is probably an unsolved problem, but I just want to rise the problem that you might get translations by this join.

  3. +++ b/dynamic_entity_reference.views.inc
    @@ -98,3 +102,150 @@ function dynamic_entity_reference_field_views_data(FieldStorageConfigInterface $
    +        // Relationship (Origin -> Target)
    

    I really like this docs!

  4. +++ b/dynamic_entity_reference.views.inc
    @@ -98,3 +102,150 @@ function dynamic_entity_reference_field_views_data(FieldStorageConfigInterface $
    +                // Entity reference field only has one target type whereas dynamic
    +                // entity reference field can have multiple target types that is
    +                // why we need extra join condition on target types.
    +                'field' => $column_type,
    +                'value' => $target_entity_type_id,
    

    Great documentation!

deanflory’s picture

When I attempted to apply provide_views-2548395-47.patch to the latest 7.x dev 2015-Aug-19, there were missing files that the patch would not create on it's own, nor could handle if a blank file of the name it needed was put in the module folder, then there were a bunch of yml files that couldn't complete either.

Notice this is with #47.

Same with #49.

patch < provide_views-2548395-49.patch
can't find file to patch at input line 5
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/dynamic_entity_reference.views.inc b/dynamic_entity_reference.views.inc
|index 7cb6de8..08133ba 100644
|--- a/dynamic_entity_reference.views.inc
|+++ b/dynamic_entity_reference.views.inc
--------------------------
File to patch: dynamic_entity_reference.views.inc
dynamic_entity_reference.views.inc: No such file or directory
Skip this patch? [y] n
File to patch:

dpi’s picture

This is for 8.x

jibran’s picture

StatusFileSize
new136.59 KB

Reroll after #2542274: Write tests for DER field as basefield on an entity.

@dawehner thank you so very much for the review. It was totally worth waiting for :).

  1. I tried it but it didn't work for cardinality > 1.
  2. Nice to know we'll keep in mind. Is there any core issue for this? Or any workaround for this?
  3. @dpi has done that so @dpi++
  4. :)

  • jibran committed 41be498 on 8.x-1.x
    Issue #2548395 by dpi, jibran, dawehner: Provide views relationships for...
jibran’s picture

Status: Reviewed & tested by the community » Fixed

Pushed to 8.x-1.x.
Thank you @dpi for working on the initial patch. Can you please create the followup issue for #24, #25?

Thank you @dawehner for all your help in IRC as well.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.