Problem/Motivation
I've spent some time playing with D8's entity query. One of the really nice things you can do is add conditions on fields on referenced entities (via the entityreference) field.
As an example: User has a field additional_info
which is a Node
with a field date_of_birth
. With entity queries, you can then:
$query = \Drupal::entityQuery('user');
// Filter by role.
$query->condition('roles', 'administrator');
// Filter by date of birth
$query->condition('additional_info.entity.date_of_birth', '2000-01-01');
This would return any administrators born on the 1st Jan 2000.
Obviously, dates of birth of users are much better stored on profiles. However, as there is no field on the user linking to the profile, it is impossible to do queries like this without using direct queries and building all the joins yourself.
In addition to this, a field would allow you to access the information via a user object:
$user = \Drupal\user\Entity\User::load(1);
$date_of_birth = $user->additional_info->entity->date_of_birth->value;
Both of these would be very advantageous, especially in the context of CRM where there is a lot of data about users and you need to be able to access and query it easily.
Proposed resolution
There a couple of routes we could solve this:
- A. Switch to storing the link between users and profiles in an entity reference field on the user
- This could result in lots of profiles and an inability to tell the difference between active and inactive
- B. Store a cache of the active profiles in an entity reference field
- This would provide an easy way to access active profiles, and we can complete ignore everything else (as we probably should
I would be in favour of B, though I mention A as I know there is a more general discussion around the active/inactive.
Remaining tasks
Decide if we're happy with this proposal and, if so, which route we want to go.
API changes
You would be able to access active profile information via the user object and query it as part of a user entity query.
Data model changes
Additionally storing a cache of the existing information in an entityreference field on the user.
Comment | File | Size | Author |
---|---|---|---|
#46 | interdiff-2654980-45-46.txt | 3.44 KB | manuel.adan |
#46 | storing_the_active-2654980-46.patch | 17.15 KB | manuel.adan |
#45 | diff-2654980-40-45.txt | 689 bytes | manuel.adan |
#45 | storing_the_active-2654980-45.patch | 16.02 KB | manuel.adan |
#40 | storing_the_active-2654980-39.patch | 16.02 KB | mglaman |
Comments
Comment #2
mglamanWhy not use the methods available the the Profile entity's storage handler?
This allows me to load multiple based on the owner of the user by type. Or just run the query on the profile entity storage, with the user being one field condition and value another?
Edit: Example query
Comment #3
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedSo the massive win of storing a cache in an
entity_reference
field on the user is that you can use it as part of a general query of users and across multiple profiles. Say you have 2 profiles, one of contact details and one of membership and you want to be able to find all users of a certain age who have an inactive membership but logged into the site in the last month. Theentity_reference
cache of the 'main' profile(s) allows you to do that in a single query.This stuff is a massive win for CRM stuff where you have a large number of field, potentially on multiple profiles and need to be able to get information about them. I know you can build reports in views, but some times you need to be able to access this information for programmatic purposes rather than just reporting, at which point entity query comes into it's own.
Comment #4
mglamanOk, now the use case makes sense. As a way to basically have a condition on fields across two different profile types. How would you control access to the field on the user, or proposed solution? I'd assume this is something that should not be editable in the UI as it is automatically populated as a backreference.
Also, should this be something the module itself provides, or implementations that require it - such as the CRM contributed module.
Comment #5
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedView access would apply the same conditions as regular view profile access, though via field access. Editing wouldn't be possible.
Very happy if you'd rather it went into CRM. Thought it might be a useful addition in general, though obviously CRM is the main winner.
Comment #6
mglamanSo I find the idea great. However, I feel it brings about a lot of questions:
I think it'd be great to see this incubated in a CRM module, with your use case. Something that implements the presave hook to add the profile reference to the user, adds the field to user entity, etc.
I'm moving this to Postponed because I would love to come back to this later once the implementation has been flushed out. I'd even like to link to the CRM module's implementation as an example. I'm always in IRC, feel free to ping me if you would like!
Comment #7
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedAttached is a patch that does it how I would suggest it is done in profile. Bits that are good about it being in profile as opposed to elsewhere:
profile_user_view()
andprofile_entity_extra_field_info()
as these are handled by theentity_reference
field. The ER field also gives more options (title display/render move/display mode etc)ProfileType
create/delete operations can be dealt with inProfileType
rather than hooks (a bit cleaner)Profile::postSave()
/Profile::postDelete()
rather thanhook_entity_insert()
/hook_entity_update()
/hook_entity_delete()
, which is cleaner and less to implement (::postSave()
is insert and update).Profile
and thereforeProfileInterface
and so be relied upon by other modules.Having said that, I'm going to leave this here and convert it to live in Decoupled User Authentication (a CRM module) - #2658264: Store active profiles in ER base fields on the user. If in the future you decide that you do want this in Profile, could you post to that issue/create an issue on decoupled auth to remove the functionality.
If you've got the motivation - would still gratefully receive any review/feedback on what I've done/how I've done it!
Comment #8
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedActually, in doing the port, we may not even need the
hook_install()
as the profile types get inserted...Comment #9
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedFYI - have just committed this to decoupled auth - #2658264: Store active profiles in ER base fields on the user.
Comment #10
mglamanandrewbelcher, I've been bouncing back and forth on this a lot lately. Has the query and simplified rendering been a big win?
Comment #11
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedThe querying particularly has been very helpful! My use case is CRM, where we have lots of information about users that we need to be able to query on!
Also, having the fields provides easy access to the profiles from a user entity which makes programmatic changes much simpler than having to query profiles.
Comment #12
mglamanOkay, I think I'm going to move forward and commit this. Does it make working with Token easier? RE: #2785585: Make profile fields avalaible as tokens
Comment #13
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedI haven't tried doing anything with token, but I wouldn't be surprised if it's easier on the basis of navigating user -> profiles is much easier.
Comment #14
Anonymous (not verified) CreditAttribution: Anonymous commentedAny update on this? I would love this being committed.
Comment #15
mglamanVic Luijkx, it's been on the back burner. I'm considering it to be a possible beta blocker. It seems like it'd simplify a lot of developer experience and possibly our own code.
Comment #16
mglamanOkay, I think we should move forward with this. It makes rendering profiles on the user much more customizable. Removes hook_extra_field_info which just stinks.
The only "bummer" is for multiple profiles there wouldn't be an easy way to just get the default when there are multiple profiles.
Comment #17
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedIf we implement our own field type extending entity reference we can override some of the methods to give us that functionality, though we will have to check that doesn't mess with the query integration as its possible that's hardcoded to the field type!
I'm happy to have a look at that and pick it up, though I'm fairly busy right this minute!
Comment #18
mglamanI'm okay with using the Profile storage to load the default. Most use cases probably won't allow for multiple profiles.
Comment #19
rlmumfordUpdated the patch so that it applies and added some test coverage making sure that the fields get created and removed with the ProfileTypes and making sure that the field points to the default item.
Comment #22
Anonymous (not verified) CreditAttribution: Anonymous commentedPatch #19 gives the following error:
Comment #23
Anonymous (not verified) CreditAttribution: Anonymous commentedSorry, my mistake. I did not realise I was using alpha5 instead of the dev version. I just tested patch #19 which works great on sites without the profile module already installed and configured. However, running update.php after updating the module causes the following error:
Comment #24
heshanlkFix the unit test issues.
Comment #27
heshanlkComment #30
heshanlkComment #33
csedax90 CreditAttribution: csedax90 commentedIs there any news about implementation?
Comment #34
mglamansedax, the main hold up is how do we handle profile types which allow multiple references? Or in that case do we just not support the referenced field?
Comment #35
niknakAny new patch is ready ?
Comment #36
csedax90 CreditAttribution: csedax90 commented@mglaman I think the best thing is to have a multiple user reference, maybe recalled through the profile ID
Ex.
[node:author:profile:customer:field_first_name]
Comment #37
mglamanbojanz had an idea: can we use a computed base field? This way we avoid the storage and can still manipulate via an entity query. Would simplify populating backreferences to profiles on user accounts.
Comment #38
mglamanI just read https://www.drupal.org/docs/8/api/entity-api/dynamicvirtual-field-values....
My thought is:
profiles
computed fieldComment #39
mglamanHere's an updated go using a computer field. It's much nicer than using extra fields and allows granular control. Still, needs some more work. Also, need to experiment with querying. It adds a calculated entity reference field for each profile type.
Here's an example of configuring user display and output.
And since it's a calculated field, we should only need to clear the entity field manager discovery cache, no storage changes, to make Drupal aware of the fields.
Comment #40
mglamanThis patch fixes chaining with the computed field. So something like
$user->customer_profiles->first()->entity->address;
works as if a normal entity reference. Also adds a test.Still need to figure out the query, if possible. Ideally if this doesn't work out for the query, it solves the whole extra fields issue.
Comment #41
mglamanYep. That does not work
Despite all efforts, this happens when compiling query conditions, which is way before altered is allowed. Looks like we'll need to use non-computed field and update the user whenever a new profile is created. Not horrible.
Comment #42
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedSo we've been running with essentially the version of code from #7 in Decoupled Auth module. I've just found an interesting issue with it though! There is a race condition, which can result in the values getting stored and then cleared.
My specific case is the registration form where we've added a profile field and a submit handler. Due to #2932258: Split notification out of RegisterForm::save we are saving the account/profile before
RegisterForm::save
runs so our profile is available for tokens in the notification email. However, due toRegisterForm::$entity
being a different object to the user returned fromProfile::getOwner
(as it's reloaded), the fields are being set by the updateProfileFields, but then getting cleared byRegisterForm::save
... D'oh.Possible solutions are:
1) Be very careful about order of saves - seems very fragile so although it may solve my immediate issue, I don't like it as a general solution...
2) Use computed fields so this never arises - requires solving the querying issue...
3) Always update the profile fields on user presave - potential performance issues...
4) Possibly some kind of flag on the field so we can detect if a change has been made by something other than updateProfileFields and, if so, re-run it at user presave - like above but mitigating the performance issues a bit...
5) Possibly use a service that will statically store the combinations, which can then be called as part of user presave - another possibly approach to improve performance or a user presave...
Comment #43
daggerhart CreditAttribution: daggerhart as a volunteer commentedGetting caught up on this issue and I think the idea of improving DX from the user object perspective would be great.
Concerning the storage of references to profiles, I do something similar on a D7 site but with an additional level of abstraction. I automatically create a "meta" profile for each user that they cannot access, this is where I store the references to each of their other profiles by profile type (each profile type is its own entref field). Maybe it's worth considering doing the same here?
If we use a profile to store all the references to other profiles, then the only thing we would need on the user to achieve the DX goal would be a single entref field to the user's meta profile. This would allow for similar api improvements requested in the issue description, and would also avoid the need to save the user due to profile changes since the user would only ever need a single reference to their "meta" profile that should never change.
Pseudo usage based on issue description, the user to meta profile entref field is named "profiles" in these examples:
I'm sure there are drawbacks to this approach, but can't speak to them knowledgeably from a performance perspective. Each query from a user to their profiles would require an additional join is an obvious one.
Comment #44
jwilson3Matt's patch in #39, basically fixes the issue I reported here: #2946670: Add computed {$profile_type}_profiles fields to users, use them to render profile data on the user profile regardless of the other issues here, I'd love to see that patch get comitted, if for nothing more than to have real widget configuration abilities on user field display.
Then the *proper* solution here could back out that code and replace it with something else, if it ever gets written.
Comment #45
manuel.adanPatch #40 no longer applies on latest -dev because of #2930966: Fix phpcs errors (short array syntax, other tweaks). Ported and working on it. Interdiff doesn't work, attached a simple diff file.
Comment #46
manuel.adanWith #45 (actually #40 ported), I got an exception in the user register form (with embed profile):
That happened for new users that have no user ID yet.
I also found that the new profile reference field looks like empty when accessing though the API. For instance, the isEmpty() method returns TRUE despite the user as one or more profiles. I reviewed the field item list, now:
Not sure if setters should be overridden as well, since it is a read-only field. It might produce some side effect. At the moment, changes performed in the field value simply are not persistent.
Comment #47
manuel.adanComment #49
jonathanshawI think that some of @manuel.adan's improvements might be no longer needed (according to https://www.drupal.org/docs/8/api/entity-api/dynamicvirtual-field-values...) because #2392845: Add a trait to standardize handling of computed item lists was committed to D8.4.
@mglaman said in #41 that in order to get querying to work, "we'll need to use non-computed field and update the user whenever a new profile is created." (which was @andrewbelcher's original proposal.)
I propose that we do two things:
1) add a base field with storage of all profiles, to help querying, which should be this issue.
2) add a configurable computed field that references a single profile (defined in the field settings). Because it is configurable, sitebuilders can add multiple instances and therefore control more finely which profiles are shown in a particular display and in which sequence. This should be a separate issue. We might even be able to figure out a widget for it to work on forms.
Comment #50
joachim CreditAttribution: joachim as a volunteer commented> A. Switch to storing the link between users and profiles in an entity reference field on the user
> B. Store a cache of the active profiles in an entity reference field
Option C: Allow entity queries to go via backreferences.
I've got a hunch there's a core issue for that...
Comment #51
joachim CreditAttribution: joachim as a volunteer commentedBingo: #2975750: Allow entity queries to query reverse relationships.
I was hoping there'd be a patch there, but sadly not.
Comment #52
andrewbelcher CreditAttribution: andrewbelcher at FreelyGive commentedI think the answer is to support entity query via computed fields, which #2962457: Handle computed fields in entity queries: throwing a helpful exception is better than a PHP fatal error talks about as a follow up.
Comment #53
bojanz CreditAttribution: bojanz at Centarro commented#2946670: Add computed {$profile_type}_profiles fields to users, use them to render profile data on the user profile has introduced a computed field per profile type. No need to do anything here.