Problem/Motivation
Note: this may not be the best place for it, but I'd hate to make such a small module for this.
Nearly every Drupal site that I work with has entities which are imported from an external source and I have to leverage some sort of mapping. Many times this is done via Migrate and it's mapping tables. Sometimes it is not and I end up following the same trick: entity field to store the remote ID and storage methods to find synced/unsynced entities.
I'd like to build an API for finding entities based on a source identifier.
Proposed resolution
- Create a new field type that so that the schema can have an index set. It should also have two properties so that we can reference the source ID and a version, if the remote source supports versions/revisions. This way local entities can be revisioned alongside the source data. The source ID is the main property, so it can work without the version value.
- A new entity key called
sourceto specify the field which stores the source ID. - A new entity repository service that allows loading an entity based on a source ID, just like loadEntityByUuid. Except it uses the source key.
Remaining tasks
Get a "go" or "no go" of scope for this in the Entity module.
User interface changes
None.
API changes
Addition of a way to look up entities based on a source identifier.
Data model changes
None, since it's an opt-in process for developers.
Issue fork entity-3269526
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
mglamanBased on #3255187: Drop support for Drupal 8 this should go in its own module. Maybe
entity_remote_sourceComment #3
mglamanComment #4
aaronmchaleIf such a module is created, please link it here, I may be interested in using it.
Comment #5
mglamanIt'll definitely be created, I just need to think of a proper module namespace for it. I don't want it to be
entity_remote_sourceas it implies that it may do more than provide a "pointer" API for remote associations. I currently haveentity_source_idbut that's not exactly settled. I have it locally, tested, etc. I just need to be happy with a name and push it.Comment #6
aaronmchaleI'll have a think as well, maybe ping you on Slack if I come up with any good names 👍
Comment #7
mglamanAwesome! I'll ping you as well if anything comes to mind.
Comment #8
mglamanI'm re-opening this and going to push it forward here. I'll have an MR next week.
Comment #10
mglamanOkay, I've pushed the code and tests. Tests give us 100% reported coverage for the field type and repository.
I've added a default widget which allows to read-only of the field values, for debugging purposes. I need to create a field formatter. But it's ready for review.
Comment #11
mglamanI'm generating 100,000 nodes with a UUID field and this field to provide some query performance notes.
Also, should provide a generate sample values module
Comment #12
mglamanBase fields added to nodes
UUID field item
SourceItem
Comment #13
mglamanSince code statements wrap and don't overflow, I've added the data into the MR https://git.drupalcode.org/project/entity/-/merge_requests/11
Comment #14
mglamanChatted with @eiriksm. He brought up shoring up the testing more.
For instance, let's say someone already uses a textfield with modified schema to have an index and then decides to use this field. We should verify the following code still works:
Since we have the main property name set, it should just work. But let's get that tested!