To give a clear picture, i'm condensing this in a single issue, but we can later close this and make separate issues.

I like the idea of makeing a next generation usage service in contrib. IMHO this module has several premature optimaziations, let's remove them:

1. Use full references

Only having EntityType+ID makes tracking of revisions and translations a nightmare. If we reference every instance (EntityType+ID+Revision+Language) a lot of stuff that currently is difficult (think paragraphs) will be easy.

2. Make usage an entity

We can make a UniversialEntityRevisionReference (EntityType+ID+Revision+Language) reference field and leverage that as 2 base fields in a usage entity. Of course I'm eager to learn about possible drawbacks. But we'd get an awful lot of stuff for free.

3. Use a separate service for transitive closure

Transitive closure is a nontrivial but well understood thing. Which means translating a set of relations "A is father of B" to "What is the set of offspring of A". The only conceptually fesible way to solve this is: Use a separate service for transitive closure. We can steal from Taxonomy Edge (D7) and Tree framework (D7) (the impressive architecture of which some people call over-engineered).

Comments

geek.merlin aka axel.rutz created an issue. See original summary.

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented
Related issues: +#3022265: Make it possible to get targets for a specific revision (e.g. current), +#3056562: Updating existing revisions doesn't purge previously created usage, +#3007752: Entity usage list does not scale, +#3056026: Remove orphan handling of paragraphs when parent exists, +#3002332: Track composite (e.g. paragraph) entities directly and only as their host, +#2949952: Make it easier to retrieve full relationship chain, +#3065291: Usage count does not match the number of entities, +#3015287: Allow usage records to be registered in background, +#3075249: 100s to 1000s of revisions cause a timeout on batch update.

Some issues that i think this solves.

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented
Related issues: +#3042418: Layout Builder created blocks do not clean up when removed from its layout.

And a related issue about inline blocks' equivalent revision issue.

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented

TBH, i groked that having transitive closure is needed, but i did not dive so deep as of why. Maybe someone can elaborate that.

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented
Issue summary: View changes
marcoscano’s picture

marcoscano
Primary language Portuguese, Brazil
Location Barcelona, Spain
commented

Thanks for contributing!

I'm not sure I understand all your suggestions though.

Only having EntityType+ID makes tracking of revisions and translations a nightmare. If we reference every instance (EntityType+ID+Revision+Language) a lot of stuff that currently is difficult (think paragraphs) will be easy.

Can you elaborate on this? We are indeed currently tracking type+id+revision+language, so I'm not sure what the suggestion here is for.

We can make a UniversialEntityRevisionReference (EntityType+ID+Revision+Language) reference field and leverage that as 2 base fields in a usage entity. Of course I'm eager to learn about possible drawbacks. But we'd get an awful lot of stuff for free.

Here too, I'm not sure I understand the suggestion... the current approach relies on the idea that the usage API is agnostic about which relationships should be "trackable", so any contrib or custom module could just implement the plugin, and a new relationship type will be tracked as well. Are you suggesting we drop this and favor a unique relationship method in all cases?

Use a separate service for transitive closure

The idea of #3060802: Refactor module architecture in a simpler, opinionated and more performant approach is exactly to be more opinionated and simplify what level of reporting we can extract from the data this module generates. Trying to solve everything for everyone is indeed complex, and at this point I'm advocating to identifying the problems users are most interested in getting from this module (i.e. probably the basic use cases), and focus on that. Maybe we can move the conversation about this point to that issue?

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented
Issue summary: View changes

Thanks for elaborating!

And a big sorry: When i cross-checked, i had the 1.x branch checked out in my IDE... Which makes my statement "1. Use full references" confusing (but not invalid).

But i'll start elaborating with 2), 'cause this is the most minor:

>> 2. Make usage an entity
> Are you suggesting we drop this and favor a unique relationship method in all cases?
Oh i think that's a misunderstanding.
In other words: IF this is rewritten anyway, we might consider this: Instead of a naked table (declared in .install file), create an entity with 2 suitable reference base fields (in src/Entity) (Yes we'd have to create that universal entity too...)
But as this is only about "The code may become simpler and cleaner", i'd say it's minor.

>> 3. Use a separate service for transitive closure
>The idea of #3060802: Refactor module architecture in a simpler, opinionated and more performant approach is exactly to be more opinionated and simplify what level of reporting we can extract from the data this module generates.
Yes i read that issue and patch, but i think the direction there is different (e.g. i don't think we should drop plugin/field columns).
To state it differently:
* ONLY track direct usages
* IF indirect usages are needed (e.g. node / 2nd-level-paragraph), use a separate table and service for that

>> 1. Use full references
> We are indeed currently tracking type+id+revision+language

Yes i see, in 2.x this is better. We have
- source: ID (int/string), type, revision, langcode
- target: ID (int/string), type

I'd say we need revision and langcode for target, too:
- langcode: There are usage scenarios (at least entity embedding) where source and target langcode differ
- revision: We can not reliably use this for garbage collection if we do not track target revision. (For the simplest case, think of paragraphs, add multiple source/target revisions and then delete some source revisions.)

(This will also make views integration and lists easier: We only add relations to the *revision* tables, views knows how to relate to the default revision.)

HTH!

marcoscano’s picture

marcoscano
Primary language Portuguese, Brazil
Location Barcelona, Spain
commented

In other words: IF this is rewritten anyway, we might consider this: Instead of a naked table (declared in .install file), create an entity with 2 suitable reference base fields (in src/Entity) (Yes we'd have to create that universal entity too...)

I'm not sure how this would simplify things?

To state it differently:
* ONLY track direct usages
* IF indirect usages are needed (e.g. node / 2nd-level-paragraph), use a separate table and service for that

Maybe we are not talking about the same things... but in the current version (2.x), all direct usages are tracked (and only those). How would the new system be different then?

I'd say we need revision and langcode for target, too

Would you mind pointing out how that could be possible for the fields we support nowadays? Most of them (like entity_reference fields, link fields, HTML links, etc) don't have any information about target language/revision.

Maybe an easier way to move the discussion forward would be to do that around some more specific code? If you were willing to put together a POC with these ideas, I would be more than happy to review.

Thanks!

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented
Status: Active » Closed (works as designed)

Yes you are right, i should let code speak. I'll close this for now and make a dedicated issue or add to an existing one. Thanks!