[META] Support asset hierarchy

Current status: I have created a parent entity reference field on all asset types in the 2.x branch, which includes Plant, Animal, Sensor, Equipment, Group, and Compost.

In my areas2x branch, I'm working on adding additional asset types including: Land, Structure, and Water (which will be migrated from the area taxonomy terms, see #2363393: Make "Area" into a type of Farm Asset). These asset types will also have the same parent field.

Adding #3151233: [META] farmOS 2.x Data Model as a parent issue.

This has a few aspects to it.

One is the data model itself: the entities and the fields that relate them. We pretty much have that done, with the parent field.

The other aspect is UI/UX...

For most asset types (eg: Plants and Animals) we don't need much more than that. We just want to be able to see that "this animal is the parent of that animal" which is easy to do with the Asset Entity View Display, and with Views (as we are currently doing in 1.x).

I think the bigger task here is replicating (and improving upon) the features that we currently have in 1.x, where areas are taxonomy terms. Taxonomy terms already have hierarchy, and Drupal provides some basic tools to manage that like the "Relations" field in the term edit form (for selecting parent and weight), and the click-and-drag hierarchy editor (eg: /admin/structure/taxonomy/farm_areas, which we link to from /farm/areas to allow changing the hierarchy).

The "click and drag interface for changing hierarchy" (for "location" assets specifically) is therefore a minimum requirement for this issue.

But within that, we need to consider how to make that interface more scalable generally. We have some future use-cases that will have 10s of thousands of location assets.

So perhaps we need to think about making our own UI for organizing hierarchies - which limits the depth of the hierarchy that is included in the form. So rather than organizing ALL location assets in the same form, you would only be working with sub-sets at any given time.

The other missing piece here is "weight". Taxonomy hierarchy has two fields: parent (for defining the parent entity) and weight (to define how the term is sorted among other children at the same level). We have a parent field in 2.x - but we don't have a weight field.

And honestly, I wonder: do we need weight? I don't think we really do. Organizing alphabetically 100% of the time seems perfectly reasonable for every use-case I've encountered thusfar, and I would be OK with making that an official decision, to keep this simpler. In fact, I often find myself using the core taxonomy "Reset to alphabetical" button in 1.x.

And honestly, I wonder: do we need weight?

Thinking more broadly about this... even if we DO have use-cases where custom ordering is required, I'm not sure that the weight should be stored on the asset itself. It might make more sense for it to be saved separately. Consider the case where one asset has multiple parents. With a single "weight" value, you can't control the order of that asset among the children of both parents separately - which sort of breaks the whole concept of ordering altogether.

This is an existing limitation in Drupal core as well. Terms can technically have multiple parents, but only a single weight. I've run into issue with this in previous projects, and there seem to be some long-standing core issues still open because of it (eg: #969194: Drag’n’drop only partially disabled when term has multiple parents).

So my thought is: let's forget about "weight" for now (because I don't think we have a clear need for it). And if we need it in the future, it should be managed separately from the asset entities themselves, which should ONLY reference the parent. "Weight" within hierarchy is really a separate feature from the data model itself, IMO.

It might make more sense for it to be saved separately.

It's reminding me of the Nodequeue module I used in D6/D7. ;-)

The comparison there is: nodequeues (intentionally ordered lists of nodes) are a separate concept from the nodes themselves. And one node can be in multiple queues with different weights.

Adding this module to consider as well: https://www.drupal.org/project/hierarchy_manager

It sounds like it is primarily focused on creating the draggable list feature, which is one of our primary needs. It works with the core Entity Reference field type.

Combined with https://www.drupal.org/project/entity_reference_tree (which provides a form widget), these two modules may cover our UI/UX needs.

My only hesitation is that they use a JS library that depends on jQuery (https://www.jstree.com/). That isn't a deal-breaker, but in general reducing dependencies on jQuery is a good thing moving forward (Jamie is actively working on removing Bootstrap and jQuery entirely from Field Kit).

But that said, maybe they would provide a good short-term solution with little effort. It might be worth putting heads together with Jamie about potential Vue.js-based approaches (perhaps more custom) - because in the medium term we are hoping to merge more of those UI efforts/components with Field Kit work.

The other two module (https://www.drupal.org/project/entity_hierarchy and https://www.drupal.org/project/entity_reference_hierarchy) also look interesting. I'm hesitant about the first one (entity_hierarchy) because it is providing its own field type, rather than using core entity reference, and I'm not sure we need/want that added layer on the data model level. And I'm not sure I understand what the second one (entity_reference_hierarchy) does. :-)

I'm hesitant about the first one (entity_hierarchy) because it is providing its own field type, rather than using core entity reference, and I'm not sure we need/want that added layer on the data model level.

Specifically, this new field type is extending the core entity reference and adding a "weight" field, which we don't really want/need: https://git.drupalcode.org/project/entity_hierarchy/-/blob/8.x-2.x/src/P...

And... it sounds like it only supports a single parent (probably for the same reasons described above regarding the "weight" and ordering conundrum under multiple parents).

Configure this as a single value field and give it the ability to reference the bundles that can be its parent.

-https://www.drupal.org/project/entity_hierarchy

Adding some conversation with @jgaehring here regarding "weight" for posterity/context:

jgaehring
what are some reasons a user might want areas to be weighted? I'm not too familiar with the feature in 1.x

mstenta
i can't think of any
it's already possible to order them manually in 1.x, simply because they are taxonomy terms and Drupal allows that
so we're basically wondering if we can say "no you can't do that anymore - just alphabetical" in 2.x
(i think it would be fine to make that decision - and would greatly simplify things - and could even be considered a usability improvement because the fact that sometimes things are NOT alphabetical is annoying)

jgaehring
seems reasonable to me
yea, that does seem annoying

mstenta
yea, and my thought is: if you DO need intentionally ordered lists for some reason... that's a separate feature... not part of the data model itself
(one of the big challenges is: have one weight with multiple parents doesn't actually make any sense - and that's a limitation of the taxonomy term system already)
(you really need a weight per parent)
(that said... multiple parents won't really be used for locations i don't think... but it will technically be possible)

jgaehring
yes, I had a similar thought that sorting should be part of the business logic, not part of the data model
the data model should just contain objective metrics that you can sort against
the concept of weight seems too subjective and variable

It seems that rather than having a complex field type that builds on top of an entity reference to facilitate us a hierarchy, we would like to keep the parent as it is and maybe have a widget and a hierarchical formatter for the parent field?

This sounds right to me.

And maybe a second question is: do we want to be able to manage the hierarchy in BOTH the individual asset context AND in broader contexts?

The former would be the responsibility of the widget, in my mind. So when you are editing an individual asset, and you want to modify where it lives in the hierarchy, you do so via the widget. This maps to the current "Relations" field in 1.x, provided by Drupal core taxonomy module (and woefully insufficient for managing large hierarchical lists).

The latter would be a standalone UI, akin to the "list terms" form provided by Drupal core taxonomy module for drag-and-drop management of the full hierarchy.

I do think we want both - to be able to manage hierarchy from the standpoint of an individual asset AND from the standpoint of the hierarchy as a whole (and/or from subsets of that hierarchy).

Seems that we need at least "special pages" where these hierarchies are represented

Interesting idea - can you elaborate what you mean/envision? Perhaps this is how we handle the latter feature (managing sets/subsets of the hierarchy as a whole)?

I can imagine a page at location/hierarchy (or something) which is a FULL hierarchy manager of all "location" assets, perhaps with options for filtering down to certain asset types (Land / Structure / Water) and other conditions.

This brings up an important distinction we should make: we really only want these "advanced" hierarchy management features for "location" assets. We do not need/want them for animal/plant/etc lineage management (I don't think). The difference, in practice, is that locations will have a relatively shallow hierarchy. Whereas the "lineage" hierarchy of living assets giving birth to children has a potentially unlimited depth (animal 1 begets animal 2 begets animal 3... and onward). Further, animal hierarchy won't change. Location hierarchy will. Location hierarchy represents the land owner/manager's mental model of the land - whereas lineage hierarchy represents ACTUAL parental lineage.

Those differences don't necessarily mean we need different fields - I think the simple shared parent entity reference field is still sufficient. It only changes how we need to treat different asset types in the UI.

How we draw that distinction is an open question. We could simply say "Land, Structure, and Water" asset types will appear in the hierarchy management features - others will not. But there may be contrib modules that want to add additional asset types to that set. Another option is using a field on the assets themselves (or on the asset types perhaps) as the filter. In my areas2x branch, I am adding a location boolean field to all assets (defaulting to TRUE on Land, Structure, and Water assets, FALSE on others) - so this could be used as the filter, perhaps.

How would we want to manage the hierarchies? Drag and drop is out of the question since we don't want weighted entities.

Good point about "drag and drop" and "weighted entities" being practically incompatible. I overlooked that. I suppose being able to drag horizontally (to set hierarchy) but not vertically (to set order) might be possible... but it would be tricky. Vertical dragging is a prereq for horizontal dragging - and that would create an implication in the user's mind that ordering is possible. If the order then changed to alphabetical after saving (while still maintaining the hierarchy), that might be confusing/frustrating - even if we make it clear that order will not be preserved. Still - it is an option. (I could also imagine some JS logic that automatically keeps the order alphabetical during drag-and-drop, while still allowing hierarchy change.... in theory... but the usability of that may end up be horrible in practice).

Does the editor need to access the parent to see the children? what about grandchildren?

This is a very good question also, as it relates to access control. We'll need to remember to consider this in any custom code we build for managing these hierarchies.

I can imagine a page at location/hierarchy (or something) which is a FULL hierarchy manager of all "location" assets, perhaps with options for filtering down to certain asset types (Land / Structure / Water) and other conditions.

Oh I forgot to finish this thought...

Beyond location/hierarchy, which has the FULL hierarchy, perhaps we can also have location/hierarchy/%parent which limits the management form to only include children of the specified parent - so you are only editing a subset of the hierarchy.

For small farms, with a small number of areas, they may only ever use location/hierarchy. But for larger operations, or country-wide area management use-cases with 10s of thousands of areas, maybe they only edit the sub-sets.

And in those very large use-cases, we may even need to disable/restrict the root location/hierarchy path, or risk memory_limit errors.

This has got me thinking... maybe we should just focus on this location/hierarchy/%parent idea as a first step. Having the ability to manage hierarchy from individual assets (via widget/formatter) might not be necessary if we have an easy to use location/hierarchy/%parent form.

The parent reference field widget is really just for selecting the parent - it doesn't necessarily need to have much more than that - so maybe https://www.drupal.org/project/entity_browser is all we need across all asset types. And maybe we do customize the formatter - but just to include a link to the location/hierarchy/%parent page.

Perhaps location/hierarchy/%parent is all we need for "minimum viable product" on this?

And in those very large use-cases, we may even need to disable/restrict the root location/hierarchy path, or risk memory_limit errors.

One simple idea here might be: only allow managing 2 or 3 levels deep. And to go deeper you must click on one of the nested items to load its children in location/hierarchy/%parent.

So location/hierarchy would show levels 1, 2, and 3, and let you arrange all of their relative hierarchies. But in order to arrange levels 4, 5, 6, ... you would click on one of the parents in level 3. Does that make sense? Essentially "drilling down" to manage sub-hierarchies - but never working with more than 3 layers deep at a time?

Hot question: Can the hierarchy be mixed?, if I have a lake in my property, I might want the water asset being in the hierarchy of a larger land asset.

Yes absolutely. The hierarchy will include assets of any type, potentially. Especially if we filter based on the location boolean field. Paul has talked about cases where he would like to make Plants into locations themselves - eg: with an Orchard (long-term crop spanning years/decades), rather than having two separate Land and Plant assets. In that scenario, the Orchard asset (a Plant asset type) would have the location boolean set to TRUE, and would therefore be included in the hierarchy editor.

I agree it makes sense to spin off sub-issues and prioritize them separately.

I would leave "dragging" out of the MVP until we really know what we want.

I think the "dragging" piece is arguably the most important - because it is what current users depend on 99% of the time for managing their hierarchy. So maybe we just need to dig into that a bit more and think about how it should work.

And in the larger use-cases, it may be even more critical. I'm not sure I can imagine using the individual asset widget to manage 10,000 areas.

Quick update: @jgaehring has some ideas for making the drag and drop work as a Vue.js component. And @pcambra is going to explore using Entity Browser for the widget (perhaps with a custom plugin).

So perhaps this will be a good opportunity to explore adding a Vue.js component to farmOS. We can add a location/hierarchy/%parent page that builds the markup, and load the Vue.js component in client side.

Oh just remembered: one thing we will need to prevent is circular relationships (eg: Asset A is parent of Asset B is parent of Asset A). This logic will need to be somewhere in farmOS. Perhaps we make a new farm_hierarchy module (and move the current parent field storage over there), which can add this constraint somehow. This should probably be split off to a separate issue.

one thing we will need to prevent is circular relationships

Created a child issue: #3179016: Prevent circular asset parent relationships

Split off some more standalone issues:

• #3179023: Location hierarchy drag-and-drop editor
• #3179024: Asset parent entity reference widget
• #3179025: Asset parent entity reference formatter

They are just stubs right now... might want to copy the relevant bits of this discussion into them (as well as some of the things we said in Matrix chat).

#3179023: Location hierarchy drag-and-drop editor will be merged soon...

I think we can close this. The important pieces of this discussion have been figured out/implemented.