Card Sorting survey

Motivation/problem

We need to collect data around how users categorize options in the Administration User Interface main navigation.

The purpose of this issue is to request participation in a card sorting survey. The survey is anonymous and will take approximately 5-10 minutes to complete. Your feedback will help us make decisions about restructuring the admin toolbar based on your mental models, rather than our assumptions.

Link to the Card sorting survey: [Card Sort closed]

After you have completed the survey, add a comment to this issue if you'd like to get credit for participating and helping us with your answers.

Feel free to add any kind of feedback on the comments, as well.

Card sorting survey results

Card sort background

The Admin UI initiative organized a card sort in order to understand how Drupal users organize the various links that are present in the existing Drupal core administration menu. The call for participants took place on social media and in the Drupal Slack. The sorting was completed by each participant asynchronously. Participants in the card sort were members of the Drupal community and who volunteered. The cart sort coincided with Drupalcon Pittsburgh 2023, and some participants were able to do the card sort during the Drupalcon general sprint. Participants were asked to self-identify their role using Drupal with the question “Which of these roles would describe better your interactions with the administration UI?”. More than half of the participants selected “Back-end developer / module developer” with 123 responses.

The card sort was set up using Maze. Cards were pre-populated with link names from the existing administration menu. Participants were asked to review the cards, create category names that they would use to organize similar cards, and then group cards into those categories.

The card sort received 202 responses in total. In order to analyze the results, we manually reviewed the participant’s groupings, and combined them with similarly named groups from other participants.

Participants grouped cards by similar types of tasks that jobs or roles do for example “Site builder” or “Developers” or by similar types of things, but which cards belonged in which categories were not definitive overall. For example the top three cards that appeared in the most number of different grouped categories after condensing similar categories:

• “Contact forms” appears in 15 categories
• “Search and metadata configuration” appears in 13 categories
• “Content authoring configuration” appears in 13 categories

The clearest category by sorting agreement to emerge from the sort relates to “user management” for people or user-related links.

Card sort results

After condensing the results, three groups emerged with a high average agreement rate. From Maze: “The agreement rate is the percentage of users agreeing that a card belongs in this category.”

The “User and Account and roles and permissions” category with its top five cards:

• Create a user account
• People list
• People permissions
• Roles
• Role settings

The “Content and content administration” category with its top five cards:

• Content list
• Create content
• Comments list
• Media list
• Files list

Noting that the “Create a taxonomy term” card and “Taxonomy terms” card also had strong associations with the “Content and content administration” category.

The “System logs, reports, status, settings, maintenance” category with its top five cards:

• Recent log messages
• Status report
• Top “page not found” errors
• Help
• System configuration

The “Regional settings” card also had a strong association with the “Content and content administration” category.

There were participant categories that were not straightforward to group, that in order to perform analysis we grouped as “Ambiguous”. The “ambiguous” groupings included:

• All
• Auth
• Code related
• Meta
• More
• My stuff
• Organization
• See all the things
• Tech stuff
• Technisch/dev (technical)

There were also participant categories that indicated things to remove, which we grouped as “Remove suggestions”. The “remove suggestions” groupings included:

• “Remove from admin”
• “Things that scare the editors”
• “Does anyone actually use these?”

All groupings are available on the card sort report in Maze.

Additional feedback from participants

After the card sorting portion, the survey included the question “Were there any categories or groups of cards you were unsure about?”. 65 participants provided a response. 17 of the 65 participants who responded to the question gave a “No” or blank response to the question.

Some of the participants included responses that illustrate the wide variety of sorting approaches:

• “Menus and blocks because they are sometimes content related and sometimes directly site structure related (their audiences can vary)”
• “yes, of some I am not sure what their purpose was (even with the extra information). Therefore I put them in a bigger group together.”
• “The line between development and site building is easy to blur.”
• “There are very narrow lines between admin settings and content authoring, content authoring and structure, and not sure if I should have done more groups but tried to keep them as low as possible.”
• “There were some items that I wasn’t 100% sure I understood what they were used for.”
• “Menus and roles don’t feel as substantial to the content of a site as taxonomy, but theoretically they’re more related than, say, a tags vocabulary and comments. I tried to differentiate between “user generated content entities” and “administrator managed things, like menus and roles.” Search and metadata feels out of place... mostly because it feels like a content type definition concern.”
• “First I was thinking to group them by configuration entities vs content entities, but for example menu items can be created from UI, views, or yaml files, so as soon as I start to add the other items, then it’s not that easy”

“Dashboard” was a card that stood out that several participants mentioned in response to the question:

• “dashboard, maybe a separate item?”
• “Dashboard should not be there it should be the main page of the admin screen”
• “Multiple dashboards? Comments list missing? Several menu cards sounds similar”
• “Yes - dashboard”
• “Help and Dashboard feel icky…”
• “The one I put the dashboard and help in. It’s a bit pointless, but they don’t go with the other things.”
• “I wasn’t sure where to put Dashboard or Comments List or Extend. I don’t know if Extend is needed, actually, unless it’s a top level item. As for Dashboard, I don’t know that would be for, or is it different depending on what role you have?”
• “Dashboard, seeing lists of things. I created the “all” group late in the process and might have put more there”
• “Dashboard and help could possibly be on their own”

Participants indicated a desire for smaller groupings or for better organization:

• “would have created sub-groups definitely.”
• “Content currently contains a lot. That might need some extra layers to make everything manageable.”
• “mostly concerned with the length of certain categories, but “nesting” should be able to sort that (create X doesn’t always have to be a direct option, like for menu links for example).”
• “I think there are also opportunities for subcategories.”
• “Maybe the site structure one contains too many items and could be divided into 2 groups.”
• “Wanted to make subgroups, so create and list content are grouped in the my site group.”
• “Generally thinking that the “Structure” menu should go away, that the items in the current structure should live next to the things they define the structure for. And make the things an author would look for appear at the very top – Content, Media, People, Structure (structure being content structure like Menus and Taxonomies instead of content types/etc).”
• “Content vs Structure. Some this are in a grey zone. Also the People settings, should they go together with the other people stuff, or should all settings (including permissions and role etc) into the settings group instead of the people group (now that I think about it I think I prefer the user settings in the settings group, so the people group is focused on creating and managing users.”
• “Only in a nesting structure. Wherever there is a “list”, the appropriate actions should display alongside that - for example - content list, should be a direct parent of “create content” (and other options). There should be a single “content” link that then delivers all appropriate functions within that. Organised by sensible roles – ie. Content creation vs. Content architecture vs. Structural element creation (custom block/views)”

Some participants called out the need for updating naming:

• “I got a little stuck thinking about current naming in the Drupal admin interface, or in the Drupal community as a whole. I think “Features” is a very human word to use when describing what things your site is able to do... but because that word has history in the Drupal world, I hesitated to use it for the modules/updates category... However, if I start without presupposition, I’d use words like “features” and “structure” to categorize some of these groups, despite their current history and usage in Drupal.”
• “Views” is tricky. Might be better in structure, or called “Content Lists”? There wasn’t much related to layout builder but perhaps “Layout and Lists” is a good category. Also - please, let’s kill the “Configuration” category! It’s so broad.”
• “The “Meta” name is not great, and it groups a lot of stuff.”
• “Better name for “extend”.”