One week before Drupalcon DC, we will be conducting a usability sprint on Drupal 7. The first half of the sprint will involve a round of formal usability testing at the University of Baltimore (UB) led by Becca Scolan, who produced the previous Drupal usability report from UB. She will be joined by members of the Drupal community: Drupal documentation team lead, Addison (add1sun) Berry; Key members of Drupal's usability team, Brad (beeradb) Bowman, Nathaniel (catch) Catchpole and Bojhan (Bojhan) Somers.

Previous usability testing at the University of Minnesota (UMN) and the University of Baltimore (UB) highlighted a number of issues with Drupal's usability and resulted in several solutions - some of which have made it into core, some of which are actively being worked on. This will be the first time Drupal 7 has been usability tested, and the first formal testing held while Drupal is in 'code thaw'. This means we have a unique opportunity to field-test these Drupal 7 improvements when we still have some time to make adjustments before release. Learn more in the "how you can help" section below.

Continuing this effort at this year at the University of Baltimore will allow us to make more informed design decisions.

During the second half of the sprint, the community members attending will analyze the test data and produce actionable issues to work on before Drupal 7 is released. Dries has predicted a release date for Drupal 7 some time in the fourth quarter of 2009. This means there will be time after the testing is complete to implement solutions to many of the issues discovered by the tests. All test results, including much of the raw data, will be made available publicly on the web and during a presentation at Drupalcon DC.

We are asking the Drupal community to contribute to funding accommodation and board during the sprint for participants, each of whom will be donating their time for the duration of the testing.

Dates

The testing itself will be conducted from February 25th-27th and evaluation from the 29th until the 3rd of March.

Costs

During our stay in Baltimore we will need financial support for the currently confirmed attendees. We have only a few weeks to raise this money in order to book accommodations in time. We've been able to secure significant help from employers and community members for airfare and lodging, but we still expect expenditures for testing subjects and the group of contributors to be around $2500 for the week.

Want to get involved?

Password strength checking before and after testing
The password strength checker on user registration was found to be a critical usability issue during testing of Drupal 6. Now replaced in Drupal 7.

There is still space for a handful of contributors. If you have a history of contributing, would like to come, and can secure sponsorship for airfare + board, feel free to contact one of the current attendees. While there's limited space in the lab, there's still plenty you can do apart from attending or donating money to help make the testing as successful as possible.

In preparation for the testing, we are actively encouraging the community to resolve major outstanding issues from the previous tests so that we can verify how effective the changes are, and to avoid treading the same ground testing interfaces we already know to be confusing. There are patches in the queue for some of these changes, so as well as donating some money for the sprint, you can also help us by spending time reviewing, updating and testing these patches and by following the discussions in http://groups.drupal.org/usability

Additionally, we'd love to get community help in reviewing and critiquing our testing plan. Our current working copy can be reviewed at http://groups.drupal.org/node/18579

Comments

webchick’s picture

We need to haul ass and get as many usability improvements as possible into Drupal 7 by February 24th. This is less than a month, which is not a lot of time.

The usability team has highlighted some of their high-priority items on the "hit list" here, and I will be focusing my review/commit time on these and other usability-related patches for the next several weeks. If you want to help make Drupal's usability skyrocket, come find us in #drupal. We'd love to brainstorm with you and get you started.

Kudos to you folks for organizing this. I think that we have huge potential for this to be a great success!

AdrianB’s picture

The ambitious and thorough usability testing did wonders for WordPress recently with version 2.7. Good to finally see this important issue getting the attention it really need in the Drupal developement as well.

SeanBannister’s picture

One of my main usability concerns has always been the "read more" link displayed at the bottom of a node teaser in the links section. The problem is it gets lost between a heap of other links and the user doesn't realize there's a "read more". You do need to have a few modules utilizing the links section for it to get lost so I don't think this issue comes up in usability tests because the Drupal configuration doesn't have a lot of these modules installed.

There's talk of getting this fixed in core but it's somewhat stalled and I'm having trouble getting my head around it.

gpk’s picture

>getting my head around it

This http://11heavens.com/read-more-link#comment-4487 has links to related issues, plus there's various related and unrelated debate further up on the page itself.

gpk
----
www.alexoria.co.uk

SeanBannister’s picture

Thanks for that, didn't know all of those issues existed.

jinlong’s picture

I'm looking forward drupal 7!!! GO!!!

---------------------------------------------------------------------------
Drupaluser.org---Sea water droplets together--Today, you drupal it?

RobLoach’s picture

I think that users should learn some of the terminology before stepping straight into Drupal. It's true that there is a huge learning curve, and part of that learning curve is understanding. It takes a long time to learn how Drupal works and how to use it correctly. I think if students watched Understanding Drupal before jumping in, they'll understand more about what they're creating during the tests............ Maybe just listen to the audio without any of the video screencasting segments?

ultimateboy’s picture

I would tend not to agree. The majority of new Drupal users will not have the luxury of watching such a video. So although it might better help the participants understand what is going on and the terminology being used throughout the Drupal UI, I think that their lack of understanding will show us more of what needs to be done. It has been a long standing issue to improve the help system and if attendees of usability tests such as this are struggling with terminology, it will be more apparent that ample time be spent on the help system, or reworking navigation in such a way as to guide new users to where they want to be. I think it is far more important to test on a complete novice than to give usability attendees knowledge that others will not have.

--
-- matt tucker
    pingVision

-- matt tucker

Pasqualle’s picture

I agree with Rob. Short introduction to Drupal is a must before trying to complete any specific task with it. And I believe it was a common practice on all previous usability tests.
Don't forget it is a 3 days test only. Anyone can learn Drupal without help, but nobody will be able to learn it from the inline help within an hour.. And even a better help system can't change that fact..

catch’s picture

For previous testing, participants didn't have any kind of induction before undertaking the tests, and IMO this is still the best approach. We're going to be testing people with previous experience of using CMS, and they'll be doing very basic tasks (posting content, adding a user etc.). This mirrors the sort of thing someone who's just downloaded Drupal for the first time might try having come across the site and decided to try it out - in those cases there's very often not the luxury of an induction/tutorial before they get started.

drupalisme’s picture

Compare to other CMSs, Drupal usability more difficult. Do you know why?

I found other CMSs supported by icon (image) which can describe more easier than text, especially for non-native English people.
For example: put a DISK image for SAVE action, or a PRINTER for PRINT action. And ofcourse, put a KEY for PASSWORD!!!

Drupal 7? Wow ... rock!

Pasqualle’s picture

Drupal is multilingual and actually most people can read, they just don't want to.. Any theme can add images to the text..

hgurol’s picture

If using menu icons instead of text (or menu icons along with the text) helps to improve Drupal's usability, which I believe it definitely will do, then this should be something to consider.

If it is just a theming issue then maybe D7 should release with a default theme as such and this would encourage the other theme developers follow the same track. If it requires development at the core then the team might consider, evaulate the issue and decide to go or not to go with it.

What kind of a funny response saying that most people can read but they are being lazy and not reading?!! Whats the main purpose here? Its obviously NOT helping to improve Drupal's usability.

Pasqualle, there is a word in my limited English vocabulary; its call "VISUALIZATION". You may want to google for it when you have time....

Pasqualle’s picture

There is no issue, it's done already. Any theme can add images to the text. If you need images then change the theme..
http://drupal.org/project/rootcandy
I disagree with the original poster, Drupal is not less difficult to understand with shiny images.. You need to read to understand the functionality, changing save to disk and print to printer icon will not help you..

dirkca’s picture

Pasqualle,

while the link you supplied is useful, the content and tone in your comments suggest a general ignorance towards users that learn and function differently than yourself. Drupal is a complete nightmare for visual learners. It is also hell for those whose native language is not English. (Yes, I know that one can change the system language).

A less exclusive tone in your public commenting might be something to think about.

Pasqualle’s picture

I am sorry, I mostly go berserk when I hear a bullshit like: Drupal is very difficult to use because it does not have pictures, and change the save button to disk image.
But I don't see which sentence made you think that I am ignorant towards users. I just stated the fact that most people do not want to read, and you can use images if you like. I also use the rootcandy theme because it looks nice, and I would like to see a nice admin theme in core, but the icons will not teach you Drupal, it will just make you feel that Drupal is nice..

dirkca’s picture

'most people can read, they just don't want to' and 'that is called laziness' .

Flying Drupalist’s picture

I hope the results of this testing will be packaged into a module for drupal 6.

misty3’s picture

I also feel that some prior understanding of cms should be there on the part of testers, probably it is already there.
I wish that there are innumerable usability feature and support requests like http://drupal.org/node/134657 which also get equal importance and a filtering or short listing done of the many such unsolved requests ( apart from regular bugs which are getting regularly crushed )

Were these testers building websites before ? Will they be building websites in future passionately ? Unless so the test results will have many false effects.

Since drupal is multilingual and comprises users from innumerable countries online usablilty tests will be also so welcome.

Since test results are test results based on "tests" and individual opinions will not count I hesitate to say but to many users ( and also if test is again conducted in another part of globe with another set ) the password on user registration before testing appears more OK actually and more logical and according to user's normal expectancy. One simple reason is that many sites that an user gets to visit follows that pattern, and its an expected pattern!

catch’s picture

All testers will have a degree of experience using a CMS beforehand - this is part of the recruitment process.

We're also hoping to get more online and informal testing going this year, stay tuned for more on that.

Francewhoa’s picture

I just made a small contribution $ using the ChipIn. Thanks UX Team.

---

How to contribute to Drupal.

Setol’s picture

In my view - Drupal got a lot of features and additional tools. The last thing - there's not good usability. If Drupal get it, it will be the best of the best

juampynr’s picture

Hello everyone!

I have just discussed within my company, www.ideup.com, about helping with further tests.

Webchick says that there are more areas of Drupal 7 to be tested by people who does not know Drupal.

I would like to know to whom I need to talk to in order to schedule a set of tests, date, recording formats and submission of results. We already have a usability lab here at the office so I think we have everything we need to start moving it on.