It would be cool for TWiki - on the TWiki.org site - to have a set of guerrilla usability tests
that people can self-administer, to check out existing and new features, locate problems, pinpoint exactly why something sticks...
Tests probably don't have to be formally constructed, simple logic and, uh, common sense should be fine. Like, loosely:
- SETUP: Intro to the test - why, what, how to record the results.
- OBJECTIVE: Starting with a designated topic, change some text and add new text, using TWiki notation to perform basic bold and ital text formatting. Spell out the task
- REPORTING: A fill-out form...
- Elapsed time?
- List of steps taken
- Brief overall comment
- Specific problems encountered
- FOLLOW-UP: optional further Qs/interview
Editing as a test might not turn up much - but you never know. A little more involved tasks, on the user side (ex: find five specific items of info in a web) and the admin side (ex: modify a template in a certain way), could produce really useful observations, with even a couple, and defintiely with five or six, participants.
If a standard set of tests developed, they'd be useful in configuring other TWiki installations, and an alternate way to intro people to using TWiki, at all levels. Collaboration platforms, and TWiki in particular, are prime for this sort of KISS testing.
- 26 Sep 2001
You seem to be talking about two things:
- Tests to help people learn TWikiMarkup
- Tests to check that an install is done properly.
- Test to see how easy it is to find things
- Tests to see how easy it is to use specific features
- Tests to see how easily and how often diffs are used
- Tests to see how well, say, three people can back-and-forth real-time to come to a conclusion in a fixed time
- etc etc - important to see what people are really doing, or trying to, and how well it's going... -- MikeMannix - 28 Sep 2001
I started putting together some for our site. I think if we all used the same checks this would really help people get going, document their install consistently. This would mean we could help them better to know what was missing from their install. Cool, when start? - MM
- 28 Sep 2001
checksheet is cool, it should turn up problems that can be corrected, until, in this case, the installation process is near enough foolproof. In "usability testing" as above, I was thinking in more individual, open-ended, micro terms. Kind of lie the reply chart you did with the instllation procedure in TWikiOnWebHostingSites
Basic tests could be less structured as far as steps, and require open-ended answers from the subjects. For example:
New user installs TWiki, gets it up, naturally wants to customize look-and-feel, thinks CSS
, of which there's nothing much in the bundled docs. So, comes back to TWiki.org to check Codev. Wants to: 1
Find a page that's called "CSS Overview", though that is not necessary the topic title; 2
Post a question - for simplicity, a fixed one: "*URGENT:* Really need to demo a CSS
template system by Monday. Can anyone help?"
Each test is a topic, outlining the task as above, giving some guidelins (ex: a precoded search field on the page; instructions like, "Spend no more than five minutes scanning the pages you've found before deciding on which CSS
page to post on", and asking a few basic questions about the subject: familiarity with TWiki, etc. Then, a clear way back to the reporting page, and a form for multiple choice and open-ended answers about how things went.
It can be pretty lo-tech. In some tests, the time taken matters - that can be left to the subject, but the system will note the time if subject starts and ends by modifiying his own report topic.
It may not be "scientific", but even a handful - even one or two - subject reports will no doubt be very useful. Questions can be tweaked, but open-endeds like: Comment on search - was it easy to evaluate results, did you have particular problems, what bothered you, etc, will get great response. People love to be interviewed, right, we all want to be authorities.
- In the above, unexpected/unusual things could come up, and from unexpected people:
- "I wasn't sure where to post. At the bottom?"
- "It was annoying having to make out the raw text to see where I was as I scrolled down in Edit to find a place to post."
- "It's silly, but it took five minutes before I realized there was a lower toolbar with the Edit link."
- "No problem. Took me 74 seconds! I even skimmed the page. I use TWiki, don't admin, but CSS is an interesting subject - is there a list of all pages with significant CSS comments, even if 'CSS' isn't in the title? I'd like to read more if I don't have to spend hours searching through."
This isn't complicated stuff. After the first template, we're off and running with making more to test whatever. And ANYONE can take these tests: experienced users, newbies, Wiki lovers and haters - if they fill out the survey at all meaningfully, hardcore dev considerations and conclusions can be drawn.
- 22 Nov 2001