Date: Wed, 25 Feb 1998 14:04:04 -0600
From: escargo@anubis.network.com (David S. Cargo)
Message-Id: <199802252004.OAA08002@brutus.network.com>
To: beec0018@tc.umn.edu, mcco0062@gold.tc.umn.edu, nord0157@maroon.tc.umn.edu,
Subject: Evalutation of Group 0 application
David S. Cargo
CSci 5110, Winter '98
"Other group" evaluation
Group 0 was our originally-assigned counterpart group. When confusion
arose over what group to test with, the members of group 5 told us that
they already had user tests, so we muddled through testing with group 0.
They tested Group 4 in groups of one and two. I was part of one of the
groups of two.
They asked us about our familiarity with UNIX, and told us a little
about what their project did. By chance, the person in our pair who
knew the most about UNIX (me) was also the person giving instructions to
the person with the mouse.
They started us with the application already initialized. We were given
a total of three tasks to perform.
The first task was to change the color of the top of the frame around
application windows. We were able to determine which of their
navigation buttons to use to select the correct part of the application
to effect that change. There was a model of a desktop with labels on
the different parts of the window. There was a listbox with names of
the different window parts. There was also a place to start a color
chooser. Once we identified the right item in the listbox, starting the
color chooser was straightforward.
There was some confusion over the save and apply buttons, but partly
this seemed to be due to the limitations of their prototype.
The second task was choosing the screensaver. Again, the navigation
buttons were clearly marked for this. There was some confusion because
the instructions we were given (choose the screensaver with the bouncing
soccer balls) wasn't matched by the text describing the actual
screensaver (but only because the US has a different definition of
football from most of the rest of the world).
Once we picked the screensaver, we were able to see a model of the
screen running the screensaver, but this presented us with a rectangle
and bouncing squares. There was a preview button we were able to use
that brought up a full screen using the screensaver, where we did see
the soccer balls. We had to guess about how to exit out of the
screensaver. Again we had the questions about save and apply.
The third task was choosing a background picture. There was a
navigation button for background. Once there it was possible to use a
listbox to select a picture by name. As I recall, it was not actually
possible to set the image. And again we had the problem deciding what
was meant by save and apply.
At the end of the test they asked intelligent questions about our
thoughts on their project, especially about what we thought save and
apply should do. There was a cancel button, but we never had reason to
use it.
Finally, they offerred us candy as a "Thank you" gesture, which we
thought was very thoughtful of them.
Now the evaluation.
>From the initial screen of their project, it seemed clear to me how to
navigate to their subscreens. The navigation controls were prominent.
All of the controls for carrying out the directed tasks were familiar to
me, but might have been confusing for less experienced users. Except
for the color chooser, most controls used listboxes to display and
choose items.
In some cases there was not as much feedback as might be possible.
While the save and apply buttons were not easy to distinguish
functionally, since they didn't seem to say that whatever was selected
had been saved, it wasn't clear that an action had taken place.
When picking a new background picture, a set of radiobuttons for the
picture always seemed to be active, and there was not default. It
wasn't possible to actually apply or save that setting to see what would
have happened; it's possible there was a default, but that the
application simply wasn't far enough along to activate it.
>From the standpoint of the nine heuristics, the navigation buttons
seemed natural to me as a UNIX user; non-UNIX users probably wouldn't be
using the application normally.
The one failing of "Speak the user's language" came from the task that
directed us to choose the football screensaver, as mentioned above.
Strictly speaking this was not a problem with the application, but with
the design of the user tests, but it could have been avoided.
The tasks were simple enough, and the functions of the application
disjoint enough, that memory load did not seem to be a problem. At
least, it probably wouldn't be a problem for users; I had a problem
remembering what to do because of the chaos the environment. Perhaps
users given a task should have it in writing so they can refer back to
it as they progress, if they need to.
>From a feeback standpoint, it might be desirable to have the save,
apply, and cancel buttons disabled when there hasn't yet been a choice
made. If apply and save have different semantics, then after apply has
been pressed, perhaps apply and cancel would be disabled, while save was
still active. If after save has been pressed the apply and cancel
choices aren't available, all three could be grayed out.
There weren't so many options that shortcuts would seem to be necessary.
I don't remember getting any error messages. Either I didn't stray from
the expected path much (which is possible, because I knew what I was
doing better than most of Group 0 did), or the application was designed
so that errors were prevented.
dsc