Date: Wed, 25 Feb 1998 14:20:59 -0600 (CST)
From: "Brent J. Dahlen" <dahlen@cs.umn.edu>
To: "David S. Cargo" <escargo@anubis.network.com>
Subject: Re: Evalutation of Group 0 application
In-Reply-To: <199802252004.OAA08002@brutus.network.com>
Message-Id: <Pine.GSO.3.95q.980225141752.2706A-100000@deci.cs.umn.edu>
Yes--I'm well aware of the problem. It arose when I told your group
(Group 4) by word of mouth that you would be testing with Group 0,
however, an email later that day should have confirmed that a change
had been made in which you were reassigned to Group 2. In any case,
it now looks like things have ironed themselves out. I really don't
care all that much which group tests with which other group, just as
long as everyone gets involved in the testing/observation process.
Hope everything went well. See you all today,
Brent
On Wed, 25 Feb 1998, David S. Cargo wrote:
> David S. Cargo
> CSci 5110, Winter '98
> "Other group" evaluation
>
> Group 0 was our originally-assigned counterpart group. When confusion
> arose over what group to test with, the members of group 5 told us that
> they already had user tests, so we muddled through testing with group 0.
>
> They tested Group 4 in groups of one and two. I was part of one of the
> groups of two.
>
> They asked us about our familiarity with UNIX, and told us a little
> about what their project did. By chance, the person in our pair who
> knew the most about UNIX (me) was also the person giving instructions to
> the person with the mouse.
>
> They started us with the application already initialized. We were given
> a total of three tasks to perform.
>
> The first task was to change the color of the top of the frame around
> application windows. We were able to determine which of their
> navigation buttons to use to select the correct part of the application
> to effect that change. There was a model of a desktop with labels on
> the different parts of the window. There was a listbox with names of
> the different window parts. There was also a place to start a color
> chooser. Once we identified the right item in the listbox, starting the
> color chooser was straightforward.
>
> There was some confusion over the save and apply buttons, but partly
> this seemed to be due to the limitations of their prototype.
>
> The second task was choosing the screensaver. Again, the navigation
> buttons were clearly marked for this. There was some confusion because
> the instructions we were given (choose the screensaver with the bouncing
> soccer balls) wasn't matched by the text describing the actual
> screensaver (but only because the US has a different definition of
> football from most of the rest of the world).
>
> Once we picked the screensaver, we were able to see a model of the
> screen running the screensaver, but this presented us with a rectangle
> and bouncing squares. There was a preview button we were able to use
> that brought up a full screen using the screensaver, where we did see
> the soccer balls. We had to guess about how to exit out of the
> screensaver. Again we had the questions about save and apply.
>
> The third task was choosing a background picture. There was a
> navigation button for background. Once there it was possible to use a
> listbox to select a picture by name. As I recall, it was not actually
> possible to set the image. And again we had the problem deciding what
> was meant by save and apply.
>
> At the end of the test they asked intelligent questions about our
> thoughts on their project, especially about what we thought save and
> apply should do. There was a cancel button, but we never had reason to
> use it.
>
> Finally, they offerred us candy as a "Thank you" gesture, which we
> thought was very thoughtful of them.
>
> Now the evaluation.
>
> >From the initial screen of their project, it seemed clear to me how to
> navigate to their subscreens. The navigation controls were prominent.
> All of the controls for carrying out the directed tasks were familiar to
> me, but might have been confusing for less experienced users. Except
> for the color chooser, most controls used listboxes to display and
> choose items.
>
> In some cases there was not as much feedback as might be possible.
> While the save and apply buttons were not easy to distinguish
> functionally, since they didn't seem to say that whatever was selected
> had been saved, it wasn't clear that an action had taken place.
>
> When picking a new background picture, a set of radiobuttons for the
> picture always seemed to be active, and there was not default. It
> wasn't possible to actually apply or save that setting to see what would
> have happened; it's possible there was a default, but that the
> application simply wasn't far enough along to activate it.
>
> >From the standpoint of the nine heuristics, the navigation buttons
> seemed natural to me as a UNIX user; non-UNIX users probably wouldn't be
> using the application normally.
>
> The one failing of "Speak the user's language" came from the task that
> directed us to choose the football screensaver, as mentioned above.
> Strictly speaking this was not a problem with the application, but with
> the design of the user tests, but it could have been avoided.
>
> The tasks were simple enough, and the functions of the application
> disjoint enough, that memory load did not seem to be a problem. At
> least, it probably wouldn't be a problem for users; I had a problem
> remembering what to do because of the chaos the environment. Perhaps
> users given a task should have it in writing so they can refer back to
> it as they progress, if they need to.
>
> >From a feeback standpoint, it might be desirable to have the save,
> apply, and cancel buttons disabled when there hasn't yet been a choice
> made. If apply and save have different semantics, then after apply has
> been pressed, perhaps apply and cancel would be disabled, while save was
> still active. If after save has been pressed the apply and cancel
> choices aren't available, all three could be grayed out.
>
> There weren't so many options that shortcuts would seem to be necessary.
>
> I don't remember getting any error messages. Either I didn't stray from
> the expected path much (which is possible, because I knew what I was
> doing better than most of Group 0 did), or the application was designed
> so that errors were prevented.
>
> dsc
>
Brent J. Dahlen
__________________________________
|dahlen@cs.umn.edu |
|http://www.cs.umn.edu/~dahlen |
|________________________________|