Message-Id: <l03110701b119fb1a5183@[128.101.88.19]>
Date: Wed, 25 Feb 1998 10:45:37 -0600
To: calvin-ui98@dagobah.stwing.upenn.edu
From: brad hokanson <bhokanson@che2.che.umn.edu>
Subject: Notes on User Testing, February 23
First User: Fred
(Fred is a senior in Technical Communications and has some familiarity with
Unix. He is the non-programmer on the team.)
Scenario One:
Rye (design team member) gave a brief description of the project. The task
was read by Hokanson. Scenario One is creating an archive at the default
location from the mailbox "mbox" in the working directory.
Mail Archive was already up and running.
The user first clicked on the Archive tab, and did not view the contents of
the first tab, called "Project". He was confused by what he saw; "What
is...." comments followed.
He appeared to understand that he needed to find a mailbox, and then
clicked on the Mailbox tab. He clicked on Browse and then typed in "MBOX".
He stopped, and then pulled down the File menu and went to Open. At this
point the testers intervened and restarted the process.
He selected "mbox" and clicked on the ADD button. The name "mbox" moved
down to the list of mailboxes. He clicked on the name to select.
He clicked on the Generate tab and stops and reads all the information,
mentally checking off all the items. One of the testers says that he should
accept all the defaults.
He clicks on the Generate button, and then when the feedback for that
button said he had completed the task, he clicked on OK.
He said he was done.
Observations: There was a high level of confusion in starting the project
and getting it running. Discussions after testing with the user confirmed
this confusion, and seemed to indicate that more direction at the beginning
of the project was needed. The Project tab was not used to any great extent
and was ignored during most of this scenario.
Scenario Two:
Rye added comments to as to the location of the materials. The scenario was
read by Hokanson: Task: create an archive with he program's default
location from the mailbox "mymail" in the /tmp/foo directory.
The user went straight back to mailboxes, said "cool" and deleted the
default/previous mailbox.
Some confusion was noted as to the location of the mailbox. The user had
assumed that the directory location was within the current working
directory and not a location as give from the root directory (which will
also occur with the second test subject) . The user clicked on browse to
fine /tmp/foo. He browses, gives up after about ten seconds.
He types in the addresses and before he can add it, he notices the pull
down menu to go up a directory. He stops typing and goes up two levels to
root, and sees the /tmp directory. He navigates to the /tmp/foo directory
and finds the file "mymail". He clicks on ADD, it immediately shows up on
the list.
He clicks on Generate Tab, clicks on the generate button. He responds to
the successful dialog box and clicks on OK.
Scenario 4
Create an archive in the /tmp/web/fun/ directory from three mailboxes john,
mark, sue in the /tmp/foo/ directory.
He goes to mailboxes, deletes previous by dragging and hitting the DELETE
button. He clicks on BROWSE, opens /tmp/foo, looks for the various
mailboxes.
He attempts to multiple select the three named at one time using a series
of key combinations; control, alt and shift. None work. He says he'll have
to do them one at a time.
He selects "john", clicks on ADD. That's successful. He clicks on "mark"
clicks on ADD and gets an error box. He clicks on OK, but then is confused
as to what he needs to do next. No information has been given in the error
box as to fixing the problem.
He clicks on Archive, Generate tabs and reads each of them looking for a
solution. He clicks on Project tab (the first one) and after some looking
finds the multiple mailbox radio button. He returns to Mailbox tab. adds
"john", it works, he clicks on "sue", clicks on ADD button and it adds that
name to the list as well.
He clicks on Generate tab.
Clicks on TEST. Tries to type in address, changes mind.
Clicks on Archive tab. Changes path by typing in /tmp/foo.
He clicks on Generate tab.
Clicks on Generate. Gets OK dialog box.
Observation: Does constraining the user to not be able to do something,
such a selecting multiple mailboxes when not in the multiple mode help or
make him more frustrated?
Users are not encouraged to input data in a sequential manner. This seems
to be due to giving them freedom of choice in a number of different areas.
-------
User: Nikki
She describes herself as having a fair amount of Unix experience, but by no
means an expert. She is a computer science major.
The scenario was described by Corey. The task was read by Hokanson.
Scenario One is creating an archive at the default location from the
mailbox "mbox" in the working directory.
Mail Archive was already up and running.
She begins by looking hard at the interface, apparently trying to figure
out what to do.
She click on Mailbox.
She clicks on ADD to add the mailbox to the list.
She paused for a significant period of time.
The first screen was not examined in the process of what she is doing.
She said she was not sure what was meant by "Project"
She clicks on Archive.
Back on Mailbox. She selects the mailbox from the list, and deletes it.
(Removing the default and leaving nothing in there.)
She says she is not sure.
She notes the Project tab is really unclear.
"test" catches her eye on the Generate tab.
Testers directed the user to reload the mailbox file to restore the default
settings.
Clicks on Archive tab. Says "Use the default?? Waits and studies for ten
seconds.
Says I think I might be done. (waits ten seconds.
Sees something at generate. Clicks on tab. Waits 5 seconds.
Reads that tab.
Clicks on generate. Clicks on OK dialog box.
Her Observations: Her use was not in a logical order. She said some thing
is needed that says, "this is what you click on to go to now."
Tabs completely frustrated her.
Thought she needed to go to each tab before she should go on.
At points she did not notice that Generate had been made available. Lack of
difference between greyed and black.
Scenario Two:
The scenario was read by Hokanson: Task: create an archive with he
program's default location from the mailbox "mymail" in the /tmp/foo
directory.
Goes straight to mailbox and click on Browse. Can't find /tmp/foo. Finds
the up menu early, clicks on an navigates to the root directory. Clicks on
add. Clicks on generate tab/generate button and gets the OK dialog box.
Scenario 4
Create an archive in the /tmp/web/fun/ directory from three mailboxes john,
mark, sue in the /tmp/foo/ directory.
Goes straight to mailbox, and adds John, clicks add.
Clicks on Browse to add Mark, gets error message.
about 1 minute passes before she goes directly back to Project and finds
the radio button for multiple mailboxes.
She noted that she felt guided by TAB layout.
That the Project tab layout was not user friendly.
She was distracted at the Archive tab, particularly by the tons of stuff.
Most important thing on the Project Tab is buried.
Brad Hokanson
UC Coordinator
Department of Design, Housing, and Apparel
University of MInnesota, St. Paul.
612.624.4918 voice
612.624.2750 fax