From: "brad hokanson" <Brad.Hokanson-2@tc.umn.edu>
To: calvin-ui98@dagobah.stwing.upenn.edu
Subject: User testing
Message-Id: <34f4c5036ac1389@mhub1.tc.umn.edu>
Date: Wed, 25 Feb 98 19:27:37 -0600
Notes on User Testing, February 23
First User: Fred
(Fred is a senior in Technical Communications and has some familiarity with
Unix. He is the non-programmer on the team.)
Scenario One:
Rye (design team member) gave a brief description of the project. The task
was read by Hokanson. Scenario One is creating an archive at the default
location from the mailbox 3mbox2 in the working directory.
Mail Archive was already up and running.
The user first clicked on the Archive tab, and did not view the contents of
the first tab, called 3Project2. He was confused by what he saw; 3What
is....2 comments followed.
He appeared to understand that he needed to find a mailbox, and then
clicked on the Mailbox tab. He clicked on Browse and then typed in 3MBOX2.
He stopped, and then pulled down the File menu and went to Open. At this
point the testers intervened and restarted the process.
He selected 3mbox2 and clicked on the ADD button. The name 3mbox2 moved
down to the list of mailboxes. He clicked on the name to select.
He clicked on the Generate tab and stops and reads all the information,
mentally checking off all the items. One of the testers says that he should
accept all the defaults.
He clicks on the Generate button, and then when the feedback for that
button said he had completed the task, he clicked on OK.
He said he was done.
Observations: There was a high level of confusion in starting the project
and getting it running. Discussions after testing with the user confirmed
this confusion, and seemed to indicate that more direction at the beginning
of the project was needed. The Project tab was not used to any great extent
and was ignored during most of this scenario.
Scenario Two:
Rye added comments to as to the location of the materials. The scenario was
read by Hokanson: Task: create an archive with he program9s default
location from the mailbox 3mymail2 in the /tmp/foo directory.
The user went straight back to mailboxes, said 3cool2 and deleted the
default/previous mailbox.
Some confusion was noted as to the location of the mailbox. The user had
assumed that the directory location was within the current working
directory and not a location as give from the root directory (which will
also occur with the second test subject) . The user clicked on browse to
fine /tmp/foo. He browses, gives up after about ten seconds.
He types in the addresses and before he can add it, he notices the pull
down menu to go up a directory. He stops typing and goes up two levels to
root, and sees the /tmp directory. He navigates to the /tmp/foo directory
and finds the file 3mymail2. He clicks on ADD, it immediately shows up on
the list.
He clicks on Generate Tab, clicks on the generate button. He responds to
the successful dialog box and clicks on OK.
Scenario 4
Create an archive in the /tmp/web/fun/ directory from three mailboxes john,
mark, sue in the /tmp/foo/ directory.
He goes to mailboxes, deletes previous by dragging and hitting the DELETE
button. He clicks on BROWSE, opens /tmp/foo, looks for the various
mailboxes.
He attempts to multiple select the three named at one time using a series
of key combinations; control, alt and shift. None work. He says he9ll have
to do them one at a time.
He selects 3john2, clicks on ADD. That9s successful. He clicks on 3mark2
clicks on ADD and gets an error box. He clicks on OK, but then is confused
as to what he needs to do next. No information has been given in the error
box as to fixing the problem.
He clicks on Archive, Generate tabs and reads each of them looking for a
solution. He clicks on Project tab (the first one) and after some looking
finds the multiple mailbox radio button. He returns to Mailbox tab. adds
3john2, it works, he clicks on 3sue2, clicks on ADD button and it adds that
name to the list as well.
He clicks on Generate tab.
Clicks on TEST. Tries to type in address, changes mind.
Clicks on Archive tab. Changes path by typing in /tmp/foo.
He clicks on Generate tab.
Clicks on Generate. Gets OK dialog box.
Observation: Does constraining the user to not be able to do something,
such a selecting multiple mailboxes when not in the multiple mode help or
make him more frustrated?
Users are not encouraged to input data in a sequential manner. This seems
to be due to giving them freedom of choice in a number of different areas.
-------
User: Nikki
She describes herself as having a fair amount of Unix experience, but by no
means an expert. She is a computer science major.
The scenario was described by Corey. The task was read by Hokanson.
Scenario One is creating an archive at the default location from the
mailbox 3mbox2 in the working directory.
Mail Archive was already up and running.
She begins by looking hard at the interface, apparently trying to figure
out what to do.
She click on Mailbox.
She clicks on ADD to add the mailbox to the list.
She paused for a significant period of time.
The first screen was not examined in the process of what she is doing.
She said she was not sure what was meant by 3Project2
She clicks on Archive.
Back on Mailbox. She selects the mailbox from the list, and deletes it.
(Removing the default and leaving nothing in there.)
She says she is not sure.
She notes the Project tab is really unclear.
3test2 catches her eye on the Generate tab.
Testers directed the user to reload the mailbox file to restore the default
settings.
Clicks on Archive tab. Says 3Use the default?? Waits and studies for ten
seconds.
Says I think I might be done. (waits ten seconds.
Sees something at generate. Clicks on tab. Waits 5 seconds.
Reads that tab.
Clicks on generate. Clicks on OK dialog box.
Her Observations: Her use was not in a logical order. She said some thing
is needed that says, 3this is what you click on to go to now.2
Tabs completely frustrated her.
Thought she needed to go to each tab before she should go on.
At points she did not notice that Generate had been made available. Lack of
difference between greyed and black.
Scenario Two:
The scenario was read by Hokanson: Task: create an archive with he
program9s default location from the mailbox 3mymail2 in the /tmp/foo
directory.
Goes straight to mailbox and click on Browse. Can9t find /tmp/foo. Finds
the up menu early, clicks on an navigates to the root directory. Clicks on
add. Clicks on generate tab/generate button and gets the OK dialog box.
Scenario 4
Create an archive in the /tmp/web/fun/ directory from three mailboxes john,
mark, sue in the /tmp/foo/ directory.
Goes straight to mailbox, and adds John, clicks add.
Clicks on Browse to add Mark, gets error message.
about 1 minute passes before she goes directly back to Project and finds
the radio button for multiple mailboxes.
She noted that she felt guided by TAB layout.
That the Project tab layout was not user friendly.
She was distracted at the Archive tab, particularly by the tons of stuff.
Most important thing on the Project Tab is buried.