Date: Sun, 1 Mar 1998 17:30:06 -0600 (CST)
From: Quang-Tuyen Dang <dang0019@gold.tc.umn.edu>
To: CSCI-5110 list <calvin-ui98@dagobah.stwing.upenn.edu>
Subject: testing log
Message-Id: <Pine.GSO.3.95q.980301172834.6534A-100000@beluga.itlabs.umn.edu>
Hi all,
I will try to make a summary of problems/solutions thing from the
testers' comments and ours. Following is my test log:
User test for log Mon, 2/23/98
By Quang-Tuyen Dang
* Setup:
- Participants: Cargo and Dang with one tester at a time
(two testers for this log).
- Cargo started the application.
- Dang introduced the application to the tester of what the
application is, what it can do and the task we had the user to
complete.
- Cargo and Dang both took notes throughout the tests.
* Tester 1:
- Task 1: Create an archive in the program's default location with the
mailbox "mbox" in the working directory.
He started with Generate tab but it was inactive, so he explored the
window and jumped back to the Project tab, read through the option
questions quickly, kept repeating "default options: OK",
"default working directory: OK", and then clicked on Mailboxes tab.
He saw the default mailbox in the textbox, so he hit <Enter> but
it did not get anywhere. He found the "Add" button, then clicked on
the button. He said that he had added the mailbox. He clicked on the
Archive tab and went through the text quickly. Then he clicked
Generate tab, also skim through the text fast, and clicked "Generate".
- Task 2: Create an archive with the program's defualt location from the
mailbox named "mymail" in the "/tmp/foo/" directory.
He then was familiar with the applicationa a bit, so got the mailbox
fine, changed the archive directory. He stopped a moment at the textbox
for test directory, then skipped it. He got it done.
- Task 3: Create an archive of the mail in the mailbox named "research" in
the "/tmp/foo/" directory and put the HTML archive in the default
location set by the program. Only select messages that do not contain
"fun" in the subject.
This was the most difficult task among the tested tasks. Tester could
select the mailbox and use the default HTML archive but was slow down
when facing the requirement about filtering subject. The only options
regarding subject available were "any subject" and "subject matches",
so he could only pick "subject matches" and typed in "fun" to the
textbox next to the choice. Then he got it done quickly.
- Wrap-up:
The tester was concerned about lacking knowledge of regular expression
for matching. He suggested to have more options and to change the text
for easier understanding such as "subject equals" and
"subject contains". He wondered why there was a list of filters. Cargo
explained to him that order of filters might produce different results.
He asked why he needed to add filters since he thought that once he
typed in the filter name and chose options for the filter, the list
should be updated.
He did not understand what "New filter" meant. He suggested to change
the label into something more close to the context. The location of
the "Add" button also caused him a problem since it was located on
the left of the textbox. He added that the button should be grayed out
before something was entered into the textbox.
Dang asked tester about his memory of the stuff under Project tab and
Archive tab but he did not remember much. He said he did not even
bother to read the sentences under the Project tab. It was just too
much for him. He did not have a clue between the test directory under
Archive and the option under "Generate".
The tab order is meaningful to him.
* Tester 2:
- Task 1: Similar to tester 1
Right after listening to the task description, he just jumped into
Archive tab and did not find any "default" information. He tried around
and got stuck. Dang gave him a hint to try Mailboxes tab. He finally
found default mailbox under Mailboxes tab, then clicked "Add" to add
the mailbox. He proceeded to Generate tab and completed the task.
- Task 2: Similar to tester 1
This time he jumped all the way to Generate tab and tried to change
the mailbox name there. He could not do that because the textbox was
disabled. He then tried to change the working directory under Project
tab into mailbox name. As a result, he did not accomplished that task.
Dang again provided a hint, so he could browse to find the mailbox.
- Task 3: Create an archive in the "/tmp/web/stars/" directory from the
mailbox named "/tmp/foo/jmail".
He assumed working directory as archive directory. He kept struggling
with the tabs, questions and the text. Finally, he completed the task.
- Wrap-up:
He told us that he did not have feelings about the order of the tabs.
He realized the difference between the black tabs and grayed tabs.
He did not understand the questions under Project tab. We realized
that the filter list should be flushed when the Filter tab grayed out.
He said that he had no reason and did not like to put his mailbox on
the Web archive. However, after all, we showed our web archive, he
was impressed and said he would like to use the application in future.
His UNIX background is intermediate.