Google DTorial: An interactive tutorial framework for blind users in a Web 2.0 world - Manual (Page 6)

text and re-reading content evolved through the incremental design process, and this
example is representative of the final iteration of the DTorial system.
5.2 Incremental Design and Evaluation Methodology
In order to test learning via DTorial, we compared it against the standard form of
instruction for VIUs, separate HTML web pages containing a tutorial. While most
HTML online tutorials are not screen-reader friendly, we eliminated bias towards
DTorial by ensured that the HTML tutorial had marked headers, no references to
video or images, and provided the same content as the interactive counterpart.
We followed a rapid cycle of evaluation and redesign. Visually impaired subjects
were recruited from centers and organizations for the visually impaired in California’s
Silicon Valley and San Francisco. See Table 1 for demographics. Though we
recruited 20 users, 17 individuals participated (eight men and nine women). Two were
determined to be ineligible for the study when it was discovered that they were not
visually impaired. One was removed from the study due to technical difficulties that
arose during the session. A typical session with one subject lasted for one-and-a-half
hours. Subjects were remunerated ($75/hour). Participants had no prior experience
with Gmail, though some had accounts that were forwarded to desktop email clients.
Tutorial text was based on Gmail’s Getting Started guide. We limited Gmail’s
feature set to Compose Mail, Inbox, Drafts, Spam, Trash, and Message Threads so we
could focus on the tutorial and the learning experience. Because Web 2.0 applications
require users to be in PCM when using hot-keys, we added audio feedback via
AxsJAX, to increase accessibility (e.g., so that audio-based alerts were spoken when
pages changed and updated).
During a session, a participant was exposed to each tutorial for approximately 30
minutes. The participant was instructed to “do as you normally would, as if trying this
out for the first time on your own and as if we were not here.” During a session with
the HTML tutorial, a participant was provided with two windows, one pointing to
Gmail and the other to the tutorial. After an exposure, each participant was asked a
series of questions focusing on usability of the tutorial, accessibility of the tutorial
content, how much the participant had learned about using the application, and
methods for improvement. Following exposure to both tutorials, a series of questions
were asked comparing the two methods. Participants were asked to indicate and
justify a preference between the two tutorials.
The order of exposure was randomized to overcome learning effects. Overall, eight
Table 1. Demographic in Iterative Usability Study
Number of Users 17
Mean Age 40 years
20-29 (4)
30-39 (2) 40-49 (4) Number of Users Per Age Group
50-59 (3) 60-69 (4)
Computer Experience (years) 1.5-23 Average 12.1 years
JAWS Experience (years) 1.5-16 Average of 10.5 years
Number of Users Exposed to DTorial First 8
Number of Users Exposed to HTML Tutorial First 9