By Adam Bennett
Introduction
A website is an important medium for
small organizations to get information to their target audiences.
However, lack of budget, technical prowess, or awareness of the
importance of software usability can lead an organization to not
invest in having a usable website. Unfortunately, this can result in
users having difficulty finding the information or services that the
organization provides; in extreme cases the user may become
frustrated and give up their search altogether.
Children's Resources On Wheels (CROW)
is a non-profit organization that provides childcare information and
services to Lanark County. Their current website is somewhat dated
and contains a number of usability issues. An alternative version of
the CROW website was designed with the intent of addressing the
usability issues pertaining to navigation. In terms of usability,
navigation can be one of the most important design factors; it
doesn't matter how usable a webpage is if the user cannot access it
(Valacich et al., 2007). Other aspects of usability were not
addressed in the new version of the website.
Changes in the redesigned website
include adding, moving, and removing options from the main menu,
making links always open in the current tab except for external links
and downloads, and changing the terminology of navigation options to be
more consistent. Some of these changes can be seen below:
Side by side comparison of the fully
expanded main menus of the original and redesigned website versions
However, just because the new design
was intended to improve upon the usability of the original website
doesn't mean that it will succeed at that goal. In order to know that
an alternative design is actually an improvement, an experiment may
be performed to measure the usability of the redesigned website.
The Experiment
The experiment was designed to measure
the navigation usability for the original and redesigned websites. A
list of twenty navigation tasks was made based on suggestions from
CROW staff, with care taken to ensure that the order of the tasks
didn't bias the experiment's results toward either website version.
To test the website designs,
participants would perform as many of the tasks on the list as they
could for a given version of the website within a time limit of five
minutes. After the time elapsed, the number of tasks successfully
completed up to that point was tallied up and the total was used to
quantify the usability of the website tested.
Testing conditions were kept as close
to the same as possible for all participants and both website
versions. Each participant performed one test of each version of the
website. To prevent bias due to participants becoming familiar with
the task list and similarities between the two website versions, half
of the participants tested the original website followed by the
redesigned version; the other half tested the redesigned website
first and then tested the original version.
After testing both website versions,
each participant was asked to fill out a short survey. The survey
asked their opinions of the websites in general, and how easy they
found navigating each version to be.
There are two main categories of users
for the CROW website: parents of young children looking for child
care information or services and child care providers looking to
register with the CROW organization. To help obtain more accurate
results, participants were selected who could fit into one of these
categories. Six participants took part in the experiment, with four
being parents of young children and two being child care providers.
Results and Discussion
The results obtained from the
experiment were the objective measure of how many tasks each
participant completed for each website version, and each
participant's perception of the websites based on their
responses to the survey questions.
Number of Tasks Completed
Ideally, the redesigned website version
would have been more usable than the original website and the
experiment results would have reflected that. Unfortunately, this was
not the case. As it turns out, there was no significant difference in
the number of tasks completed for the two website versions (results
are considered significant only if it is probable that the same
results would be found 19 times if the experiment was repeated 20
times).
Box plot showing the number of tasks completed for the original and redesigned website versions
So why didn't the redesigned version
perform better than the original? The most straightforward answer is
that the redesigned version simply wasn't different enough in terms
of navigation usability than the original. Another possibility is
that while there were improvements to the usability of the redesigned
website, there were also new usability issues added unintentionally.
One of the main examples of a change
which caused issues with the redesigned website was moving the
Programs menu to become a sub-menu of the Ontario Early Years menu. Participants often had trouble finding the Programs sub-menu
in the redesigned website. The Programs menu should be restored to its original position for subsequent experiments.
It is also possible that the small
sample size of six participants was too small to provide accurate
results. A small sample size causes outliers (results that differ
from the overall trend that we are trying to measure) to have a
larger impact on results than they would in a larger sample. The only
way to know if this was the case would be to run the experiment again
with more participants.
Finally, allowing participants to skip
tasks which they found difficult or frustrating may have had an
impact on the results. Participants were allowed to skip tasks so
that they would not become frustrated during the experiment, but
outside the experiment a user might give up entirely on a website
(and the corresponding organization) if they are unable to complete
basic navigation tasks. It is possible that this skewed results in
some way, especially since many changes in the redesigned version
were intended to prevent certain tasks from taking as long as they
did in the original website.
Perceived Performance
Most users indicated on the survey that
they preferred the original version of the website over the
redesigned version and found the original website easier to navigate.
This is interesting, since the number of tasks completed by
participants doesn't seem to reflect this. Based on the behaviour
of participants, one possible explanation for this is the difference
in the perceived number of options between website versions. When
participants could not immediately complete their current navigation
task, they often resorted to looking at the options under each
top-level menu item. While in practice a new user might be
overwhelmed by the sheer number of options in the original website
(in fact, one of the primary design goals for the redesigned website
was to not overwhelm the user with options), in this experiment
participants who were actively searching for an option tended not to
give up on the current task so long as there were unexplored
top-level menus. Thus, while presenting the user with more options
might not improve the usability of website navigation in terms of the
number of tasks completed per unit time, it may encourage users to
spend longer on navigation tasks that they would otherwise give up
on. This effect further reinforces the idea that subsequent
experiments should do more to take into account skipped tasks (such
as penalizing the total number of tasks completed), since it is
better that a task is completed slowly than not at all.
Conclusion and Future Work
It is clear from the results of the
experiment that the redesigned website was not a substantial
improvement over the original website. This does not necessarily mean
that all changes introduced in the newer version should be discarded,
but it does indicate that there are still improvements to be made and
that some problems were introduced.
If the experiment is repeated, it
should be modified slightly based on the above observations. In
particular, more participants should take part and the measure of
usability should address skipped tasks and the participants'
satisfaction while performing tasks. The new website design should be
redesigned again, to take this experiment's results into
consideration and remove flaws that were not in the original website.
References
Carol Barnum. 2011. Usability Testing
Essentials – Ready, Set...Test!. Elsevier.
Ruiili Geng and Jeff Tian. 2015.
Improving Web Navigation Usability by Comparing Actual and
Anticipated Usage. IEEE Transactions on Human-Machine Systems 45, 1
(Feb 2015), 84-94
Timothy Lethbridge. 2017. Software
Usability Winter Course Notes.
Joseph S. Valacich, D. Veena
Parboteeah, and John D. Wells. 2007. The online consumer's hierarchy
of needs. Commun. ACM 50, 9 (September 2007), 84-90.
DOI=http://dx.doi.org/10.1145/1284621.1284624
No comments:
Post a Comment