Posted by:
Andrew Chan, Information Architect, Mass.gov, Information Technology Division
In my previous blog post, I gave you an introduction to usability testing. We discussed the three key phases necessary for success: planning the test, conducting the test and analyzing the results.
We didn’t, however, spend much time discussing how Mass.gov actually performs usability testing. We typically use rapid paper-prototype tests. We wanted to describe how it works and why we like it with a little more detail.
Planning the test
Like with everything else in life, if you plan for usability testing the odds of success are much higher. During the planning phase, you want to think about what you are trying to accomplish. Do you want to restructure whole areas of your website or just determine what category labels work best? These types of questions can help you identify your usability test goals.
Once you define your objectives, you then want to craft your test. If you review your web metrics, you can easily develop a list of about 10-15 tasks that you want your test participants to accomplish. Some frequently used metrics include most common search terms and pages with the most hits .
You’ve figured out your goals and created your test. Now you have to find people to take it. The key here is to use a diverse customer base. For example, if you were working on a website about education, you might want to test students, parents, teachers, administrators, and researchers. This way, you’ll get a good sense about how different customers in your target audience interact with your website and whether or not your website meets their needs. For best results, you should make sure you test at least ten participants.
Conducting the test
At Mass.Gov, we perform our own style of paper prototype usability testing. What is paper prototype testing? First, we take the navigational structure of the website and recreate it on paper. Each level of the website has it's own sheet. Why do this on plain sheets of paper? In this way, our test participants interact with a version of our website devoid of distractions, such as banner images, search boxes, left/right columns, etc. As a result, they focus solely on a step-by-step “click-through” of the high level topics and subtopics to find the requested information.
As our participants perform our test tasks, we mark down the paths they choose. We ask our participants to talk out loud as they "navigate" so we can understand their thought process. In an effort to keep the results as unbiased as possible, we do not provide hints or lead the test participant down any particular paths. The idea is to simulate real time successes and failures that customers have with a given navigational structure.
We specifically test one participant at a time instead of doing focus group sessions. This method works best because individual feedback isn’t swayed by popular opinion.
Analyzing the results
After all tests are completed, we compile all of the results into a test results report (sample). This report provides a general overview of the test, a listing of the results on a task by task basis, and some conclusions and recommendations for changes in the labeling or information architecture of the site.
When you examine these reports, you might find you only need slight changes. Or, you may find you need to re-architect or re-label some areas of the website and perform another round of usability testing. In either case, you now have actionable data to support the management of your website.
Final thoughts
Rapid paper-prototype usability testing is a great way to measure the usefulness of your website. Not only is it quick and easy, but it is also free to perform. It provides you with feedback from real customers that will help improve the quality and presentation of your information.
As government employees, it is our duty to make sure that citizens can find information and services as efficiently and effectively as possible. Paper prototype testing is one way of making sure that we do that. Please let us know if we can provide you with any help as you plan your own tests. Good luck and happy testing!
Comments