WAI self-examination
I was surprised and delighted to read last week that the Web Accessibility Initiative commissioned a usability study of its Web site. Done pro bono by the American Institutes for Research, the study demonstrated what we knew already – and then some.
It was commendable for WAI, which is behind the times in Web development in so many other ways, to use state-of-the-art evaluation methods to assess its site. As I’ve argued for ages, WAI will not be taken seriously by designers until WAI takes design seriously.
We can now dare to dream of a future in which the WAI portion of the W3C site actually looks professionally designed, with proper information architecture, multiple selectable styles available, and full WCAG compliance. (Curiously, the WAI site itself only claims to comply with Priority 2. It seems impossible for an entire site to comply with Priority 3 if the authors of the spec cannot do it.)
The AIR study isn’t available in a convenient single page yet, though I assembled a single HTML file and single PDF (at a svelte 98 pages) if anyone needs something to print.
Highlights
Participants
[W]e recruited a total of eight persons involved in Web development to serve as usability evaluation participants. One of the participants was a Web development professional who is blind. That participant kindly brought a personal laptop to our lab space in order to facilitate exploration of the WAI site using a Jaws setup customized to her needs. Another participant had a hearing disability. In addition, one of our participants self-reported as colourblind.
Tasks
Subjects were asked to carry out a selection of plausible tasks that was quite well-considered. Some included:
Your team at work is developing a Web site and you have some concerns about how accessible the Web site might be to people with disabilities. Using this Web site, determine whether or not it contains information about the basic things Web developers need to know about Web accessibility.
A few of your colleagues are interested in finding out how to be a part of WAI’s effort to develop guidelines for Web accessibility. Using this Web site, determine whether or not opportunities exist for becoming involved in WAI Web Content guideline development.
You have just been handed a report, generated by a Web accessibility evaluation tool, which informs you that your company Web site contains complex information graphs that do not meet “Checkpoint 1.1.” Using this Web site determine how you would meet Checkpoint 1.1 and make the graphs accessible.
Your company is revising the online forms on its Web site. Find specific information on how to make the online forms accessible.
Task success
It’s hard to fare worse than this: “Participants were largely unable to complete these tasks. With the exception of the WAI mission task, no more than five of eight participants were able to complete any task without assistance from the test administrator.”
Too much information
A common complaint (even voiced on Slashdot) holds that there’s just too dang much to read on WAI pages. It’s true.
Perhaps the most significant finding of the WAI Web site usability test was the observation that the Web site users consistently stated that they were overwhelmed by the information available to them.
They stated that the WAI developers “had not parsed” the information for users, hadn’t provided adequate “information and technique summaries” and asserted that using the WAI Web site was like “walking into a firehose.” They repeatedly suggested that the information they were receiving could be “summarized” into “manageable chunks” for easy, or quick use.
Participants uniformly stated that the WAI Web site information that was most relevant to their work was always “hidden.” They described the process of using “a W3C site” as: “you go to the page, scroll down, scroll down, scroll down, about four or five screens, past all the useless stuff, and then you get to the good stuff. Which is great. But it’s always in the middle part of a really, really, really long page.” [...]
They uniformly suggested that the WAI development team consider ways to parse the Web site’s information rather than giving it to them “all at once.”
Graphic design
Participants were asked to rate various factors on a scale of 1 (“Disagree strongly”) to 7 (“Agree strongly”). Graphic design fared poorly:
- “The homepage is attractive”: 4.0
- “The overall site is attractive”: 3.4
- “The site’s graphics are pleasing”: 3.3
- “The site has a good balance of graphics vs. text”: 3.2
- “The colours used throughout the site are attractive”: 4.6
- “The typography is attractive” [!]: 5.6
Complaints to address
The study quotes some free-form complaints by participants. My suggestion to WAI is to work to counter every single one of these, plus each of the later comments and “words or characteristics” participants used to describe the site.
- There should be a category for myths about accessibility so designers know they don’t have to sacrifice style.
- There should be examples of pages that are accessible.
- It would contain a set of guidelines for developing Web sites. It should have test tools, information about troubleshooting….
- There should be hot topics to be aware of, industry specific issues. It should provide recommendations or experiences other people have.
A great place to start
WAI still has a desperate need for participants. There’s too much work for too few people, and some of us simply aren’t experts in important areas.
Still, the AIR study amounts to a galvanized, wheelchair-accessible nail in the coffin of WAI’s history of being ignored by professional developers. WAI now has a full catalogue of documented issues it can fix, along with ways to do the fixing. Based, in addition, on further information I have, I expect that the WAI site at the time of release of the Web Content Accessibility Guidelines 2.0 will be a total shock to anyone used to trudging through what we’re stuck with now.
As they say in the Maritimes, good on yez. Now don’t blow it!