Axxlog

Boy, it’s been a while since I did one of these. A couple of research papers on Web accessibility and one on audio description:

“Comparing Website accessibility evaluation methods and learnings from usability evaluation methods” from Peak Usability (195 K PDF)

Tania Lang’s paper states: “The aim of this paper was to determine the most effective accessibility evaluation method to determine Web accessibility and concludes that a fully integrated approach (combining automated and manual evaluation and accessibility testing) is… best.”

The paper states that accessibility and usability evaluations share such measures as:

• number and severity of problems
• task completion (yes/no)
• user satisfaction
• duration

Lang makes note of the following (excerpted):

• Whilst there are relatively few papers comparing accessibility evaluation methods, there are numerous papers discussing usability evaluation methods (UEMs). There are many similarities between UEMs and AEMs. For instance, accessibility testing with users generally appear to use similar measures as those used in usability testing such as task completion, satisfaction and efficiency (although user satisfaction is often excluded as a measure in accessibility testing).
• [A]ccessibility evaluation methods often limit measures to efficiency and effectiveness whereas usability testing usually also includes a measure of user satisfaction (as highlighted in Table 1). The workshop consensus was that accessibility testing should measure efficiency, effectiveness and user satisfaction – the same measures typically used in usability evaluations.
• May be difficult to determine if issues are accessibility issues or usability issues applicable to all users – need to test with people with no disabilities first to establish a baseline…. Generally if an issue is only experienced by users with a disability, it can be categorised as an accessibility issue.

And by far the best part (also excerpted):

• Colwell and Petrie… conducted an experimentwith 12 students who were learning HTML. Students were asked to adapt an existing Web page making it accessible by following the WCAG 1.0. Results suggested that several improvements could be made to the WCAG 1.0…: “their structure and tone; navigation within and between the documents; the content and presentation of examples; and additional information to be provided.” [...]
• Of perhaps greater significance was a second experiment conducted by Colwell and Petrie… which involved accessibility testing of the pages developed by students using WCAG 1.0. 20 visually impaired participants used a wide variety of browsers and screen readers and performed the test in their own environment. Results showed that even though the Web sites were designed using WCAG 1.0, some major usability issues still existed for some participants with specific browsers…. Other results showed that some design changes made by the developers that were not based on the WCAG 1.0 actually appeared to improve accessibility more than some of the changes outlined in the guidelines….

This article, by the way, is in dire need of copy-editing. Among many errors, the man who won a complaint against the Sydney Olympics is Bruce Maguire, not Brian Maguire. (My Reader’s Guide to Sydney Olympics Accessibility Complaint is still online, and was slightly expanded in my book.)

“Assessing the accessibility of fifty United States government Web pages: Using Bobby to check on Uncle Sam” (great pun there) by Jim Ellison

You know, we really don’t need any more papers – not a single additional one – in which Web accessibility is tested using some automated method, let alone Bobby. Automated testing can never prove your site is accessible and misses a large number of important criteria. And these papers actually admit those limitations, which are crippling and which undercut the entire project.

I know it’s convenient as hell, but can we stop pretending that automated accessibility checking has any real meaning whatsoever?

En tout cas, this paper is useful for its massive list of references of previous papers. Beyond that, well, we know that even sites with a requirement to be accessible occasionally aren’t, and most of the errors found in this limited survey bordered on the inconsequential.

“A comparative assessment of Web accessibility and technical standards conformance in four EU states” by Carmen Marincy and Barry McMullin

This one’s got a bit more going for it, despite also using automated testing. Excerpts:

• The typical practice of many Web content developers is to test Web site functionality only against a small number of “popular” Web browser platforms in “normal” configurations. Although the content might seem to be rendered correctly in these tests, this does not guarantee that it is designed correctly. By definition, users of specialized assistive technologies are not using “popular” platforms – or at least, not using them in “normal” configurations. Rather, they must depend on equipment tailored to their particular needs. As a result, it frequently happens that sites are poorly accessible, or completely inaccessible, to such users….
• However, because they do not conform to technical standards for interoperability, their rendering is – at best – unpredictable. This is likely to have a disproportionate affect on users who rely on specialized, tailored, client technologies – specifically, users with disabilities. Content may thus fail to be rendered, may be garbled, or may be otherwise inaccessible to such users. Worse, precious development effort in individualizing assistive technologies may have to be spent on attempting to compensate for these server-side defects, rather than improving the client-side functionality that the user really needs. In the worst case, this effort may have to be wasted repeatedly for each different client accessing each different (non-conformant) server. Obviously, conformance to technical interoperability would substantially reduce or eliminate this waste.
• Only four U.K. sites (less than 0.2%) and six German sites (less than 0.4%) had completely valid HTML markup. No Irish or French sites had completely valid markup…. Additionally, even after reducing the samples on the basis of usable DOCTYPE information, just one Irish site (3%), 29 U.K. sites (4%), 16 French sites (4%) and 29 German sites (2%) had valid HTML markup in the remaining pages….
• The site capture mechanism used in this study suffers from significant limitations with respect to sites which require user “registration.” More generally, automated evaluation of “interactive” sites is fundamentally problematic. This is particularly important given the growing number and diverse roles of such sites. A special issue relates to those sites that allow users to “personalize” site appearance or behavior. In those cases, it might be argued that such a site’s content can be tailored to the needs of each user, and is therefore fully accessible even though this may be invisible to an automated mirroring robot. However, since a user needs to access the default Web presentation in order to “personalize” it in the first place, the default configuration – the one that would be automatically retrieved – should be conform to WCAG 1.0.

“Sight into Sound” by Lisa Gibson (232 K PDF; text-only)

My esteemed colleague Lisa Gibson was in town from Oz last year on her Drowned World Tour researching audio description. They don’t really have DX in Australia, you see, and her stated goal was “to study and compare different mediums relating to audio description and gain an understanding of the impact of relevant disability access policies in USA, Canada and UK.”

It was curious indeed not to read of Lisa’s triumphant viewing of X-Men 2 here in Toronto. Hey, I was there. It happened.

Also, the section on CRTC audio-description requirements is not quite up-to-date, but then again, even I am not maintaining a centralized list. (I really should, I suppose.) There’s also been a bit more “consumer feedback” on “film-based art” like TV and movies than the report suggests. (I have most of the studies, actually.)

A finding of note:

Discussion was held as to whether the accent of the describer and use of colloquial terms should match the country of origin or the international viewing audience. Many highlighted the need to educate the viewer to new terms, if country-of-origin terms were used (flat vs. apartment [as a rudimentary example]), and a contrasting accent to their unaccustomed ear. In all countries, debate surrounded the use of politically-correct terminology. Whether identification should be made regarding things like race, physical appearance, people with disabilities, etc., and which terms to use. In essence, even censorship [sic] finds its way into audio description in some parts of the world.

And some of the implementation plans are of interest:

• To hold an annual conference to share, discuss and develop the different facets of audio description with practioners, stakeholders and users.
• To work towards inviting international speakers to the annual conference. [...]
• To establish accredited training guidelines and an accreditation process for audio description as a whole within Australia.
• Incorporate voice training into the training programme to enhance the delivery of the description.

Dedicated readers will have slogged through my turgid War and Peace of an exegesis on the accessibility and usability failings of the Scientific Atlanta Explorer 8000 PVR. Curiously, somebody else in town has been all up PVRs’ noses: Teehan + Lax (not Tegan & Sara), a small hive of graphic designers–cum–usabilitistas.

You may wish to download their inaccessible, untagged PDF report on usability failings of Bell ExpressVu and Rogers PVRs. (Curiously, ExpressVu was an early client of mine.)

I visited the duo of Geoff Teehan and Jon Lax and their three staff last week, and, in a frequently-pleasant conversation, we discussed usability, accessibility, and graphic design. As you can see on their Weblog, the company is arse-deep in plans for some supersecret new technology that has to be either a new software platform or another PVR- or Windows XP Media Edition–like product. Certainly they have no interest whatsoever in improving existing products.

So I told them they’d need to include accessibility, and I intended to hold them to that. Tons of my friends and I can help.

The BBC Web presence has been reviewed by an outside consultant, Philip Graf. Press coverage has focused on the sections of BBC Online that will be shut down, and how “external content providers” can add to the site.

• Independent Review of BBC Online
• More-convenient PDF links on this page of BBC coverage

Torin Douglas writes: “Amazingly, Philip Graf seems to have pleased everyone with his report into the BBC’s online services.”

Well, except disabled people. The issue of accessibility is barely even mentioned in 290 pages of reports (including the Independent Review of BBC News 24), all of them published as tagged PDFs.

What do the reports say about accessibility?

Let’s check some of the references, taking care to sort out the disability sense of accessibility from others. Glancing references are excluded.

BBC Online Review, Module 1: Assessment of BBC Online’s Use of Technology (PDF)

The overall site design and accessibility of BBC Online has undoubtedly made the service easy to use… A number of features are also in place to help users with visual or certain physical impairments to navigate their way around the site relatively easily. Every page has a “Text only” link which, when clicked, redraws the page, removing all of the graphics and other objects, leaving just the text. This feature uses an application developed by the BBC called Betsie… which also allows the user to change the colour and size of the text, as well as removing embedded items, such as games and applets. All BBC Web pages have also been created so that the font size can be changed in the client’s browser to make reading easier. [...]

To date, there has been only one instance of BBC Online actively contributing to the “open source” community – an accessibility tool called BETSIE was developed by BBC Online to enable text within Web pages to be adapted in size to suit the needs of visually-impaired users.

1. Text-only pages not only are not accessible, they’re a form of apartheid. The idea that text-only pages – no matter how presented or by what clever tool – are a form of accessibility is entirely wrong. This is quite the red flag to wave before accessibility advocates.
2. Font properties and colour can be controlled much more finely with stylesheets; you don’t need Betsie.
3. The ability to remove embedded items is of dubious merit, though I can see some applications. But why not make the embedded items accessible?
4. Font-size alteration is only a problem using some kinds of px (pixel) sizing and only in Internet Explorer for Windows. Avoiding the problem on a commercial site is so rudimentary it’s not worth mentioning.

Report of the Independent Review of BBC Online (PDF)

[C]entral New Media defines and manages standards in a range of areas which include… [a]ccessibility standards, such as usability for visually-impaired audiences.

Separately, users and non-users of BBC Online ranked “services for specific client groups” like “people with disabilities” as “important” or “very important” about three-quarters of the time.

Under “content issues”:

…the need for accessible and appropriate services for the elderly, the partially sighted and people with learning difficulties. “…It is therefore of prime importance for the BBC to take full account of the issue of accessibility and to consider for themselves whether the information that they provide is truly accessible by everyone who needs it” – Disabled Living Foundation

Service objectives flowing from social purposes: [S]ervices considered to be of social value, but which the commercial market place is not providing and is unlikely to provide [such as] services for users with disabilities (e.g. Ouch!).

Did the authors know of the existing research?

Apparently not.

The BBC is one of the few organizations anywhere whose Web accessibility has been studied through usability testing with actual disabled subjects.

→ Accessibility study of BBCi: Problems faced by users with disabilities (PDF; previous mention)

The report found, in part, that:

• BBCi was a “medium-compliance site” (but then again, it named Amazon.co.uk as a high-compliance site). Users with all disabilities tested, including deafness, had notable trouble using the site, but total inaccessibility and complete user frustration were rare.
• Text-only versions were ignored. “None of the participants selected the text-only version of the site, despite the fact that some of our participants (those with partial vision and dyslexia) had used an ‘accessible version’ of sites before…. One reason for not selecting the text-only site may have been because they were unaware of this facility…. Many disabled people express a dislike of separate, text-only sites. There is a concern that the text-only site may be out-of-date compared to the main site and it may exclude certain information…. [S]ome disabled people regard it as disempowering because choices are being made on their behalf that result in less information. A frequently-repeated criticism of text-only sites is the dislike of ‘special’ provision, as it is felt to be stigmatizing.”

The current Graf report disregarded the existing research. It’s no surprise the report pretty much ignored accessibility, and, in a self-incriminating touch, presented text-only sites as a claimed accessibility triumph.

Does the BBC know what it is up against?

As mentioned before, the BBC plans to digitize an untold sea of audio and video material and make it available online (if only to British subjects).

We have, as it stands now, a set of accessibility deficiencies with the BBC’s text-and-graphics sites and its relatively uncommon multimedia. Now add thousands of hours of multimedia. Is everyone aware of the enormity of the task of making the entire experience accessible? Do you have any idea what an undertaking is involved in captioning, audio description, and transcription of such clips, let alone configuring thousands of database-generated pages to serve them correctly?

Has accessibility even been considered?

Surmounting politics

BBC Online, with a reputed two million pages, is not a badly-run Web site. In fact, it is widely admired and even I like it much of the time. They do a lot of things right, and I know for a fact that smart people work there. It’s a pretty solid foundation, really.

Except:

The Graf report was politically motivated. It was only commissioned because of private-sector complaints that the public broadcaster was treading on ground the private sector coveted. The CBC hears the same accusations from time to time.

The response has been equally political, as, for example, in shutting down five BBC Web sites. I fear that the response will continue to be driven by superficial and ignorant misunderstandings of the Web – the sort of thing we suffered through in the dot-com era:

Imagine the scenario of a qualified developer attempting to present a really solidly built Web site to a manager who can barely type. All he (or, rarely, she) knows about the Web is that the brighter and more saturated the colours, and the more blinking and flashing, the better. That is his (or, rarely, her) sole method of evaluation. It’s very high-level if it can be said to have a level at all, and it has nothing to do with how real people, with and without disabilities, use the Web. It also ignores relevant expertise, as from usability, standards-compliance, and accessibility authorities.

If BBC Online really needs to be fixed, do so intelligently. Start with users, including users with disabilities. Do no harm to them. This will require an inclusion of usability and accessibility at the ground level rather than trying to tack it on later. We are dealing with a very large task of technical architecture; it’s necessary to build accessibility into the very fabric of the improved site. Doing so may require jettisoning some ideas attractive to non-experts that can’t be made to work, but on the other hand, it ensures that every idea that does work works for everybody who uses the site.

In other words, if this is the set of problems we’re given, let the experts solve them.

Version history

2004.07.06

Posted.

2004.07.15

Title changed (from “BBC Online: You’re in trouble”) because the latter was actually unrepresentative. Thanks to an esteemed colleague on that sceptred isle for yelling at me, which resulted in my realizing that.

Support a truly free and robust BBC Creative Archive

Last August, Greg Dyke, the former director general of the BBC, announced that the BBC would soon launch its “Creative Archive” – a project to put much of the Beeb’s programs on the Internet, so that the licence-paying British public could have access to it….

Here are some of the elements critical to the creation of a real, useful, relevant Creative Archive:

• It must be accessible: files must be made available in open, standards-defined formats without “digital rights management” or other technology locks that will keep Britons from creatively re-using the BBC’s offerings.

Well, yes, but indeed DRM has implications for accessibility. And in any event, when I hear the word “accessible,” I take out my screen reader.

Is the BBC quite aware of the work involved in captioning and describing decades’ worth of video?

Don’t get bitten by Jaws

“Hmm, $40 for a 60-day evaluation of Jaws. OK, I’ll get it, learn it, and then I’ll be able to test for accessibility by people with poor vision.” Wrong. Very, very wrong. As a sighted person, I found it exceptionally difficult to make any sense at all of web pages using Jaws, even those that had been thoughtfully designed for accessibility. There was just too much information coming at me….

A person using a Jaws is actually using three applications at once: the web site or web application, the browser, and the screen reader. If you’ve tried to teach a complete web novice such as an older relative then you may be familiar with the issues this creates, trying to help them understand when they should be looking at the grey bar at the top and when to concentrate on the meat of the page. Now add in a third thing to concentrate on, and remember that you can’t see any of it.

Webcredible boosts UK postal watchdog Web site accessibility

The new website aims to conform to level AAA of the W3C Web Accessibility Initiative guidelines…. The usability needs of non-disabled visitors were also a major consideration in the design. When a site visitor prints a page from the website navigation, formatting and images disappear and the page prints perfectly on paper. Page download time has additionally been improved.

Postwatch was keen to avoid creating a separate text-only version and used the expertise of accessibility and usability specialists Webcredible. “Creating a separate text-only version is the worst thing a company can do when it designs a website,” said a representative of Webcredible. “Not only does it marginalise blind and disabled web users by segregating them from non-disabled users but it can also be costly and time-consuming to implement.

“Web accessibility is not rocket science. It is far easier to implement than most organisations believe and needn’t place any limitations on the design of a website,” he added.

Duocom Receives Two International Awards of Excellence

The Best Overall Staging for a Corporate Event, Large Venue award is considered the “Best Picture Oscar” of the AV Rental & Staging industry and was awarded for Duocom’s work on the 14th World Congress of the World Federation of the Deaf, held in Montreal…. Simultaneous Interpretation of three visual sign languages (French, American and International SL [which does not exist]) [and] numerous spoken languages, as well as closed captioning into French and English[, were] required.

Kodak Demos E-Cinema Solutions at CineExpo

The Kodak system will handle multi-language audio tracks, subtitling, and closed captioning.

Oh? How? And will audio description be deemed a “multi-language audio track”?

Alan Spencer comments on lack of captioning

However, in all my fights to get a release that would satisfy each and every fan. it appears I’ve let a sizable quotient of you down. The reason I didn’t service your needs is because I was unaware I even had to: Sledge Hammer: Season One has not been closed-captioned. I apologize to all fans of “Sledge Hammer!” who are hearing impaired; as I was under the false assumption that closed captioning was mandatory….

The good folks at Anchor Bay, the majority of whom have been lobbying for captioning all along, certainly didn’t set this precedent. They’ve been fighting it. This shameful policy was established by their previous parent company because it saved to few dollars…. There is a bit of good news: Anchor Bay has new owners. Key people, many of who have been on the receiving end of complaints regarding this policy they weren’t responsible for, are attempting to ensure that all their titles are made available to the hearing impaired from now on.

(Cf. Neil Gaiman: Neverwhere; Neverwhere commentary.)

With Cabbie, all’s fair

Some of what they’re saying is anybody’s guess, a mixture of hip-hop
slang and God-knows-what-else. What is clear is that Martinez is ragging on the interviewer about his singing voice and his taste in clothing….

The interviewer is Cabral (Cabbie) Richards and getting pro athletes to let down their guards is what he does…

As for the Cabbie-speak that obviously appeals to the players and to Sportsnet’s target audience, even Sanderson doesn’t understand it. “We got a call from our closed-captioning people one time asking for a script. They couldn’t figure out what he was saying. But there is no script, it’s just Cabbie.”

I always find it amusing when Sportsnet claims to give a shit about captioning.

Narration lets blind enjoy films

[T]he theaters are also looking at installing a $5,000 “rear-window captioning” system for hearing-impaired moviegoers…. [Dave] Sarle had written letters to every movie chain in Utah before he got an OK from the Megaplex 12 Theatres at the Gateway to install the DVS. [T]he sister theaters at Jordan Commons might also install DVS “if we get enough call for it.” Already, he says, The Gateway theater has attracted at least one blind patron who had never been to a movie before.

Great: Another blind person agitating to get his needs served at the expense of deaf people’s.

Is Dave Sarle a hero or a hypocrite?

Good on yez, Dave, for fixing things up for blind people. But whatever possessed you to even suggest that only description should be installed?

And what possessed WGBH to sell them such a system? You’re actively excluding deaf people. Actively including blind people does not excuse it!

Subtitle expert has £20m float in view

“It may not be high profile but it’s a great business,” says Ms Sheridan, who starts marketing IMS today for a planned flotation on the Alternative Investment Market, valuing it at up to £20 million.

(Another story claims IMS “made profits of £800,000 on turnover of £6.5 [million].”)