In Development

The 2008 assessment of government web sites is under way now, coordinated by the Web Standards team at SSC.

We have 500-something sites to test and QA, so I hope it’s useful to talk a little about the process, how it’s being handled and - in later posts - how it’s progressing.

What’s the objective?

The objective is to provide a 10-point ‘health check’ based on the Web Standards 1.0 and good practice. Some of the 10 points are direct measures of compliance with specific standards, some are indicators of good practice, such as;

• Does the search engine return meaningful and relevant results?
• Does the site use language broadly appropriate to its audience?
• Do pages scale well if fonts are enlarged?
• Do pages have a meaningful semantic structure?

The 10-point checklist is on the Web Standards wiki. Each checkpoint is assessed by testing up to five indicators to arrive at a ratings of Good, Room for Improvement, or Fail. You could read these ratings as;

• Good
• Not a blatant failure as such, but worthy of improvement when possible
• (you guessed it) A blatant failure - fails the requirements of the Standards, or is considered less than good practice, and should be fixed promptly.

The intended outcome is a report back to agencies providing a snapshot of their website(s) and areas where they could put some focus, or (where necessary) a heads-up on problematic sites.

It’s worth noting here that

• agencies should be conducting annual self-audits of their standards compliance using the tools available on the wiki, and
• some agencies are mandated to comply with the Web Standards 1.0 and others are invited to do so. The assessments should be seen in that context.

What it’s not

• A name and shame
• A league table - best to worst
• Any substitute for user testing and accessibility testing

The methodology

In a nutshell;

1. The assessment criteria (the 10 points and indicator assessments) were developed by the Web Standards team. The Web Standards team consulted with the Web Standards Working Group and a specially-convened User Testing Working Group to refine the criteria (and the methodology).
2. The testing is being carried out by several New Zealand companies which specialise in testing web sites - primarily accessibility testing as this is the foundation for the current Web Standards. The testing criteria were further refined in initial discussions with the testers.
3. Each site assessment is returned by the testers to the Web Standards team for QA and a consistency check by a three-person team.
4. Agencies receive a health-check report as above.
5. The Web Standards team gains an insight into trends and common issues (text alternatives or summaries for PDFs might turn out to be one such, vision-safe colours might be another) across the government domain. The findings might identify areas where we could provide further education or whatever, and provide a sound base for the next assessment round.

And that’s it. There’s a bit more detail underneath the process, but that’s the outline.

Where we’re at so far

Testers completed their first few assessments (a handful of sites each) a couple of weeks back, and the results went through an initial QA/consistency check. Findings from this first round resulted in a more precise ‘level-set’ for testers (the level of ‘descriptiveness’ of an issue, URLs for specific issues, and levels of detail for suggestions for improvement, in the main). Testers are now well into the bulk of the 500 or so sites, and all results are being QA’d. The focus of the QA team is to normalise the assessments so that the process is fair to all agencies, but still allow useful analysis and identification of key issues across government web sites.

More to come…

Technorati Tags: accessibility, assessment, Web Standards