8.9: Tour of an Audit Report
- Page ID
In the unit Introduction to WCAG 2.0, you were introduced to the “Web Auditing Review Template” and the elements it contains. Here we will look more closely at the “WCAG 2.0 Review” elements of the report: the Executive Summary and the Appendices. Download a copy of the Canvas LMS General Review [PDF] and follow along.
The General Comments or Executive Summary
The General Comments in the Canvas LMS audit can also be thought of as the Executive Summary, and provides an overview of the report.
General Comments or an Executive Summary will commonly include:
- The scope of the audit
- A description of key issues
- Terms used to evaluate issues
In this case the scope of the audit is the student facing components of the LMS, presented below:
“This evaluation looks at the accessibility and usability of the student facing components of the Canvas LMS. It is not an exhaustive review of all features, but rather more general coverage of the accessibility and usability of the student features as a whole.”
Following the scope comes a description of the key issues that have been identified in the report. These are typically the more significant Level A issues that would result in barriers to access for a given group of users, or issues that are recurring throughout the site or application being reviewed. Depending on the scope of the review, this can be anywhere from a few paragraphs to a few pages.
Finally, the General Comments describe the Evaluation terms, before leading into the full WCAG 2.0 Review.
“In places throughout the review, “Fail?” or “Pass?” have been used where a fail or pass is questionable. “Pass?” is used in places where a single instance of a barrier has been identified, perhaps an oversight, or where it could be argued that an item might fail or pass, typically a minor issue, leaning toward a Pass. “Fail?” is used in cases where an item could be argued as a fail or pass, leaning toward a fail. In all cases, developers should consider the recommendations made to remove any potential argument.”
The WCAG 2.0 Review
The WCAG 2.0 Review area of an audit report is a table formatted into 4 columns and 62 rows. Each row represents one of the WCAG 2.0 guidelines. The four columns present the following key elements of the review:
- Success Criterion: The guideline numbers and a brief description of the guideline.
- Level: The conformance level for the associated guideline, i.e., A, AA, or AAA. Note that the AAA requirements are greyed out, indicating that these are optional requirements. You may also notice that Guidelines 1.2.4 and 1.2.5 are also greyed out; these are optional requirements for Ontario organizations, though may be required in other jurisdictions.
- Evaluation: This is the outcome of the evaluation of a particular guideline, and there are 4 options.
- “Pass” is specified if the success criteria for a guideline are met.
- “Pass?” indicates that you have dismissed an issue as trivial, though it should still be considered when making accessibility updates to the site. For example, if just one image on a website is missing alt text, while all others include it, a questionable pass would be specified.
- “Fail” is specified when the success criteria for a guideline are clearly not met.
- “Fail?” is used when it could be argued that a strategy arguably meets the success criteria, but there are perhaps better approaches. In either case, questionable pass or fail, a subjective judgement is being made. This should be clearly stated in the General Comments.
- “N/A” marks those guidelines that are not applicable in the current context.
- Comments: This is the “meat” of a web accessibility audit. Here, potential barriers are described with enough detail so that readers familiar with the site being reviewed are able to reproduce or find the barriers themselves. Reading through the Comments section of the Canvas LMS review, you can gather a sense of the detail provided.
The Comments Area – Use of Graphics
Continuing to look at the Comments section in more detail, you will see that some graphics have been added to explain issues more clearly, particularly where the written explanations may be too complicated or technical for some people to understand, or in places where a key issue needs to be highlighted. And, there are times when just pointing to an issue in a graphic is more effective than trying to describe it in words.
Graphics to Enhance Explanations and Summarize Main Issues
Take Success Criterion 1.4.3 in the Canvas LMS review for instance. In the explanation of the first issue, it can be difficult to pinpoint exactly which colours are problematic. An added graphic explanation, like the figure below, makes it clear which colours are being referred to almost instantly upon viewing the image. In fact, one may be able to understand the issue based on the graphic without needing to read the text explanation.
Figure: Example of a graphic used to enhance the explanation of an issue
The collection of graphics in the report as a whole can act as a summary of the main issues. If you scan through just the graphics in the report, you will notice they address most of the main issues.
The Comments Area – Issue Descriptions
When describing an issue, always include:
- a brief description of the issue itself
- the reason why it is an issue
- one or more potential solutions to correct the issue
For each type of issue, reasoning need only be provided once. Examine the following description of an issue and note these elements in the description.
Reason why it’s an issue
In the main calendar view, the days of the week that appear in the top row should be marked up with table headers (th).
Currently when navigating the data cells for each day, Assistive Technology users hear only the number, with no indication of the day of the week.
scope="col" attribute could be added in each header cell to help ensure day of the week is announced with each number.
For a General Review, the Appendices should include a list of the pages sampled for the review, including the page title and URL.
For a Detailed Review, the Appendices should include a list of page titles and their associated URLS for the process or feature being reviewed. This might include a description of a user scenario that was tested, such as the process of making a purchase (described earlier).
The Appendices may also contain information not directly related to the accessibility review, such as pointing out potential bugs, providing recommendations on processes for addressing issues outlined in the report, or outlining strategies to address accessibility of web content as part of everyday business practice.