Ensure the new Portland.gov website provides equal access to all visitors regardless of visual, mobility, auditory, and cognitive abilities, by establishing a set of standards, tools and processes to help developers correctly plan, design, and develop the project; and by discussing tools and methods to support content editors in providing accessible content.
Will we be establishing a target comprehension level for site content? If so, what is the target?
Purpose of this document
Evaluate and compare accessibility testing tools
Discuss areas where automated testing is not sufficient to meet accessibility goals
Make recommendations for integrating accessibility testing into the development process, and for ongoing testing pages and content as it is entered by site editors
Developers will need to adopt accessibility as a mindset, and integrate best practices throughout the planning, development and acceptance phases, not just as a final test;
Accessibility best practices will need to be integrated into content editor training; features and processes will need to be developed to re-enforce these practices
The legacy site contains numerous electronic documents and time-based media that must be individually addressed [How many? What is the impact?]
WCAG 2.0 Checklist
This checklist cannot be used to verify conformance with WCAG 2.0, and should not be referenced in policies or policy adoption. It is intended to be a helpful, informational resource for technical implementation, recommended techniques, and success criteria. It is WebAIM’s interpretation of the guidelines and success criteria.
Web Content Accessibility Guidelines 2.0
Very clear, easy to understand checklist that includes WCAG 2.0 and Section 508 guidelines.
Web Content Accessibility Guidelines (WCAG) 2.0
This is the official specification that should be referenced when determining policy and success criteria.
W3C WAI Evaluation Tools List
A comprehensive list of accessibility evaluation tools
Useful reference of accessibility best practices for SVG images.
The POWR project will adopt the A and AA levels of conformance as described by the WCAG 2.0 guidelines. Where feasible based on timeline and budget, the project will attempt to meet the AAA level.
The official W3C guidelines outline the following four principles of accessibility and associated guidelines:
Not all of these guidelines can be thoroughly tested using automated tools. The tools can identify when page headings are out of hierarchical order, whether landmarks and ARIA enhancements are used, or whether alternative text is missing from images, for example, but it cannot determine whether the headings or alt text convey appropriate meaning, or whether the “pages appear and operate in predictable ways.” Human evaluation will always be needed to ensure the project conforms to the more subjective guidelines.
The base CMS for the POWR project, Drupal 8, includes numerous features to support accessibility and the WCAG 2.0 standard, both for developers and content editors, which are not described in this document. However, testing processes should be established to set a baseline of compliance and ensure the accessibility goals are being met.
Several classes of accessibility testing tools are described here, for testing web pages, electronic documents, and content. The tools are further identified by type:
Browser extensions (“Ext”)
Desktop applications (“Desktop”)
Command line interface applications (“CLI”)
Web-based applications (“Web”)
Applications that have an API that can be integrated into other tools (“API”)
Web Page Testing Tools
|WAVE Evaluation Tool (Chrome Extension)||Ext||No||No||$0||Excellent browser ext for developers, seems most useful in its class|
|Accessibility Developer Tools (Chrome Extension)||Ext||No||No||$0||Not as useful as WAVE but may be useful for troubleshooting|
|PayPal AATT||Web API||Yes||Yes||$0||Seems a good solution for full-site testing, NodeJS app|
|Accessibility Bookmarklets||Ext||No||Yes||$0||Not as useful as WAVE|
|PowerMapper SortSite||Desktop||Yes||No||$149||Solution for full-site scanning|
|Pa11y Dashboard||Web||No||Yes||$0||Web interface for Pa11y, doesn’t seem super useful|
|Pa11y CI||CLI||No||Yes||$0||Not a crawler, intended for CI, so only as useful as accessibility can be detected in Drupal code or in the content in the integration environment|
|Pa11y-crawl||CLI||Yes||Yes||$0||NodeJS app, variation of Pa11y that crawls sites|
|The A11y Machine||CLI||Yes||Yes||$0||Crawler, HTML5 requires Java, reporting seems good|
|Total Validator||Desktop||Yes||No||GB£35||Crawler, comprehensive reports but a little hard to read|
Electronic Document Testing Tools
Electronic documents must also be tested for accessibility, though there are scant free or low cost tools available for automated testing, and none were found that can scan files while crawling a site. Manual document testing and manual updates will be required.
PDF Testing Tools
|Acrobat Pro (XI or DC)||Desktop||No||No||-||Can analyze and fix PDF files, useful tool but can only process single files, already licensed by City|
|PDF Accessibility Checker (PAC3)||Desktop||No||No||$0||Single document scanner, not great, identifies problems but doesn’t offer solutions|
Office Document Testing Tools
|Microsoft Office 365||Desktop||No||No||$0||Can analyze individual files and provide recommendations for fixing issues, already licensed, informed by WCAG 2.0 but doesn’t link issues to specific guidelines|
Determining whether text content is readable and understandable is particularly challenging. Tools exist that can roughly determine the reading level of text content, but overall, content must be manually reviewed by humans. This requires thoughtful evaluation and use of checklists to determine adherence to the WCAG principles.
Microsoft Word and Outlook have a built-in tool to estimate the Flesch-Kincaid readability level of documents. There are also numerous online testers available for analysis of individual web pages or text snippets, and there may exist tools that can crawl entire websites, if that level of content testing is desired.
It would be handy to utilize a built-in readability checker in the new CMS, but no Drupal 8 modules currently exist for this. There are some modules for Drupal 7 that we might consider porting to 8 if this is a desirable feature.
Developers utilize built-in Drupal 8 accessibility features as much as possible.
Developers use one of the browser extensions or bookmarklets to periodically test files/templates/pages during development and unit testing. Every template, widget, page, etc., should be checked before acceptance and promotion. The WAVE Evaluation Tool, or similar browser extension, is recommended.
The full site is fully tested at the end of each sprint using a crawler tool, as part of acceptance. In-depth evaluation of the crawler tools is needed before a specific recommendation can be made.
After/during content migration, all electronic documents (PDF, DOC, PPT, etc.) to be retained are scanned for accessibility compliance and updated accordingly, or alternate content is created.
After/during content migration, site is scanned using a crawler tool to identify any issues in both legacy and newly created content.
After launch, the site is scanned on a regular basis using a crawler tool to verify accessibility standards are being followed by editors and allow issues to be resolved. This includes scanning of electronic documents, if feasible.
Use of Continuous Integration tools for accessibility testing is not recommended, since CI testing is only as useful as accessibility can be verified in Drupal code files or in the content that exists in the integration environment. This seems like it would be more trouble than it’s worth.
Accessibility issues are considered bugs, given the same gravity as other code defects, and logged and processed as such. The severity of the defect is somewhat related to the WCAG criteria level (A, AA, AAA).
Tools, messaging and documentation are integrated into the new CMS to help content editors create accessible content and documents.
If there are comprehension and reading level requirements, additional study will need to be made of testing tools and techniques, as well as tools for assisting content editors.