Cognitive accessibility tested in Norway

As more and more public services are being digitised, there are increasing demands on citizens to be able to use these services independently. For many users, it is difficult to manage digital services from a cognitive perspective. It is therefore important that public digital services are understandable, both in terms of navigation, appearance and content.

In the project, we have applied four guidelines for cognitive accessibility to prototypes of digital services in one of Norway’s key public sector websites, the Labor and Welfare Administration (nav.no).

The user tests were carried out as A/B tests on prototypes containing the actual content of the services. The texts were not altered during the tests, only the layout, order and placement of objects. The testers were 30-70 years old, mainly from target audiences identifying themselves as neurodiverse, with a gender balance of 50-50.

When collecting data for the needs analysis, we collaborated with NAV to select services that users find cognitively challenging to access and that are much in use. Among other things, these services have a lot of text to get through. All these services are also aimed at people who may have cognitive impairments, due to age, disability, or life situation.

We then analysed these services to see what cognitive barriers might be present on these interfaces and what types of existing cognitive requirements might be relevant to use for the user testing. The goal was to select services with different types of cognitive barriers to test. In this step, we matched the user needs from the literature with the selection of services for prototypes. The match was verified with user organisations for people with cognitive disabilities.

The requirements tested:

1) Placement and design of controls should be consistent; having similar activation patterns for similar actions.

Source: Guideline developed in the Vinnova-funded research project "Cognitive Guidelines". Available on cogreq.eu.

• Focus area: Initiating, organising, performing a task

2) A mechanism is available to identify specific definitions of words or phrases used in an unusual or limited way, including idioms and jargon.

Source: WCAG 2.1 AAA Success Criterion 3.1.3 Unusual words

• Focus area: Reading and understanding text

3) Clarify the purpose of large amounts of text content before presenting the content.

Source: Guideline 7.2.1 of ETSI EG 203 350 v1.1.1

• Focus area: Reading and understanding text

4) Organise content into well-defined groups or chunks using headings, lists and other visual mechanisms.

Source: WebAim guidelines for evaluating cognitive accessibility

• Focus area: Attention (directing, retaining, switching)

All user tests show that the recommendations are relevant and have a positive impact for users. However, not all of the solutions tested in the project have produced the best results for users. Too much information on the same page makes it difficult for users to find their way around and reduces their confidence in using the service (increasing stress). This is regardless of whether the information is well structured or not.

To noone’s surprise, an important conclusion is to always test every solution with users before deciding on the final design and long before implementation.