Much early usability work used summative methods [14], but was not always supported by other user centred design activities. It therefore gained the reputation for being an expensive way to identify problems when it was too late to fix them! So the emphasis moved to formative evaluation (so-called "discount" usability methods) that could be used earlier in development [9]. It is essential to introduce usability early in the development process, but without subsequent summative testing, it is difficult to judge the effectiveness of the usability work [2].

Summative usability testing using the CIF has advantages to both consumer and supplier organisations during procurement. It is one of the most effective ways to enrich the consumer's requirements document with objective user performance and satisfaction goals (based on the existing system/product used). It provides a platform on which to evaluate potential competitive products from a number of supplier organisations during the procurement of a new system/product. By using the CIF test structure, suppliers are able to demonstrate that their product complies to the usability metrics defined in the requirements document.

The PRUE trials have demonstrated the potential value of the CIF in a wide range of European environments, and this experience with the CIF contributed to the final version of the text for the CIF standard.

The adoption of the format as an international standard should provide a strong case for the wider use of summative testing reported in the Common Industry Format. The PRUE case studies provide strong evidence for the benefits of this approach. The CIF builds on the methods developed by the EU MUSiC project [3,7], and it is appropriate that it should be used in Europe.

Last updated 12-Mar-02