The Common Industry Format (CIF)
for usability test reports specifies the format for reporting the
results of a summative usability evaluation. The most common type
of usability evaluation is formative, i.e. designed to identify usability
problems that can be fixed. A summative evaluation produces usability
metrics that describe how usable a product is when used in a particular
context of use [3,7]. The CIF
report format and metrics are consistent with the ISO 9241-11 [5]
definition of usability:
The extent to which a product can be used by specified users
to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use.
The type of information and level of detail that is required in
a CIF report is intended to ensure that:
- Good practice in usability evaluation had been adhered to.
- There is sufficient information for a usability specialist to
judge the validity of the results (for example whether the evaluation
context adequately reproduces the intended context of use).
- If the test was replicated on the basis of the information given
in the CIF, it should produce essentially the same results.
- Specific effectiveness and efficiency metrics must be used,
including the unassisted completion rate and the mean time on
task.
- Satisfaction must also be measured.
It was envisaged that a supplier would provide a CIF report to
enable a corporate purchaser to take account of usability when making
a purchase decision. A purchaser could compare CIF reports for alternative
products (particularly if a common set of tasks had been used).
The purchaser might specify in advance to a supplier the required
values of the usability measures (for example based on the values
for an existing product).
The original motivation for the CIF came from usability staff in
purchasing companies who were frustrated at purchase decisions made
exclusively on the basis of functionality. These companies experienced
large uncontrolled overhead costs from supporting difficult to use
software [4].
The CIF format was agreed by the IUSR
working group of usability experts from purchasing and supplying
companies, including companies such as IBM, Microsoft, Hewlett-Packard,
Boeing, US West and Kodak.. It was based on collating good practice
from the different companies, and aligning this with ISO 9241-11.
|