At both IR and IAI, a usability maturity assessment was used to identify any gaps in the organisations’ ability to apply user centred design. The assessment was based on the Usability Maturity Model, which consists of a list of activities that represent good practice in user centred design. In the assessments, each activity was rated on the scale:

Inland Revenue/EDS: full assessment

At Inland Revenue a formal assessment lasting one week was carried out by two assessors from Lloyds Register, assisted by two usability specialists from Serco Usability Services who identified opportunities for process improvement. A total of 13 stakeholders associated with the trial project at different levels in Inland Revenue and EDS were interviewed in twelve 3-hour sessions. A conventional software process assessment procedure based on SPICE (ISO 15504) was used. This produced a detailed profile and rich information about where improvements would be beneficial. At a feedback meeting on the last day, it provided the basis for an agreed set of improvement activities.

Inland Revenue comments:

• The different ways user could and should be involved throughout the lifecycle;
• The benefits that could accrue to both the project and IR/EDS;
• Professional support available from Lloyds Register and Serco Usability Services.

12 months later when the improvements had been made, a second similar assessment was carried out to see whether the improvement objectives had been achieved. Very significant progress had been made. When the results were presented to a meeting of senior stakeholders, the benefits were sufficient for the meeting to authorise incorporation of most of the methods into the standard Inland Revenue/EDS documented processes. The meeting also suggested that regular usability maturity assessments should be arranged to monitor improvement.

IAI: workshop

By contrast, a simple one-day workshop provided the basis for process improvement at IAI. The activities in the Usability Maturity Model were used as a good practice checklist. The Serco usability specialist rated each activity as not performed, largely performed or managed, based on a short discussion with one or two developers or managers who were most knowledgeable in each area. Although some ratings may not have been completely representative, they were sufficient to provide the basis for an agreed programme of improvement.

A second workshop was held 16 months later. As the Serco usability specialist who had been carrying out the improvements also organised the second workshop, it was easy to agree on the extent of the improvement with the IAI representatives in less than two hours.

IAI comments:

The one-day assessment format was appropriate for LAHAV since it is a) a relatively small organization, and b) it has a lasting culture, commitment and infrastructure for process improvement.

The first assessment revealed many areas that needed improvement including some organizational issues. These were used to select UCD methods for trial. The second assessment purpose was to evaluate the improvements made. The detailed results are very valuable and will be used in further dissemination activities in LAHAV and other IAI divisions.

Comparison

The maturity assessment was highly effective in setting an agenda for both organisations. Could the simple one-day workshop-style assessment have been used at IR/EDS? This question was asked at the final stakeholders meeting at IR/EDS, and the opinion expressed was that IR/EDS needed the confidence provided by the thorough and impartial software process assessment procedure.

Under what circumstances can major benefits be initiated by just a simple workshop? In a large and complex organisation like IR/EDS, it is more time consuming to identify the current situation and more complicated to obtain consensus on change. The formality and structure of a SPICE assessment is consistent with the organisational culture, and provides the necessary detail and authority for the results obtained.

By contrast LAHAV at IAI has a smaller more flexible group where it is easier for all the main stakeholders to be involved and make rapid decisions.

Both organisations found the results so beneficial that they have adopted the methods as a normal part of their development process. At IR some methods will be applied by usability specialists, while IAI found them sufficiently intuitive for use by members of the development team.

Lessons learned

Does this provide a model for how to introduce user-centred design in other organisations? The common features of IAI and IR were that they shared goals to:

• provide systems that meet user needs
• improve their processes

But even these organisations had initial difficulty in understanding the potential benefits of the methods, which differ in nature from other software engineering activities. It is still not clear how best to present the proposed user centred design activities in a way that can be understood and appreciated by designers and developers.

In these case studies an extra incentive was that the trials were subsidised by the European Union. It is to be hoped that dissemination of the successful results will provide an incentive for other organisations to invest their own resources in carrying out similar improvement initiatives. There is every reason to believe that the benefits obtained by IAI and IR could be replicated in other similar organisations.

Last updated 13-Oct-00.