Willem-Paul Brinkman, Reinder Haakma, Don Bouwhuis. Component-Specific Usability Testing. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 38(5):1143-1155, 2008. [doi]
This paper presents the results of a meta-analysis carried out on the results of six experiments to support the claim that component-specific usability measures are on average statistically more powerful than overall usability measures when comparing different versions of a part of a system. An increase in test effectiveness implies the need for fewer participants in usability tests that study different versions of a component. Three component-specific measures are presented and analyzed: an objective efficiency measure and two subjective measures, one about the ease-of-use and the other about the users’ satisfaction. Whereas the subjective measures are obtained with a questionnaire, the objective efficiency measure is based on the number of user messages received by a component. Besides describing the testing method, this paper also discusses the underlying principles such as layered interaction and multiple negative-feedback loops. The main contribution of the work described is the presentation of component-based usability testing as an alternative for traditional holistic-oriented usability tests. The former is more aligned with the component-based software engineering approach, helping engineers to select the most usable versions of a component.