Component-based Software Engineering (CBSE) is concerned with the development of systems from reusable parts (components), and the development and maintenance of these parts. This study addresses the issue of usability testing in a CBSE environment, and specifically automatically measuring the usability of different components in a single system. The proposed usability measure is derived from the message exchange between components recorded in a log file. The measure was validated in an experimental evaluation. Four different prototypes of a mobile telephone were subjected to usability tests, in which 40 subjects participated. Results show that the usability of the individual components can be measured, and that they can be priorities on their potential for improvement.