How to Measure Usability Preference Vs. Performance
The idea of usability in interactive systems design is an abstract one. Because of the complexity of the concept of usability, system design engineers divide usability into two types. First, there is usability preference, which is subjective. Then, there is usability performance, which is objective. In designing interactive systems, it is often important to measure both aspects of usability and compare them. This way, system design engineers can tweak their systems to make both aspects of usability fall in line with each other.
Other People Are Reading
Instructions
-
-
1
Measure usability performance. Have users perform a set task on the system. Measure how long it takes each user to complete the task. If there are possible errors that the user can make during the task, note these down and add them to the user’s time in a reasonable and pre-decided manner. For example, if the errors are not terminal, add one or two seconds to the user’s time for each error.
-
2
Measure usability preference. Create Likert scales that allow users to evaluate the system for each important component. A Likert scale is one that asks a question that requires a level of agreement as a response. The possible answers on a Likert scale typically number 1 to 5 For example, “Agree,” “Somewhat agree,” “No opinion,” “Somewhat disagree” and “Disagree”. Administer these Likert scales to the users, collecting their information.
-
-
3
Run a regression analysis on the two measurements. Use standard statistical methods of linear regression (using statistical software simplifies the process), to create a model in which performance is regressed on preference.
-
4
Evaluate the regression model’s statistics. The main statistics of interest in the regression’s output are the correlation and p-value. The correlation will be a number “usually written as “r” between -1 and 1. The p-value will be a number between 0 and 1, written as “p.” Look at the p-value first to know whether the relationship is significant (i.e. whether a comparison between performance and preference is valid). The p-value should be less than 0.05 if the comparison is to be statistically valid. If the p-value is less than five, the correlation il likely around 0, indicating that the usability preference and performance have no set relationship. That is, you can take performance and preference as two separate ideas for the system. If the correlation is close to 1, it means that user preferences are in line with usability performance. That is, the system is set up in a way that users not only find it easy to use the system but also can use the system in an effective manner. A negative correlation close to -1 implies that the usability preference and performance are not in line. That is, users do not prefer the effective form of the system.
-
1
References
- Communications of the ACM; “Measuring Usability: Preference vs. Performance”; Jakob Nielsen, et al; April, 1994
Resources
- “Information Visualization”; Robert Spence; 2007
