The roles of immediacy and redundancy in relative subjective workload assessment

Subjective workload assessment is one of the most frequently used tools for system evaluation. However, little is known about which factors contribute to the sensitivity, validity, and reliability of subjective workload ratings. One major classification of subjective workload assessment tools is based on relative judgments of the workload of different task conditions. The present research evaluated different approaches to relative subjective workload assessment in terms of their sensitivity to demand manipulations, concurrent validity with performance, and test-retest reliability. Results indicated that performing relative judgments retrospectively after having experienced all task conditions was superior to rating each task condition immediately after performing it. Further, redundant relative comparisons may produce more sensitive ratings than do relative comparisons to a single reference task. Overall, the results support the use of retrospective relative workload judgments as an evaluation tool.