Forecasting Device Effectiveness. Volume 1. Issues

This Technical Report discusses a number of issues that bear on the development of formal analytic methods for predicting the potential effectiveness of alternative training devices. The discussion encompasses theoretical, practical, and methodological issues uncovered during review of the literature and analysis of the problem. With respect to theoretical issues, two fundamental sets of concerns are discussed. First, what is actually meant by the term device effectiveness? In this connection, the following issues are addressed: What is transfer of training? How is it measured? What are the alternatives to transfer of training as measures of effectiveness? The second concern regards the classes and types of variables that hypothetically, at least, influence device effectiveness. In this discussion, the classes of variables that are identified are treated within a program evaluation framework. This structure is introduced to help organize and conceptualize the training system design and evaluation problem. In the discussion fo practical and methodological issues, several topics related to real-world constraints on developing and evaluating a training device effectiveness forecasting procedure are considered. (Author)