Reliability and Validity of a Procedure To Measure Diagnostic Reasoning and Problem-Solving Skills Taught in Predoctoral Orthodontic Education.

Preliminary psychometric data assessing the reliability and validity of a method used to measure the diagnostic reasoning and problem-solving skills of predoctoral students in orthodontia are described. The measurement approach consisted of sets of patient demographic data and dental photos and x-rays, accompanied by a set of 33 multiple-choice items with from 2 to 10 options. Students were only able to complete two exercises in a 50-minute testing period, so that content specificity of any sizable magnitude might be a problem. Members of a second-year dental school class were divided into two groups, of 35 and 33 students, to take two versions (each containing two different cases) of the examination for a total of four different cases. Prescriptive items showed uniformly higher internal consistency reliability estimates than did the descriptive items. The intercorrelations among data from the same test administration, correlations between scores for different problems at different testing times, and correlations of class attendance with test scores were all sufficiently large to support the construct validity of the measurement procedure. The method appears to offer a viable method of assessing problem-solving skills in orthodontics, with potential for uses beyond undergraduate education. Three tables present reliability data. (SLD)