Search

Multiple-choice tests are used widely because of obvious practical advantages, but inherent in them is the problem of dealing with the effects of guessing. The only widely used model for predicting effects of guessing is based on the primary assumption that, the probability of making a correct response on each test item is either 1.0 or the reciprocal of the number of available choices. The assumption that the subject either makes the correct response on each trial or guesses blindly is clearly too restrictive. Nunnally in [1] relates that because in most educational tests there is some opportunity for "narrowing" alternatives before guessing the standard correction for guessing tends to be an underestimate of the actual effects of guessing. The model proposed in this paper attempts to correct for this.

Various forms of optical memory technology (OMT) have recently become commercially viable products and thus available for use by educational measurement professionals. A single optical disk can hold the equivalent of over a thousand floppy diskettes. Further, many optical media can also be easily and inexpensively reproduced. These two characteristics make optical media a promising technology for the distribution of test item banks, test reports and statistics, student records, and other educational data. Unfortunately, factors such as high initial cost, slow retrieval speeds, and the lack of standardization of formats mitigate the use of OMT. In addition, the enhancements offered by OMT over competing media (such as magnetic disks) come into play in relatively few educational measurement situations. The article calls for a careful consideration of the true merit of such technologies and cautions against adopting the technology without an unbiased assessment of the practicality of the venture.

Researchers who design tests involving judgments of pitch face the persistent problem that test subjects may remember a pitch associated with one test item and carry that memory over to the next item. This immediate memory does weaken with time, but it weakens quite slowly. Although a number of studies report attempts to "erase" subjects' impressions by inserting distracting sounds— white noise bursts, electronically generated tone glides, excerpts from Schoenberg's Piano Concerto— between trials, no systematic study of the efficacy of pitch eraser patterns has yet been undertaken. An initial attempt was made to identify and measure a reasonably short list of variable attributes of distractor tones that might undo a listener's memory for the pitch of a test tone. Those variables were the length of the series of distractor tones, the rate at which distractor tones were presented, and the temperament system—or lack of it—used to tune distractor tones. Test results indicate that tuning variables have little effect on listeners' accuracy at identifying test tones. Series length and presentation rate, however, both have a strong effect on accuracy levels for pitch memory.

Sternberg's (1969) model of memory scanning processes is considered to be insufficient for verbal material. In Sternberg's model no considerations are given to coding and decoding processes which are relevant for verbal material. Usually Ss will represent letter strings of words as one unit with a specific meaning. If Sternberg's model should be valid for verbal material the time for coding and decoding of the memorized material has to be considered in addition to the time of coding the test stimulus, comparing the test item with the items of the memorized list and responding. Experimental data with different tasks to find a letter or a word in a list of words or a letter in a word can be explained adequately if coding and decoding processes for words are considered in the Sternberg model.

The nonverbal projective differential test item is described and distinguished from semantic differential items. Projective differential items are pairs of abstract visual stimuli, such as inkblots. Subjects choose one of the two projective stimuli as more similar to a concept being rated. Two studies showed that groups of subjects respond in a consistent manner to such items, with most subjects agreeing that one of the two abstract visual stimuli is more like the concept than the other; this is the projective differential response phenomenon. It was argued that subjects are able to express some aspects of the connotative meanings that concepts have for them through their responses to projective differential items. The potential usefulness of nonverbal items in constructing measuring instruments was discussed.

Three experiments were performed to determine the effect of figure complexity on discrimination performance. In Experiment 1, domestic cats were trained using a successive training method to discriminate vertical mirror image pairs of asymmetric 4-, 8-, and 12-sided figures. In Experiment 2, the same subjects were trained using a concurrent training method to discriminate novel pairs of figures similar to those of Experiment 1. Test results from both experiments showed that (a) visual discrimination performance of the cat was facilitated with test items of greater figure complexity and (b) the concurrent training method was more efficient. In Experiment 3, experimentally naive cats were trained using a concurrent training method to discriminate seven test item pairs. The test items were vertical mirror image pairs of asymmetric 4-, 8-, 12-, 16-, and 22-sided figures as well as horizontal versus vertical stripes and upright versus inverted equilateral triangles. Percentages of correct performance for the figures composed an inverted U function with a peak at the 8-sided figure pair. Furthermore, supportive evidence was found for the common assumption that equilateral triangle and stripe discriminations are relatively easy tasks for the cat.

A multiple-answer multiple-choice test item has a certain number of alternatives,any number of which might be keyed. The examinee is also allowed to mark any number of alternatives. This increased flexibility over the one keyed alternative case is useful in practice but raises questions about appropriate scoring rules. In this article a certain class of item scoring rules called thebinary class is considered. The concepts ofstandard scoring rules and equivalence among these scoring rules are introduced in the “misinformation” model for which the traditional “knowledge” model is a special case. The examinee's strategy with respect to a scoring rule is examined. The critical role of a quantity called the scoring ratio is emphasized. In the case of examinee uncertainty about the number of correct alternatives on an item, a Bayes and a minimax strategy for the examinee are developed. Also an appropriate response for the examiner to the minimax strategy is outlined.

Subjects carried out a difficult mental arithmetic task while viewing five colored forms. After a variable delay period, they indicated whether a randomly selected test item had a new color or shape aspect. If the test item appeared old in both aspects, then the subject indicated whether or not color and shape were conjoined as at inspection (Experiment 1). Correct mental arithmetic and recognition responses qualified the subject for monetary rewards. In Experiment 2, subjects were attentionally primed to the color or shape dimension and were asked to indicate if the test item was old or new along that dimension. The nonattended dimension acted either as a recognition facilitator or as a distractor. The stimulus features were categorically different within dimensions, and naming was inhibited by resource competition from the arithmetic task. Under these conditions, complete independence of the visually mediated memory for form and color was observed. In Experiment 1, memory for color-shape conjunctions was totally absent, and in Experiment 2, neither facilitation nor inhibition was observed as the nonattended dimension was varied.

The present study reports (to the knowledge of the investigator) the first careful factor analysis of the Social Interest Index (SII), a measure of Adler's concept of social interest. The test item responses of 416 students from heterogeneous college classes were intercorrelated and factor analyzed. Four factors emerged which accounted for 37% of the total variance. Results are discussed in terms of the congruence between the obtained and hypothesized factors. Recommendations are made for future factorial validity research with the SII, as well as for ways to improve the structure of the scale.

The substance of the case which has been presented is not that any of the procedures that are currently used to develop tests should be discarded but that a richer source of data should be used to make judgements about the validity of tests. It has been argued that the production of valid tests requires a subject-oriented approach and that interviews are a valuable source of information for assisting in the development of such tests. Interviews can be used in the initial exploration of the ways in which potential subjects perceive an area of knowledge, in the trialling of test items and even in the final administration of the completed test. One added bonus of such an approach is that it helps to bring together in a complementary fashion research paradigms which are more often seen to be in competition with one another.

Previous research on primed recognition of categorized lists has shown two discrepant patterns of results, for which there has not been a conclusive explanation. Sometimes rejection of negative recognition test items (lures) was inhibited as a consequence of semantic priming, whereas at other times processing of negative test items was facilitated. This article investigates the reasons for that discrepancy by focusing on the differences between the tasks used to effect priming in the various previous studies. The first two experiments in this paper showed that lure processing is facilitated when priming is achieved through another recognition test item, whereas inhibition is obtained if a semantic category judgment task is performed on the priming items. Thus, both patterns can be reproduced under nearly identical circumstances, with the type of prime processing being the only difference. Two additional experiments are reported that served to generalize the inhibition found in the second experiment to other semantic priming tasks. The data show that the type of processing done on the prime determines whether inhibition or facilitation of lure rejection is obtained. Inhibition is obtained when a semantic task is used to prime a recognition judgment, whereas facilitation results from priming with an episodic task. The results are interpreted in the framework of the semantic/episodic distinction.

(1979). A Teaching Method for Constructing Test Items Supported by a Data Base Approach to Test Item Banking. The Journal of Data Education: Vol. 20, No. 1, pp. 17-22.

In two recognition memory experiments, subjects rated how certain they were that a given test item had been shown previously. Subjects either received standard instructions about their rating criteria or were explicitly instructed to maintain criteria defined in the manner of Healy and Jones (1973) or along the lines of signal detection theory. When a priori probability varied, subjects in all instruction groups held constant the criteria defined by signal detection theory at the expense of shifting the criteria proposed by Healy and Jones. The results were taken as evidence for the psychological validity of recognition memory analyses based on signal detection theory.

In various situations, a multiple-choice test item may have more than one correct alternative, and the goal is to determine how many correct alternatives an examinee actually knows. For a randomly sampled ex aminee, the validity of an item is defined as the prob ability of deciding that the examinee knows i correct alternatives, when in fact exactly i correct alternatives are known. This article describes how latent class models can be used to estimate this probability.

Test bias is conceptualized as differential validity. Statistical techniques for detecting biased items work by identifying items that may be measuring different things for different groups; they identify deviant or anomalous items in the context of other items. The conceptual basis and technical soundness were reviewed for the following item bias methods: transformed item difficulties, item discriminations, one- and three-parameter item characteristic curve methods, and chi-square methods. Sixteen bias indices representing these approaches were computed for black-white and Chicano-white comparisons on both the verbal and nonverbal Lorge-Thorndike Intelligence Tests. In addition, bias indices were recomputed for the Lorge-Thorndike tests using an external criterion. Convergent validity among bias methods was examined in correlation matrices, by factor analysis of the method correlations, and by ratios of agreements in the items found to be “most biased” by each method. Although evidence of convergent validity was found, there will still be important practical differences in the items identified as biased by different methods. The signed full chi-square procedure may be an acceptable substitute for the theoretically preferred but more costly three-parameter signed indices. The external criterion results also reflect on the validity of the methods; arguments were advanced, however, as to why internal bias methods should not be thought of as proxies for a predictive validity model of unbiasedness.

The use of multiple versions of objective exams which vary in test question sequences has raised concern that such a practice may unconsciously bias students' examination scores. This issue has not been explored with respect to the principles of marketing course, where the use of such a testing practice is often necessitated by crowded sections. The current experiment attempts to broaden such knowledge. It replicates previous research that explored the effect of question sequencing on exam performance of marketing students as it related to memory factors. Unlike the earlier study, question sequencing is found to have no significant effect on examination performance.

In the present study, we contrasted models in whichfamiliarity orstrength is the sole basis for recognition judgments (e.g., Gillund & Shiffrin, 1984) with models incorporating retrieval of specific information in a recall-like process (e.g., Humphreys, 1976, 1978; Humphreys & Bain, 1984). We also examined the possibility that an item’s “strength” is determined in part by the match between the verbal encoding contexts at study and at test. In two experiments, study items were presented in triplets. In Experiment 1 recognition was tested with all possible combinations of one-, two-, or three-item targets and distractors; in addition, three different decision criteria were employed in different lists. Experiment 2 included cued recognition among other test conditions. Recognition performance did not increase as more cues were added to the probe, a finding that is inconsistent with Humphreys’s model and other models of that type. Both studies were well fit by a version of the Search of Associative Memory model for recognition (Gillund & Shiffrin, 1984).

When people recognize a test item as belonging or not belonging to a previously presented set, recognition latency increases with the number of items in the set. Although some evidence suggests otherwise, it is currently held that the rate of this increase is the same for children and adults. In contrast, the present experiments indicated a much slower search rate for second graders (mean age = 7.33 years) than for seventh (mean age = 12.50 years) and twelfth graders (mean age = 17.25 years). Moreover, search rate for second graders was invariant under instructions which did or did not emphasize speed and with presentation of the memory set in either the auditory or visual modality. Experiment II showed that the slow search rate for children was not due to differences in encoding between children and adults.

In two studies adults were given brief experiences designed to create a set to interpret class inclusion-type questions in an exclusive or inclusive manner. In the first study the effect of set experiences was measured on ambiguous class inclusion questions; in the second the test item was the typical Piagetian class inclusion problem. In both studies there was evidence that exclusive response set conditions were effective. A third study demonstrated that pretest experience alerting children to inclusive relations enhanced class inclusion performance. The evidence suggests that a tendency to interpret class inclusion questions exclusively leads to class inclusion errors, while a tendency to interpret such questions inclusively fosters success. Two theoretical approaches were discussed, one in accordance with the Piagetian tradition, the alternative maintaining that the linguistic underpinning to class inclusion represents a separate competence that developmentally is distinct from the logic of Pi...

Three rhesus monkeys were trained and tested in a same/different task with six successive sets of 70 item pairs to an 88% accuracy on each set. Their poor initial transfer performance (55% correct) with novel stimuli improved dramatically to 85% correct following daily item changes in the training stimuli. They acquired a serial-probe-recognition (SPR) task with variable (1-6) item list lengths. This SPR acquisition, although gradual, was more rapid for the monkeys than for pigeons similarly trained. Testing with a fixed list length of four items at different delays between the last list item and the probe test item revealed changes in the serial-position function: a recency effect (last items remembered well) for 0-s delay, recency and primacy effects (first and last list items remembered well) for 1-, 2-, and 10-s delays, and only a primacy effect for the longest 30-s delay. These results are compared with similar ones from pigeons and are discussed in relation to theories of memory processing.

Temporal order recognition memory has been examined previously with tasks involving a recency judgment between a pair of items in a preceding string. Recency judgments are impaired when the earlier item is repeated. The present study employed the comparative recency judgment paradigm, with the lists composed of words. The effect of the inclusion in the list of a high associate of the earlier test item was examined and compared to the effect of repetition. Associative interference was observed, but not in all conditions. Direction of association was a significant factor. The results were interpreted in terms of a model of word recognition proposed by Morton.

Storage and retrieval properties of pictures and words were studied within a recognition memory paradigm. Storage was manipulated by instructing subjects either to image or to verbalize to both picture and word stimuli during the study sequence. Retrieval was manipulated by representing a proportion of the old picture and word items in their opposite form during the recognition test (i.e., some old pictures were tested with their corresponding words and vice versa). Recognition performance for pictures was identical under the two instructional conditions, whereas recognition performance for words was markedly superior under the imagery instruction condition. It was suggested that subjects may engage in dual coding of simple pictures naturally, regardless of instructions, whereas dual coding of words may occur only under imagery instructions. The form of the test item had no effect on recognition performance for either type of stimulus and under either instructional condition. However, change of form of the test item markedly reduced item-by-item correlations between the two instructional conditions. It is tentatively proposed that retrieval is required in recognition, but that the effect of a form change is simply to make the retrieval process less consistent, not less efficient.

measure student knowledge as well when presented orally as when presented in mimeographed form.' The results have generally shown that the oral method is as good as the mimeographed method. Student reaction to the oral method, however, is often unfavorable, with the result that class morale or teacher-pupil harmony sometimes suffers if the oral method is used extensively. Students are likely to complain that they do not have a fair chance when the teacher reads the questions to the class and that, if the tests were presented to them on mimeographed sheets, the results would be much more indicative of the knowledge actually possessed. This student preference for the mimeographed method is rather pronounced at the time the first test is presented orally and decreases in amount as more experience is gained in using both methods, but some students are firmly and unalterably opposed to the oral method and persist in classifying themselves as martyrs when they are so tested. The purpose of this article is to report results of an experimental study of the question of student preference and to show the extent to which preference is supported or justified by actual performance. The oral method, as used in this study, consisted first in having each student write on a piece of paper a column of numbers-i, 2, 3, etc.-for the test items to be presented. Then the instructor read each test item aloud to the class twice, and only twice. If a statement was judged by the student to be true, he placed a plus sign

The Kraus-Weber test of minimum muscular fitness was administered to 38 trainable mentally retarded (TMR) children. The results revealed that: The TMR group was seriously deficient in muscular fitness when compared with four groups of normal children; the TMR group was not necessarily retarded in flexibility when compared with four groups of normal children; the TMR girls tended to be more muscularly fit and more flexible than the boys; the lower back test item caused the greatest percentage of failure for the TMR group; 63.3 percent of the 30 failures were multiple; there was no difference between the TMR boys' and girls' multiple failures.

Human ability to encode simultaneously different dimensions of a visual display was tested, using an “erasure” technique to control the time for which the visual information remained available for processing. Three separate vocabularies of test item were employed, one varying in terms of color and two in terms of form attributes. Simultaneous presentation of color and form stimuli gave evidence of nearly perfect parallel encoding of both types of attribute. Form-form combinations, on the other hand, indicated only partially simultaneous encoding of the primary form dimensions involved. It was suggested that, while primary encoding of different stimulus dimensions is simultaneous, within the same dimension encoding of discrete stimulus elements may occur seriatim.

It is argued that where a person is uncertain as to the true answer in a multiple choice question, he should be encouraged to express his partial knowledge in terms of the subjective probability he attaches to each alternative being correct. A variety of answering techniques are examined, together with ways of scoring them, to determine how far they provide an adequate appraisal of the subjective probabilities. Ways of choosing an appropriate method and of improving it are considered and it is suggested that, by suitable training, subjects may be provided with a better understanding of the proper basis for making sound decisions.

The most popularly used methods of item analysis assess the direct relationship of the responses to each test item with a criterion or total test score. Several methods of item selection have been proposed for obtaining maximum test validity utilizing multiple regression procedures. While multiple regression would theoretically yield items for maximum test validity, the calculation time and effort is prohibitive for selecting the number of items generally required. A variation of the multiple regression procedure which reduces the time required for calculation was proposed by Horst (1939), and modified by Gleser and DuBois (1951). The modified approach is the ratio of item-discrimination to item-total relationship by the point-biserial method, (ric/rit). This method will be referred to as the ID-IT ratio. Richardson and Adkins (1938) proposed a method of item selection based upon the concept of the beta weight. Their method assigns item selection weights as a ratio of the item beta coefficient to the test score beta coefficient divided by the standard deviation of the item, i.e., (ric-ritrtc)/(rtc ricrit) oi.

In the near future a new evaluation instrument will appear on the edu cational markets. Conceived by Dr. George Angel? of Michigan State College and perfected by Dr. Angell in collaboration with Dr. Maurice Troyer of Syracuse University its remarkable potentialities are based upon the principle that benefits derived from desirable evaluation procedures are enhanced by immediate knowledge of the results of this evaluation. This instrument, used as an examination answer sheet, resembles in appearance and operation the traditional ' 'cigar store punchboard". It is so constructed that responses to multiple choice type questions are made by punching out one of ?ve avail able spaces provided for each item. If the subject has selected the correct response the punched out space appears as a red dot due to a marked under lying key sheet. If an incorrect response has been selected a white dot appears on the underlying sheet. Subjects are instructed to continue respond ing to each examination item until they have answered it successfully as indicated by the appearance of the red dot. Thus the student knows the correct answer to each test item immediately and in effect grades his own paper as he completes the examination. The final grade is tabulated accord ing to the number of punches expended in arriving at correct answers.

The D 48 test was given to 100 6th and 7th grade students in Turkey. The results showed that: (a) the test had high predictive validity in Turkey, with school achievement over three years used as the criterion; (b) Turkish sigmas were similar to those obtained from similar age groups in other countries whereas Turkish means were lower, due probably to a lack of test sophistication; (c) test item difficulties were rather constant for different age groups and for testing in different countries. The advantages of the D 48 were discussed, namely, its high saturation with "g, " its easy administration and scoring, its non-verbal nature and its cross-cultural applicability due to the fact that by using widely known stimuli, it attempts to equalize experience.

The question of subject-controlled operations for entry and search of memory is examined in models that assume serial memory search. When words from a single taxonomic category are presented, search is serial and exhaustive. With two-category lists, the search remains serial but is not exhaustive. Instead, search terminates after the category of the test item has been completely searched. Two self-terminating search models for categorized lists, a directed entry and a random entry model, are considered. Slope and variance parameters of the latency data favor the random entry strategy.

The Medians per Item (M/I) frequency distribution technique suggests a new perspective in test item evaluation, dealing with an individual at a time and yielding a graphic distribution to show how effectively each item predicted the central score of each S. It may be used to detect heterogeneous items (to improve test efficiency) or to detect excessively representative items (to use in lieu of the rest or as a format for new items). Only significant M/I results are interpretable; for more complete item analysis the traditional correlation methods are recommended as a supplement.

Several item-types which may be used to test for the acquisition of subjectmatter concepts and principles are described. Each item-type is discussed in terms of its ability to tap more than reproductive capabilities. Research hypotheses deducible from the item-types are presented and discussed. Two higher level competencies involved in most course objectives are "knowing concepts" and "understanding principles." The purpose of this paper is to suggest procedures for assessing the acquisition of concepts and principles. Different assessment procedures vary in terms of their adequacy for measuring knowledge or understanding. The degree to which a test item allows us to infer that a student knows a concept or understands a principle depends primarily upon how familiar the student is with the material he has been asked to operate on. When a student is required to construct content which has been presented in class or in a textbook, the resulting construction is one which could have been achieved by what is usually described as rote learning. A test item in which correct performance could very likely be achieved through rote learning is not the type of item we would choose as the basis for inferring that a student "understands" the content of what he has studied. In this paper a distinction is drawn between assessment which relies on reproducing or identifying familiar substance and assessment which relies on producing or identifying novel substances. The point is made that only in the latter case is there adequate evidence for inferring knowledge or understanding.

The Perceptual Maze Test, originally devised by Elithorn and later further developed in collaboration with the author, is currently being used to study human perceptual problem-solving skills. It has been found that test performance is relatively independent of previously acquired skills; that it is impaired by certain forms of organic brain damage; and that it is related to certain mathematical properties of the test item structure. Hypotheses as to the nature of the solving processes employed by subjects led to the graph-theoretical matrix analysis of the item structure presented in this paper.

A flexilevel test is found to be inferior to a peaked conventional test for measuring examinees in the middle of the ability range, superior for examinees at the extremes. Throughout the entire range of ability, a flexilevel test is much superior to any conventional test that attempts to provi,le accurate measurement at both extremes. A THEORETICAL STUDY OF THE MEASUMMENT EFFECTIVENESS OF FLEXILEVEL TESTS* A conventional test becomes a flexilevel test when modified so that the examinee follows these rules: 1. Answer first a specified test item of median difficulty. 2. After answering an item correctly, attempt next the easiest unanswered itew of more-thanmedian difficulty. After answering an item incorrectly, attempt next the hardest unanswered item of less-than-median difficulty. A special answer sheet is used so that the examinee will know whether each answer is correct or incorrect. If the conventional test contains N items, the examinee taking the flexilevel test will attempt only n = (N + 1)/2 of these. A method for implementing flexilevel testing is described by

A technique has been developed which permits the weighting of responses of test items on the I. B. M. scoring machine on the initial scoring, heretofore impossible. This is done by making the length of the response lines on the answer sheet longer or shorter as weights are needed. It is anticipated that this method will prove useful wherever differential weighting serves to increase the validity of tests.

It is shown that in Sternberg’s item recognition task Ss need not make a judgment of the absolute size or color of the test item before comparing it with memory. However, Ss do use size or color information, when possible, to reduce long reaction times for large memory loads. The results suggest that Ss are able to scan memory for form in parallel with testing for gross stimulus features, like size or color. This finding has important implications for sequential two-stage theories of attention.

Hypnotic “depthyy is a complex theoretical concept, for which no direct measures exist at present, but which is reflected in various experiential and behavioral responses to hypnosis. Thirty-five Ss were asked to scale their depth of hypnosis on a 10-point scale after each suggestibility test item on the Stanford Hypnotic Susceptibility Scale, Form C (Weitzenhoffer & Hilgard, 1962). These self-reports were highly correlated with measures of hypnotic behavior and experience. Instructions to report depth immediately and without thinking produced reports which correlated somewhat better with the other measures than instructions to consciously make a best estimate. This self-report scale promises to be highly useful in studies of hypnosis.

For several years Miami University has been endeavoring to help freshmen who are preparing for teaching to overcome deficiencies in background essentials. This article reports the procedures utilized and the results attained in a remedial project in punctuation. The remedial work was administered on an individual basis. The time devoted to the project varied from three to fifteen hours according to the needs of the students concerned. On the average, the time did not exceed seven hours per student. During the first week of the school-year 1929-30, the Guiler-Henry Preliminary Diagnostic Test in Punctuation was given to 349 freshmen, who were enrolled in the School of Education. This test was selected because of its diagnosticsurvey character. By means of the test it was possible both to discover the students who were weak in punctuation and to identify their individual difficulties. The test covers twenty-six uses of punctuation marks, each of which is measured two or more times. Each test item has a value of one point, and the highest possible score is 120 points. The particular uses of punctuation marks included in the test are given in Table II. The scores of the 349 freshmen tested in the preliminary survey are shown in Table I. TABLE I SCORzs or 349 CouLLGe FRSHMEN ON THE PaRrIMNARY PUNcTUATIN TEST

Some years ago deception by college and high school students dis rupted completely the construction of a Group Rational Learning Test. It was necessary to abandon the test construction until a technique was devised for the elimination of deception errors. Students had to be seated compactly for test purposes, and some would invariably peek at neighbors' answers. Individualizing the administration of the test, through unique patterns for each subject, solved that problem. 2It amounted, figuratively, to placing each subject in an individual booth by surrounding him with students hav ing unknown answer patterns. The control and experimental groups set up proved valuable in shed ding light on the relative number of identical errors for control and experi mental groups. This experimenting with patterns, likewise, revealed the necessity for a particular identical errors percentage range for each sep arate test item type. Here, Identical Errors, refers to the percentage obtained by dividing the number of like errors by one's total errors. Table I was derived by first determining the actual number of differ ent ways two students may answer different items. Take a dual-choice item for example, two subjects (1 & 2) may answer in any of the following ways: (Tx F2), (Tt T2), (Fx T2) and (Fx F2). Here a two-choice item has four possible response combinations. Two of these (T j T2) and (Fx F2) are identical responses. One is the answer to the test item, the other is an identical error for each student.

THIS IS A REPORT of the f i r st of a series of studies designed, in part, to ascertain the feasi bility of administering via television group stan dardized tests which are commonly used in the public schools. Since an increasing amount of in struction is executed through the medium of tele vision, it is relevant to inquire whether testing might also be done by television in such a way as to obtain, on the one hand, the same results for students as would be obtained had they been tested under normal conditions, and, on the other hand, to obtain more valid results than are afforded by the usual method of administration. Also, it is relevant to explore the p o s s i bilities afforded by television with respect to gaining test data that simply cannot be obtained by the traditional pa per-and-pencil tests due to their inherent limita tions. This report deals only with a description of attempts to gain by television administration test results which are comparable to those obtain ed under normal administrative conditions. An examination of television as a device by which to communicate test materials revealed a number of problems. The most notable of these problems is the precise method by which the test might be presented within the limiting conditions imposed by the nature of television. When a test is administered under normal conditions, it is usual to inform the examinee of the total time al lowed and then to present him with all the test i tems. He is free to determine the amount of time he will spend on each item, to s e 1 e c t i terns he will attempt, and to re-attempt the items which he had previously omitted. But the size of the television screen precludes presenting all items simultaneously to the student, thus some of the freedoms permitted in the normal administration of tests will be denied him when the test is pre sented by television. Consequently, a substan tially different test administration technique must be used for television testing. The technique which is reported here is called "pacing." It involves the serial presentation of one test item or a small group of items at a time until all items in the test have been exhausted? It does not allow the examinee to refer back to items which have previously been exposed. A single presentation of an item or group of items is re ferred to as an exposure. The duration of the ex posure is called the intra-exposure interval. The time elapsing between the end of one exposure and the beginning of the next exposure is called the in ter-exposure interval. The serial exposure of small portions of tests, with the exposures pos sibly being separated by noticeable intervals of time, makes possible presentation of test mater ials by television. However, it is radically dif ferent from normal procedure and it p o s e s new variables in the administrative technique which must be studied.

Two experiments involving 224 college undergraduates investigated retroactive inhibition (RI) with paired associates learned under the A-B, A-C, the A-B, A-Br, or the A-B, C-D paradigm. First-list retention was assessed using one of three types of unpaced multiplechoice test. When each test item included the specific competing response from the second list, there was a substantial amount of RI; whereas no RI appeared when test items entailed only first-list responses or noncompeting second-list responses. The results were interpreted as evidence for specific response competition.

This study contrasts the effects of specific practice vs nonspecific practice in scanning for sets of five-letter targets. Ss were required to search for new targets after extended practice with one specific target set. Simultaneous memory search was not supported, since searching for one item remained faster than for several items. Serial exhaustive memory search was not supported since the search data were nonlinear. There is evidence that the categorization of a target, even after large amounts of specific practice, is not heavily dependent upon its physical features. A strategy change was detected during practice which allowed more efficient search. It is concluded that a single memory search process can account for the data under both high and low levels of specific practice.

WHEN AN ITEM has been constructed with considerable care, and has then been passed-on by three or four specialists in the field, one should reasonably expect that it would be a good item. Nevertheless, such items all too frequently will not stand up under statistical analysis, and when they do not, it is sometimes very difficult to find a reason why they don't. A study of the item itself and of the responses chosen for it may occasionally give a clue to such difficulties as vocabulary or ofcscure wording of responses, but even if one does decide subjectively that a specific fault exists in the item, he can never be certain that that particular fault is the real root of the difficulty. An experiment carried out by four members of the Board of Examiners at Michigan State College permitted a different approach to this problem. For the experiment, a number of students who had taken comprehensive examinations in the various Basic College subjects at the end of the previous term were asked to come in and retake a few items from these same examinations orally. Five to ten items were chosen from each of the examinations, and the student was asked toread these items and answer them orally, doing all his thinking aloud. His oral responses were recorded on wire. After the student had recorded his free responses to the items, the examiner who interviewed him ques tioned him further to try to draw out the real bases for his choice of re sponses. These were also recorded. More than one hundred such in terviews were made, each of them ranging from thirty to forty-five min utes in length. Eighteen interviews were held with students who had taken the Literature and Fine Arts comprehensive. Of these eighteen students interviewed, three had written A examinations; six had written B; five C; one D; and three F. The five items chosen as the basis for these interviews were con structed as indicated in the first paragraph. The items referred to a photograph of the head of the figure in Rembrandt's Woman Paring Her Nails (Metropolitan Museum, NYC. ). In their course work, the stu dents had seen and had had analyzed for them a number of reproductions of Rembrandt's works, but it is doubtful if any of them had seen a re production of this particular detail. On the same page with it was a photograph of an El Greco portrait of a man, which offered a consider able contrast of style. In this particular group of items it was antici