Search

A test of item compromise is presented which combines the test takers' responses and response times (RTs) into a statistic defined as the number of correct responses on the item for test takers with RTs flagged as suspicious. The test has null and alternative distributions belonging to the well-known family of compound binomial distributions, is simple to calculate, and has results that are easy to interpret. It also demonstrated nearly perfect power for the detection of compromise with no more than 10 test takers with preknowledge of the more difficult and discriminating items in a set of empirical examples. For the easier and less discriminating items, the presence of some 20 test takers with preknowledge still sufficed. A test based on the reverse statistic of the total time by test takers with responses flagged as suspicious may seem a natural alternative but misses the property of a monotone likelihood ratio necessary to decide between a test that should be left or right sided.

BackgroundThe optimal interval for repeat biopsy during active surveillance (AS) for prostate cancer is yet to be defined. This study examined whether risk of upgrading (to grade group ≥ 2) or risk of converting to treatment varied according to intensity of repeat biopsy using data from the GAP3 consortium's global AS database.Materials and methodsIntensity of surveillance biopsy schedules was categorized according to centers' protocols: (a) Prostate Cancer Research International Active Surveillance project (PRIAS) protocols with biopsies at years 1, 4, and 7 (10 centers; 7532 men); (b) biennial biopsies, that is, every other year (8 centers; 4365 men); and (c) annual biopsy schedules (4 centers; 1602 men). Multivariable Cox regression was used to compare outcomes according to biopsy intensity.ResultsOut of the 13,508 eligible participants, 56% were managed according to PRIAS protocols (biopsies at years 1, 4, and 7), 32% via biennial biopsy, and 12% via annual biopsy. After adjusting for baseline characteristics, risk of converting to treatment was greater for those on annual compared with PRIAS biopsy schedules (hazard ratio [HR] = 1.66; 95% confidence interval [CI] = 1.51-1.83; p

BackgroundThe inclusion criterion for active surveillance (AS) is low- or intermediate-risk prostate cancer. The predictive value of the presence of a suspicious lesion at magnetic resonance imaging (MRI) at the time of inclusion is insufficiently known.ObjectiveTo evaluate the percentage of patients needing active treatment stratified by the presence or absence of a suspicious lesion at baseline MRI.Design setting and participantsA retrospective analysis of the data from the multicentric AS GAP3 Consortium database was conducted. The inclusion criteria were men with grade group (GG) 1 or GG 2 prostate cancer combined with prostate-specific antigen

Two independent statistical tests of item compromise are presented, one based on the test takers' responses and the other on their response times (RTs) on the same items. The tests can be used to monitor an item in real time during online continuous testing but are also applicable as part of post hoc forensic analysis. The two test statistics are simple intuitive quantities as the sum of the responses and RTs observed for the test takers on the item. Common features of the tests are ease of interpretation and computational simplicity. Both tests are uniformly most powerful under the assumption of known ability and speed parameters for the test takers. Examples of power functions for items with realistic parameter values suggest maximum power for 20-30 test takers with item preknowledge for the response-based test and 10-20 test takers for the RT-based test.

The current literature on test equating generally defines it as the process necessary to obtain score comparability between different test forms. The definition is in contrast with Lord's foundational paper which viewed equating as the process required to obtain comparability of measurement scale between forms. The distinction between the notions of scale and score is not trivial. The difference is explained by connecting these notions with standard statistical concepts as probability experiment, sample space, and random variable. The probability experiment underlying equating test forms with random scores immediately gives us the equating transformation as a function mapping the scale of one form into the other and thus supports the point of view taken by Lord. However, both Lord's view and the current literature appear to rely on the idea of an experiment with random examinees which implies a different notion of test scores. It is shown how an explicit choice between the two experiments is not just important for our theoretical understanding of key notions in test equating but also has important practical consequences.

BackgroundCareful assessment of the reasons for discontinuation of active surveillance (AS) is required for men with prostate cancer (PCa).ObjectiveUsing Movember's Global Action Plan Prostate Cancer Active Surveillance initiative (GAP3) database, we report on reasons for AS discontinuation.Design, setting, and participantsWe compared data from 10296 men on AS from 21 centres across 12 countries.Outcome measurements and statistical analysisCumulative incidence methods were used to estimate the cumulative incidence rates of AS discontinuation.Results and limitationsDuring 5-yr follow-up, 27.5% (95% confidence interval [CI]: 26.4-28.6%) men showed signs of disease progression, 12.8% (95% CI: 12.0-13.6%) converted to active treatment without evidence of progression, 1.7% (95% CI: 1.5-2.0%) continued to watchful waiting, and 1.7% (95% CI: 1.4-2.1%) died from other causes. Of the 7049 men who remained on AS, 2339 had follow-up for >5yr, 4561 had follow-up for

In a recently published article in this journal, Becker et al. claim that, because of a missing slope parameter, the lognormal model for response times on test items almost never holds in practice. However, the authors' critique rests on a misrepresentation of the model, which already does have the equivalent of a slope parameter. More importantly, their extra parameter spoils the interpretation of the parameters for the test‐takers' speed and labor intensity of the items necessary for a response‐time model to be empirically meaningful while their proposed interpretation of the extra parameter seems unwarranted. An analysis of the authors' earlier empirical comparison between the original and their alternative version of the model does not seem to support much of a conclusion about the relative fit of the two models. Also, their simulation study conducted to demonstrate the necessity of the extra slope parameter appears to be based on data simulated in favor of their parameter. [ABSTRACT FROM AUTHOR]

The Bayesian way of accounting for the effects of error in the ability and item parameters in adaptive testing is through the joint posterior distribution of all parameters. An optimized Markov chain Monte Carlo algorithm for adaptive testing is presented, which samples this distribution in real time to score the examinee's ability and optimally select the items. Thanks to extremely rapid convergence of the Markov chain and simple posterior calculations, the algorithm is ready for use in real-world adaptive testing with running times fully comparable with algorithms that fix all parameters at point estimates during testing.

One of the methods of controlling test security in adaptive testing is imposing random item-ineligibility constraints on the selection of the items with probabilities automatically updated to maintain a predetermined upper bound on the exposure rates. Three major improvements of the method are presented. First, a few modifications to improve the initialization of the method and accelerate the impact of its feedback mechanism on the observed item-exposure rates are introduced. Second, the case of conditional item-exposure control given the uncertainty of examinee's ability parameter is addressed. Third, although rare for a well-designed item pool, when applied in combination with the shadow-test approach to adaptive testing the method may meet occasional infeasibility of the shadow-test model. A big "M" method is proposed that resolves the issue. The practical advantages of the improvements are illustrated using simulated adaptive testing from a real-world item pool under a variety of conditions.

Lord's (1980) equity theorem claims observed-score equating to be possible only when two test forms are perfectly reliable or strictly parallel. An analysis of its proof reveals use of an incorrect statistical assumption. The assumption does not invalidate the theorem itself though, which can be shown to follow directly from the discrete nature of the equating problem it addresses. But, surprisingly, an obvious relaxation of the problem is enough to obtain exactly the opposite result: As long as two test forms measure the same ability, they can always be equated, no matter their reliability, degree of parallelness, or even difference in length. Also, in spite of its lack of validity, the original proof of Lord's theorem has an important interim result directly applicable to the problem of assembling a new test form pre-equated to an old form.

Parameter linking in item response theory is generally necessary to adjust for differences between the true values for the same item and ability parameters due to the use of different identifiability restrictions in different calibrations. The research reported in this article explores a precision-weighted (PW) approach to the problem of estimating the linking functions for the common dichotomous logistic response models. Asymptotic standard errors (ASEs) of linking for the new approach are derived and compared to those of the mean/mean and mean/sigma linking methods to which it has a superficial similarity and to the Haebara and Stocking and Lord response function methods. Empirical examples from a few recent linking studies are presented. It is demonstrated that the new approach has smaller ASE than the mean/mean and mean/sigma methods and comparable ASE to the response function methods. However, when some of the item parameters have large estimation error relative to the others, all current methods appear to violate the rather obvious requirement of monotone decrease in ASE with the number of common items in the linking design while the ASE of the PW method demonstrates monotone decrease with the number of common items. The PW method also has the benefits of simple calculation and an ASE which is additive in the contribution of each item, useful for optimal linking design. We conclude that the proposed approach to estimating linking parameters holds promise and warrants further research.

The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use of mixed-integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test-form generation problems with a large variety of formatting constraints, and how it compares with MIP-based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixed-form; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP-based applications.

Linking functions adjust for differences between identifiability restrictions used in different instances of the estimation of item response model parameters. These adjustments are necessary when results from those instances are to be compared. As linking functions are derived from estimated item response model parameters, parameter estimation error automatically propagates into linking error. This article explores an optimal linking design approach in which mixed-integer programming is used to select linking items to minimize linking error. Results indicate that the method holds promise for selection of linking items.

We present a novel application of a generalized item response tree model to investigate test takers' answer change behavior. The model allows us to simultaneously model the observed patterns of the initial and final responses after an answer change as a function of a set of latent traits and item parameters. The proposed application is illustrated with large-scale mathematics test items. We also describe how the estimated results can be used to study the benefits of answer change and to further detect potential academic cheating.

A test of item compromise is presented which combines the test takers' responses and response times (RTs) into a statistic defined as the number of correct responses on the item for test takers with RTs flagged as suspicious. The test has null and alternative distributions belonging to the well‐known family of compound binomial distributions, is simple to calculate, and has results that are easy to interpret. It also demonstrated nearly perfect power for the detection of compromise with no more than 10 test takers with preknowledge of the more difficult and discriminating items in a set of empirical examples. For the easier and less discriminating items, the presence of some 20 test takers with preknowledge still sufficed. A test based on the reverse statistic of the total time by test takers with responses flagged as suspicious may seem a natural alternative but misses the property of a monotone likelihood ratio necessary to decide between a test that should be left or right sided. [ABSTRACT FROM AUTHOR]