Search

Digital transformations within educational systems are recurrently justified by their promise to enhance learning activities and outcomes. We examine this claim in a study of pupils' perception and experiences with computer-based adaptive tests in higher classes in Danish public primary schools. The study relies on survey and interview data and addresses whether pupils perceive their participation in computer-based adaptive tests as contributing to their learning and the degree to which the perception of teachers' attitude to the test influences their learning. A majority perceives their learning as low or non-existing. However, the perception of a more positive teacher attitude is related to more substantial learning. Theoretically, we attribute this relationship to the effect of self-fulfilling prophecies in the classroom. Learning also differs for subgroups related to gender and ability level. The findings contribute to discussions about educational policy, management, and the digital transformation of educational systems. [ABSTRACT FROM AUTHOR]

Background: The Patient-Reported Outcomes Measurement Information System® (PROMIS) Profile Computer Adaptive Testing (CAT) consists of seven CATs and one single item measuring most relevant aspects of health-related quality of life (HRQoL). The aim of our study was to determine construct validity and floor and ceiling effects of the PROMIS Profile CAT in Dutch people with stroke. Methods: People with stroke receiving rehabilitation completed the PROMIS Profile CAT and the EuroQol-5 dimensions (EQ5D). Construct validity was evaluated with hypotheses testing based on expected correlations between the profile domains and the domains of the EQ5D. The proportion of participants with the lowest and highest scores were calculated for each profile domain to assess floor and ceiling effects. Results: 160 participants were included (median age 61 years, 41.9% female). For the PROMIS Profile domains Physical Function, Anxiety, Depression, Sleep Disturbance, Pain Interference, and Pain Intensity > 75% of the results met our hypotheses. For Fatigue and Ability to Participate in Social Roles and Activities only 60% of hypotheses were met. No floor or ceiling effects were found, with the exception of a floor effect for Pain Intensity which probably indicates that many participants had no pain. Conclusion: Most domains of the PROMIS Profile CAT showed sufficient construct validity and no problematic floor or ceiling effects in people with stroke. These CATs and the single item Pain Intensity can be used to efficiently measure HRQoL in people with stroke. [ABSTRACT FROM AUTHOR]

This article examines whether the relationship between the members of the United Nations' Treaty Bodies' (TB) home countries and the countries reviewed by these bodies is significant to the amount and style of criticism the reviewed country receives. This article explains why the source–target relationship is significant to shaming. It defines two shaming styles – Socializing shaming and Disciplining shaming and defines elements that enable the identification of shaming styles. Utilizing data on CAT shaming from 2001 to 2022 and on TB members' country of origin it shows through regression analysis that the existence of a trade partnership between TB members' home countries and reviewed countries results in more criticism. This article qualitatively identifies elements of Socializing versus Disciplining shaming in the communications with countries that are trade partners of the TB members' home countries. It finds that Socializing shaming elements are more prevalent. This article draws theoretical, empirical and policy conclusions. [ABSTRACT FROM AUTHOR]

Computerized adaptive tests (CATs) are highly efficient assessment tools that couple low patient and clinician time burden with high diagnostic accuracy. A CAT for substance use disorders (CAT-SUD-E) has been validated in adult populations but has yet to be tested in adolescents. The purpose of this study was to perform initial evaluation of the K-CAT-SUD-E (i.e., Kiddy-CAT-SUD-E) in an adolescent sample compared to a gold-standard diagnostic interview. Adolescents (N = 156; aged 11–17) with diverse substance use histories completed the K-CAT-SUD-E electronically and the substance related disorders portion of a clinician-conducted diagnostic interview (K-SADS) via tele-videoconferencing platform. The K-CAT-SUD-E assessed both current and lifetime overall SUD and substance-specific diagnoses for nine substance classes. Using the K-CAT-SUD-E continuous severity score and diagnoses to predict the presence of any K-SADS SUD diagnosis, the classification accuracy ranged from excellent for current SUD (AUC = 0.89, 95% CI = 0.81, 0.95) to outstanding (AUC = 0.93, 95% CI = 0.82, 0.97) for lifetime SUD. Regarding current substance-specific diagnoses, the classification accuracy was excellent for alcohol (AUC = 0.82), cannabis (AUC = 0.83) and nicotine/tobacco (AUC = 0.90). For lifetime substance-specific diagnoses, the classification accuracy ranged from excellent (e.g., opioids, AUC = 0.84) to outstanding (e.g., stimulants, AUC = 0.96). K-CAT-SUD-E median completion time was 4 min 22 s compared to 45 min for the K-SADS. This study provides initial support for the K-CAT-SUD-E as a feasible accurate diagnostic tool for assessing SUDs in adolescents. Future studies should further validate the K-CAT-SUD-E in a larger sample of adolescents and examine its acceptability, feasibility, and scalability in youth-serving settings. [ABSTRACT FROM AUTHOR]

This evaluation study explored a two-year literacy reform effort in a school district with 10 of its 13 elementary schools in the improvement process. Schools were placed in three groups. Group 1 (four schools, 80 teachers) implemented new instructional practices in Year 1 and then added a new curriculum. Group 2 (four schools, 102 teachers) changed both the pedagogy and curriculum for both years. Group 3 (five schools, 107 teachers) did not make any changes in Year 1 and then implemented both components. After propensity score matching students, reading scores were analyzed with three-level change score models. Results suggested that two years of the combined reforms generally were effective at improving students' oral reading fluency, and two years of the pedagogical changes were effective at improving students' performance on a computer-adaptive test of overall reading ability. However, improvements on the computer-adaptive test as well as the state reading test were small, which might reflect the stability in reading comprehension found in other studies. [ABSTRACT FROM AUTHOR]

This study evaluated the forecasting accuracy of trend estimation methods applied to time-series data from computer adaptive tests (CATs). Data were collected roughly once a month over the course of a school year. We evaluated the forecasting accuracy of two regression-based growth estimation methods (ordinary least squares and Theil-Sen). The precision, or accuracy, of predictions were heavily influenced by how far performance was forecasted into the future (1, 3, or 5 months) and the number of observations available to estimate growth (3, 4, or 5). When performance was forecasted further and growth estimates were based upon fewer observations, predictions were off by as much as two times the average conditional standard error of measurement for a grade level. No combinations of data collections schedule, forecasting length, or estimation method led to consistent bias in predictions. Educators should be cautious when using trend-line decision rules to predict future performance. Suggestions for alternate decision rules to explore with CATs are offered. [ABSTRACT FROM AUTHOR]

When student achievement is assessed, we seek to elicit a student's maximum performance – a goal requiring the assumption that the student is fully engaged. Otherwise, to the extent that disengagement occurs, test performance is likely to suffer. Effectively managing test-taking disengagement requires an understanding of the testing conditions under which disengagement is more likely. This study investigated the association between disengagement and the time-of-day testing occurred for a widely used low-stakes, computer-adaptive achievement test in grades 2–8. Disengagement was measured using two indicators: response time effort (RTE) and performance decline. Results showed clear evidence for a time-of-day effect for RTE, with the prevalence of students exhibiting low RTE (<.90) tripling during the school day in Math, and nearly doubling in Reading. In contrast, the prevalence of significant performance decline showed little change across time of day. Reasons for differences in results for the two disengagement indicators are discussed. [ABSTRACT FROM AUTHOR]

We investigate two research questions using a recent statewide transition from paper to computer-based testing: first, the extent to which test mode effects found in prior studies can be eliminated; and second, the degree to which online and paper assessments offer different information about underlying student ability. We first find very small mode effects for a more recent transition in Massachusetts. Second, we investigate the predictive evidence of validity for paper and online tests for predictions of future test scores and grades. We generally find minimal differences for the extent to which scores on paper tests can differentially predict future online versus paper test scores. Finally, online and paper test scores are similarly predictive of future grade point average. We conclude that the online test penalty can vary substantially by test and that extreme care should be taken when administering online tests to some students and paper tests to others. [ABSTRACT FROM AUTHOR]

Large-scale educational assessments are sometimes considered low-stakes, increasing the possibility of confounding true performance level with low motivation. These concerns are amplified in remote testing conditions. To remove the effects of low effort levels in responses observed in remote low-stakes testing, several motivation filtering methods can be used to purify the data. We estimated scores from assessment data collected remotely in Spring 2021 six ways, applying examinee-based filtering methods (filtering examinees based on total time) and response-based filtering methods (filtering responses using the effort-moderated IRT model), varying the thresholds selected to separate effortful and non-effortful responses. We compared the 2021 scores (estimated six ways) to those obtained in previous cohorts tested in person. The results support the use of motivation filtering regardless of which method is used. Practical implications regarding method and threshold selection are discussed. [ABSTRACT FROM AUTHOR]

Primary purpose was to identify relationships between performance-based measures onto both computerized adaptive testing [Physical Function-Computer Adaptive Testing (PF-CAT)] and joint-specific legacy [Knee injury and Osteoarthritis Outcome Score-Activities of Daily Living (KOOS-ADL)] instruments pre- and 12-month post-TKA. The PF-CAT and KOOS-ADL were identified as outcomes and performance on the 40-m fast-paced walking test, stair climb test and chair stand test were identified as predictors. Linear regression was used for all comparisons after adjusting for confounders. Sixty-eight people with TKA (56.7% male) were tested. Better scores on the 40-m fast-paced walking (KOOS-ADL, p = 0.02), stair climb (KOOS-ADL, p = 0.05) and chair stand (KOOS-ADL, p < 0.01) associated with better self-reported scores pre-TKA. Better scores on the 40-m fast-paced walking (PF-CAT, p = 0.05; KOOS-ADL, p = 0.01), stair climb (KOOS-ADL, p < 0.01), chair stand (PF-CAT, p < 0.01) and range of motion (KOOS-ADL, p = 0.02) were associated with better self-reported scores 12-month post-TKA. Decrease knee range of motion related to poorer 40-m fast-paced walking (p = 0.01) and stair climb (p = 0.03) scores pre-TKA. Quadriceps weakness related to poorer 40-m fast-paced walking (p = 0.04) score pre-TKA. Self-reported instruments are a moderate, but inconsistent surrogate to performance-based measures pre- and post-TKA. Our findings indicate that both self-reported and performance-based measures are necessary to fully characterize physical function and should be used jointly to aid in the recovery analysis of people undergoing TKA. Monitoring the trajectory of pre- to post-total knee arthroplasty (TKA) physical function is important as it directly relates to mortality, morbidity and poorer quality of life in older adults. Both self-reported and performance-based measures of physical function are used to determine progress in recovery for patients pre- and post- TKA. This study provides evidence that perceived physical function measures are a moderate, but an inconsistent, surrogate to objective physical function measures pre- and post-TKA. Joint specific deficits in knee range of motion and quadriceps strength were weakly associated with deficits in function measures pre-TKA, but no association was observed 12-month post-TKA. Our findings indicate that both self-reported and performance-based measures are necessary to fully characterize physical function and should be used jointly to aid in the complete recovery analysis of people undergoing TKA. [ABSTRACT FROM AUTHOR]

Incoming students have many difficulties adjusting to college, and selecting appropriate measures to effectively screen them is indispensable, especially in China, where there is insufficient research in this area. To enrich domestic research, this study seeks to examine psychometric characteristics and develop a computerized adaptive version of the Student Adaptation to College Questionnaire (SACQ-CAT) based on a sample of Chinese students. Under the framework of item response theory, the item bank of student adaptation to college was formulated after uni-dimensionality testing, model comparison, item fit testing, and local independence testing. Subsequently, a CAT simulation, including three termination rules, was performed using real data to evaluate and verify the SACQ-CAT. The results showed reliability values exceeding 0.90 when participants' latent traits were between −4 and 3, covering majority of the subjects. The SACQ-CAT administered an average of fewer than 10 items to participants compared to 67 items on the original scale. The correlation coefficient between latency estimated by the SACQ-CAT and the SACQ is greater than.85, whereas the correlation coefficient with the Symptom Checklist 90 (SCL-90) scores ranges from −.33 to −.55 (p <.001). The SACQ-CAT largely reduced the number of items administered to the participants and avoided losing measurement precision. [ABSTRACT FROM AUTHOR]

In recent years, Computerized Multistage Testing (MST), with their versatile benefits, have found themselves a wide application in large scale assessments and have increased their popularity. The fact that forms can be made ready before the exam application, such as a linear test, and that they can be adapted according to the test taker's ability level, such as computerized adaptive tests, has brought MST to the forefront. It is observed that simulation studies are often used in research on MST. The R programming language used in the conduct of simulation studies is widely used for statistical calculation, data visualization, and Monte Carlo research. Researchers can perform their analysis according to their own research questions, both by writing their own code in R and by using the packages in the library. This study aims to demonstrate the design and implementation of MST simulation examples using the R programming language. In this context, first of all, the basic components of MST were discussed, then R packages written on MST were examined in terms of advantages, disadvantages and analysis facility. Then, three different MST simulation examples were designed with the R programming language. It is considered that this study will be useful to those who are interested in MST. [ABSTRACT FROM AUTHOR]

Screening items for parameter drift helps protect against serious validity threats and ensure score comparability when equating forms. Although many high-stakes credentialing examinations operate with small sample sizes, few studies have investigated methods to detect drift in small sample equating. This study demonstrates that several newly researched drift detection strategies can improve equating accuracy under certain conditions with small samples where some anchor items display item parameter drift. Results showed that the recently proposed methods mINFIT and mOUTFIT as well as the more conventional Robust-z helped mitigate the adverse effects of drifting anchor items in conditions with higher drift levels or with more than 75 examinees. In contrast, the Logit Difference approach excessively removed invariant anchor items. The discussion provides recommendations on how practitioners working with small samples can use the results to make more informed decisions regarding item parameter drift. [ABSTRACT FROM AUTHOR]

Computerized adaptive testing (CAT) offers an efficient and highly accurate method for estimating examinees' abilities. In this article, the free version of Concerto Software for CAT was reviewed, dividing our evaluation into three sections: software implementation, the Item Response Theory (IRT) features of CAT, and user experience. Overall, Concerto is an excellent tool for researchers seeking to create computerized adaptive (or non-adaptive) tests, providing a robust platform with comprehensive IRT capabilities and a user-friendly interface. [ABSTRACT FROM AUTHOR]

Word retraining techniques can improve picture naming of treated items in people with semantic dementia (SD). The utility of this, however, has been questioned given the propensity for under- and overgeneralization errors in naming in SD. Few studies have investigated the occurrence of such errors. This study examined whether, following tailored word retraining: (1) misuse of words increases, (2) the type of naming errors changes, and/or (3) clarity of communication is reduced. Performance on trained and untrained word naming from nine participants with SD who completed a word retraining programme were analysed. Responses from baseline and post-intervention assessments were coded for misuse (i.e., trained word produced for another target item), error type, and communication clarity. All participants showed significant improvement for trained vocabulary. There was no significant increase in misuse of words, with such errors occurring rarely. At a group level, there was an increased tendency toward omission errors for untrained items, and a reduction in semantically related responses. However, this did not impact on clarity scores with no consistent change across participants. In sum, we found no negative impacts following tailored word retraining, providing further evidence of the benefit of these programmes for individuals with SD. [ABSTRACT FROM AUTHOR]

Early identification of language delay is important as it has a serious impact on a child's life in terms of educational, social, and emotional development. Among the early language screening tools, there are some parent-administered tools; however, they are not culturally appropriate or freely available. This article documents the development and preliminary validation of a quick and easy-to-administer language screening tool for babies from 6 to 18 months of age. Parents of 100 babies ranging in age from 6 to 21 months were included in the study. The babies were classified into five screening levels according to their age. The items of a Screening Test of Early Language Development-Test version (STELD-T) were created and validated through expert opinion. The STELD-T was administered along with the Receptive Expressive Emergent Language Scale (REELS). Internal consistency using the Kuder-Richardson Formula-20 ranged from 0.457 to 0.853 across the five levels, acceptable owing to short tool length and item heterogeneity. Kappa coefficients indicated 0.459 to 0.875 agreement between the STELD-T and the REELS indicated satisfactory criterion validity. After calculating the percentage of babies with a "refer" result as well as Kappa statistics with three different pass-refer criteria, a pass-refer criterion of 75% seemed to be appropriate for screening. The STELD seems to be a reliable and valid tool to screen language development in babies from 6 months to 18 months of age in urban areas of Maharashtra. Items representing a range of language skills including pragmatics make it a unique tool. [ABSTRACT FROM AUTHOR]

Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly account for guessing, knowledge, and blunder using eight assessments (>9,000 responses) from an undergraduate biotechnology curriculum. A Bayesian implementation of the models, aimed at assessing their robustness to prior beliefs in examinee knowledge, showed that explicit estimators of knowledge are markedly sensitive to prior beliefs with scores as sole input. To overcome this limitation, we examined self-ranked confidence as a proxy knowledge indicator. For our test set, three levels of confidence resolved test performance. Responses rated as least confident were correct more frequently than expected from random selection, reflecting partial knowledge, but were balanced by blunder among the most confident responses. By translating evidence-based guessing and blunder rates to pass marks that statistically qualify a desired level of examinee knowledge, our approach finds practical utility in test analysis and design. [ABSTRACT FROM AUTHOR]

This study is an exploratory analysis of examinee behavior in a large-scale language proficiency test. Despite a number-right scoring system with no penalty for guessing, we found that 16% of examinees omitted at least one answer and that women were more likely than men to omit answers. Item-response theory analyses treating the omitted responses as missing rather than wrong showed that examinees had underperformed by skipping the answers, with a greater underperformance among more able participants. An analysis of omitted answer patterns showed that reading passage items were most likely to be omitted, and that native language-translation items were least likely to be omitted. We hypothesized that since reading passage items were most tempting to skip, then among examinees who did answer every question there might be a tendency to guess at these items. Using cluster analyses, we found that underperformance on the reading items was more likely than underperformance on the non-reading passage items. In large-scale operational tests, examinees must know the optimal strategy for taking the test. Test developers must also understand how examinee behavior might impact the validity of score interpretations. [ABSTRACT FROM AUTHOR]

Scores from computer-based tests are increasingly used to inform a variety of school- and student-level decisions. An underlying assumption is that the associated scores represent effortful responding by each student with respect to the tasks presented. An innovative method for examining evidence for this assumption involves an examination of item response times to detect rapid guessing behavior. If such behavior is present for a particular student across many items, the score is considered invalid due to low test-taking effort (TTE). Drawing on Eccles' expectancy–value theory, this study explored the prevalence of low TTE among middle school students during a computer adaptive test in reading, as well as demographic and motivational correlates of low TTE. Results suggested a prevalence rate of low TTE of 7%. Low TTE was particularly common among males and those with high cost value perceptions. Implications for research and practice are discussed. Impact Statement Seven percent of middle school students were identified as engaging in rapid guessing behavior on a computer-based reading test, to the extent that the validity of their test scores should be questioned. This behavior was particularly common among males and those who perceived the test as requiring them to give up preferred activities. Further exploration into methods for increasing test-taking effort among those who display rapid guessing behavior is needed. [ABSTRACT FROM AUTHOR]

Individuals can be classified into different chronotypes concerning their circadian rhythm features. Morningness-Eveningness (MEQ) can be defined as individual differences in sleep-wake patterns and people's time-of-day feel and perform the best. Various self-report instruments that measure the Morningness-Eveningness have been developed. The Morningness–Eveningness Questionnaire (MEQ), developed by Horne and Östberg, was the first and most used questionnaire for determining the chronotypes of an individual. The main aim of the current study was to translate and validate the Morningness-Eveningness Questionnaire (MEQ) to the Nepali language. The English version of the MEQ was translated to the Nepali language using the translation-back-translation method. Further, it was given to 10 native Nepali speakers for content validation. The final MEQ-Nepali version was then administered to 116 native Nepali speakers. Mean, standard deviation, variance, and reliability statistics such as Cronbach's alpha score, intraclass correlation coefficient, and item-total correlation were obtained for the data. Results showed that the MEQ-Nepali version has a Cronbach's alpha score and an inter-class correlation coefficient of 0.73, which is considered good reliability. The MEQ Nepali version developed in this study is found to be valid and reliable. Hence, it can be used to categorize Nepali-speaking populations into various circadian types. [ABSTRACT FROM AUTHOR]

To mitigate the deleterious effects of rapid guessing (RG) on ability estimates, several rescoring procedures have been proposed. Underlying many of these procedures is the assumption that RG is accurately identified. At present, there have been minimal investigations examining the utility of rescoring approaches when RG is misclassified, and individual scores are reported. To address this limitation, the present simulation study investigates the effect of RG misclassifications on individual examinee ability estimate bias and classification accuracy when using effort-moderated (EM) scoring. This objective is accomplished by manipulating simulee ability level, RG rate, as well as misclassification type and percentage. Results showed that EM scoring significantly improved ability inferences for examinees engaging in RG; however, the effectiveness of this approach was largely dependent on misclassification type. Specifically, across ability levels, bias tended to be on average lower when falsely classifying effortful responses as RG. Although EM scoring improved bias, it was susceptible to elevated false-positive classifications of ability under high RG. [ABSTRACT FROM AUTHOR]

Base rates of failure (BRFail) on performance validity tests (PVTs) were examined in university students with limited English proficiency (LEP). BRFail was calculated for several free-standing and embedded PVTs. All free-standing PVTs and certain embedded indicators were robust to LEP. However, LEP was associated with unacceptably high BRFail (20–50%) on several embedded PVTs with high levels of verbal mediation (even multivariate models of PVT could not contain BRFail). In conclusion, failing free-standing/dedicated PVTs cannot be attributed to LEP. However, the elevated BRFail on several embedded PVTs in university students suggest an unacceptably high overall risk of false positives associated with LEP. [ABSTRACT FROM AUTHOR]

Introduction: The quality of an Emergency Medical Technician (EMT) or paramedic training program is likely one factor that contributes to a candidate's success on the National Registry Cognitive Examination. However, program pass rates and their associated geographic location have not previously been evaluated. Our objective was to evaluate the performance of EMT and paramedic programs in the United States, pass rates on the National Registry Cognitive Examinations, and relationship to geography. Methods: We conducted a cross-sectional evaluation of EMT and paramedic programs' first and cumulative third attempt pass rates on the National Registry Cognitive Examination in 2015. Included were civilian EMT and paramedic programs with >5 graduates testing for certification. Descriptive statistics were calculated for program pass rates, total graduates testing, and pass rates by National Association of State EMS Officials (NASEMSO) regions. Results: We included 1,939 EMT programs (6–1,892 graduates testing/program) and 602 paramedic programs (6–689 graduates testing/program). We excluded 262 (11.9%) EMT and 51 (7.8%) paramedic programs with ≤5 graduates testing annually due to unstable estimates of program pass rates. EMT programs in the highest quartile for total number of graduates testing outperformed the lowest quartile in both first attempt (65.7% vs. 61.9%, p < 0.001) and cumulative third attempt pass rates (79.1% vs 72.7%, p < 0.001). This difference was also seen for paramedic programs on first attempt (77.3% vs. 62.5%, p < 0.001) and cumulative third attempt (91.9% vs. 76.9%, p < 0.001). EMT program pass rates for first and cumulative third attempts also varied by NASEMSO region (first: 62–68%; third: 74–78%) with the Great Lakes and West regions outperforming the other regions. Paramedic program pass rates differed by NASEMSO regions as well (first: 65–83%; third: 81–95%) with highest pass rates in the West region. Conclusions Program performance for both EMT and paramedic programs varies by total number of graduates testing and geographic location. Graduates from larger EMT and paramedic programs have higher first and cumulative third attempt pass rates compared to graduates from smaller programs. Additionally, there is variability in program pass rates across NASEMSO regions for both certification levels. Further evaluation is necessary to better understand the variability in program performance in the United States. [ABSTRACT FROM AUTHOR]

This article discusses the use of multidimensional computerized adaptive testing (MCAT) as a framework for ecological momentary assessment (EMA) in psychological research. The authors conducted two simulated studies to evaluate item selection rules (ISRs) and stopping criteria (SC) in highly multidimensional MCATs. The results showed that MCAT methods outperformed single-item approaches and yielded precise estimates similar to full item batteries. Both D-optimal and A-optimal ISRs were effective in recovering the latent trait, and the resulting test lengths were suitable for EMA deployment. Further research is needed to evaluate MCAT performance in rapidly fluctuating situations. [Extracted from the article]

Testing programs are confronted with the decision of whether to report individual scores for examinees that have engaged in rapid guessing (RG). As noted by the Standards for Educational and Psychological Testing, this decision should be based on a documented criterion that determines score exclusion. To this end, a number of heuristic criteria (e.g., exclude all examinees with RG rates of 10%) have been adopted in the literature. Given that these criteria lack strong methodological support, the objective of this simulation study was to evaluate their appropriateness in terms of individual ability estimate and classification accuracy when manipulating both assessment and RG characteristics. The findings provide evidence that employing a common criterion for all examinees may be an ineffective strategy because a given RG percentage may have differing degrees of biasing effects based on test difficulty, examinee ability, and RG pattern. These results suggest that practitioners may benefit from establishing context-specific exclusion criteria that consider test purpose, score use, and targeted examinee trait levels. [ABSTRACT FROM AUTHOR]

Semantic distance is increasingly used for automated scoring of originality on divergent thinking tasks, such as the Alternate Uses Task (AUT). Despite some psychometric support for semantic distance – including positive correlations with human creativity ratings – additional work is needed to optimize its reliability and validity, including identifying maximally reliable items (objects) for AUT administration. We identify a set of 13 AUT items based on a systematic item-selection strategy (belt, brick, broom, bucket, candle, clock, comb, knife, lamp, pencil, pillow, purse, sock). This item-set resulted in acceptable reliability estimates and was found to be moderately related to both human creativity ratings and a creative personality factor (Study 1). These results replicated in a new sample of Participants (Study 2). We conclude with the following recommendations for reliable and valid assessment of AUT originality using semantic distance: 1) make choices based on theoretical/practical considerations, 2) administer (some or all of) the 13 items from this study; 3) if other items must be used, avoid compound words as AUT items (e.g., guitar string); 4) include as many AUT items as time permits; 5) instruct participants to "be creative"; and 6) address fluency confounds that conflate idea quantity and quality (e.g., via max scoring). [ABSTRACT FROM AUTHOR]

Health literacy among patients is crucial for effective stroke management. The European Health Literacy Survey Questionnaire is a theory-based measure that comprehensively captures 12 domains of health literacy. We aimed to develop a computerized adaptive test of the European Health Literacy Survey Questionnaire to efficiently assess health literacy among patients with stroke. The European Health Literacy Survey Questionnaire data of 311 patients and item parameters were retrieved from a Rasch validation study. Real data simulations were performed to develop a computerized adaptive test of the European Health Literacy Survey Questionnaire and explore its efficiency and reliability. The computerized adaptive test of the European Health Literacy Survey Questionnaire displayed suitable reliability in all 12 domains (0.72–0.84) with a mean test length of 17 items (36.2% of the 47-item European Health Literacy Survey Questionnaire). Our findings indicate that the computerized adaptive test of the European Health Literacy Survey Questionnaire, which assesses 12 domains of health literacy among patients with stroke in a timely and precise fashion, is efficient and reliable. The computerized adaptive test of the European Health Literacy Survey Questionnaire assesses the subjective fit of personal health literacy competencies to environmental demands, providing insight into patient strengths and weaknesses when dealing with health tasks and interacting with health systems. The computerized adaptive test of the European Health Literacy Survey Questionnaire assesses the 12 domains of health literacy among patients with stroke in a timely and precise manner. Our study has demonstrated the utility of the computerized adaptive test of the European Health Literacy Survey Questionnaire in reducing the assessment burden of patients in clinical practice. Developing of the computerized adaptive test of the European Health Literacy Survey Questionnaire provided findings that may benefit researchers and clinicians interested in developing efficient outcome measures. [ABSTRACT FROM AUTHOR]

The hypersexualized song and dance spectacles of Bombay cinema known as item numbers are hybrid ensembles that operate as standalone 'attractions' outside narrative and theatrical cinema to later acquire a life via the internet and live circuits. With digital as the vortex, this multimedia constellation of the item number and its loudness cartographs spaces and renders visible bodies that are increasingly affected by a powerful sensorium. This article maps the auditory afterlife of the item number based on case studies of election campaigns, low-cost media consumption, and club culture. It demonstrates the recalibration of sense-perceptions in specific socio-economic urban formations, as the item number moves through a complex matrix of sleaze, power, and labor. [ABSTRACT FROM AUTHOR]

Copyright of Language Assessment Quarterly is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Although many educational and psychological tests are labeled as computerized adaptive test (CAT), not all tests show the same level of adaptivity – some tests might not have much adaptation because of various constraints imposed by test developers. Researchers have proposed some indices to measure the amount of adaption for an adaptive test. This article shows some limitations of the existing indices. A new index of adaptivity is proposed in this article. Its performance was evaluated in a simulation. The results show that the new index was able to overcome some of the limitations of the existing indices in the simulated scenarios. [ABSTRACT FROM AUTHOR]

A common practical challenge is how to assign ability estimates to all incorrect and all correct response patterns when using item response theory (IRT) models and maximum likelihood estimation (MLE) since ability estimates for these types of responses equal −∞ or +∞. This article uses a simulation study and data from an operational K − 12 computerized adaptive test (CAT) to compare how well several alternatives – including Bayesian maximum a priori (MAP) estimators; various MLE based methods; and assigning constants – work as strategies for computing ability estimates for extreme scores in vertically scaled fixed-length Rasch-based CATs. Results suggested that the MLE-based methods, MAP estimators with prior standard deviations of 4 and above, and assigning constants achieved the desired outcomes of producing finite ability estimates for all correct and all incorrect responses that were more extreme than the MLE values of students that got one item correct or one item incorrect as well as being more extreme than the difficulty of the items students saw during the CAT. Additional analyses showed that it is possible for some methods to exhibit changes in how much they differ in magnitude and variability from the MLE comparison values or the b values of the CAT items for all correct versus all incorrect responses and across grades. Specific discussion is given to how one may select a strategy to assign ability estimates to extreme scores in vertically scaled fixed-length CATs that employ the Rasch model. [ABSTRACT FROM AUTHOR]

We report on an innovative computer-adaptive assessment, the Comprehensive Research-based Early Math Ability Test (CREMAT), using the case of 1st- and 2nd-graders' understanding of geometric measurement. CREMAT was developed with multiple aims in mind, including: (1) be administered with a reasonable number of items, (2) identify the level(s) of thinking at which the child is operating for each developmental progression (domain) of significant mathematics, and (3) provide information about specific concepts and skills that constitute the cognitive components of thinking involved in various tasks. CREMAT relies on Q-Matrix theory and a resulting blend of sophisticated psychometric and statistical techniques (Rule Space Method, poset models) to determine not only students' overall performance but also their specific level of thinking along developmental progressions. Results indicated that the current tests range from satisfactory to excellent in their ability to diagnose 1st- and 2nd-graders' understanding of measurement, but also suggest revisions and implications for the approach. We conclude with a description of why CREMAT is important for early childhood practitioners' ability to formatively assess and document children's progress in learning math. [ABSTRACT FROM AUTHOR]

To determine if preoperative physical and psychological health risk factors are influential on physical function and satisfaction scores in a cohort of relatively younger adults one year post-total knee arthroplasty. Sixty-five patients were recruited pre- and one year post-total knee arthroplasty. Physical (i.e., body mass index, sex, physical activity level, number of comorbidities) and psychological (i.e., depression, expectations, pain inference) health risk factors were obtained pre-total knee arthroplasty and compared to Patient-Reported Outcomes Measurement Information System physical function computerized adaptive testing and satisfaction scores one year post-total knee arthroplasty. Bootstrap inclusion fraction methods were performed to compare the stability of each predictor variable prior to final regression analyzes. Poorer preoperative physical activity level (standardized regression coefficient = 0.37; p < 0.01), pain interference (standardized regression coefficient = –0.24; p = 0.03) and greater comorbidities (standardized regression coefficient = –0.27; p = 0.01) resulted in worse physical function scores one year post-total knee arthroplasty. Greater preoperative comorbidities (odds ratio = 0.63; p < 0.01) resulted in worse satisfaction scores one year post-total knee arthroplasty. No other predictor variable had any relationship on either clinical outcome (p > 0.05). Younger adults with greater comorbidities, lower physical activity level and higher pain interference scores preoperatively reported poorer functional recovery one year post-total knee arthroplasty. Understanding how preoperative status of younger adults undergoing total knee arthroplasty may lead to better improved surgical decision making and health care delivery for this vulnerable and growing patient subgroup. Projections estimate total knee arthroplasty in relatively younger adults (<65 years) will make up the majority of surgical procedures performed in the United States within the next two decades, however little is known in this patient population. This study provides evidence that relatively younger adults with greater comorbidities, lower physical activity level and higher pain interference scores preoperatively result in poorer functional recovery one year post-total knee arthroplasty. The results also suggest greater comorbidities identified preoperatively have the largest influence on one year satisfaction scores in younger adults post-total knee arthroplasty. Modifiable characteristics such as improving physical activity level, incorporating pain management strategies and reducing comorbidities preoperatively or adopting into postoperative rehabilitative care may influence the postoperative physical function and satisfaction scores in relatively younger adults undergoing total knee arthroplasty. [ABSTRACT FROM AUTHOR]

The present paper describes beautiful conservation relations between areas formed by different geometrical shapes and area relations formed by algebraic functions. The conservation properties were investigated by students at an academic college of education using a computerized technological tool and were subsequently accompanied by justified proofs. [ABSTRACT FROM AUTHOR]

Prader-Willi Syndrome (PWS) is a neurodevelopmental genetic disorder with executive deficits. Planning is one of the impaired executive functions implied in the regulation of behavior and everyday actions. We aimed to explore the feasibility and the effectiveness of a metacognitive strategy training designed to improve planning in adults with PWS using a double-blind between-group (training versus usual care) randomized controlled trial, with computerized tests and paper-pencil ecological outcome measures targeting planning, other executive functions, and achievement of personalized goal. Results showed better performances in several executive tasks and in achievement of personalized goals after both interventions, but better improvement for the experimental group (n = 27) compared to control (n = 26) only on the task assessing planning abilities. Interviews with occupational therapists demonstrated the feasibility of this training with this population. Despite a small number of sessions, the metacognitive strategy training showed encouraging results on planning abilities of patients. [ABSTRACT FROM AUTHOR]

The existence of outliers in the anchor items can be detrimental to the estimation of examinee ability and undermine the validity of score interpretation across forms. However, in practice, anchor item performance can become distorted due to various reasons. This study compares the performance of modified INFIT and OUTFIT Rasch statistics with the Logit Difference approach with 0.3 and 0.5 as the predetermined cutoff values, and the Robust z statistic with 1.645 and 2.7 as the cutoff values through a simulation study by varying the sample size, proportion of outliers, item difficulty drift direction, and group difference magnitude. The results suggest that both modified INFIT and OUTFIT statistics perform very similarly and outperform the other methods in all aspects, including sensitivity of flagging outliers, specificity of flagging non-outliers, recovery of translation constant, and recovery of examinee ability in all simulated conditions. [ABSTRACT FROM AUTHOR]

Objective: The Emotion Dysregulation Inventory (EDI) is an informant questionnaire developed based on the Patient-Reported Outcomes Measurement Information System (PROMIS®) Scientific Standards and refined through factor analyses and item response theory (IRT) analyses. Although it was developed to improve measurement of emotion dysregulation in youth with autism spectrum disorder, emotion dysregulation has transdiagnostic significance. Therefore, the aim of this study was to evaluate the EDI's psychometric properties and to establish IRT-based scores for a general population of youth. Methods: Data were collected from a sample of 1000 caregivers of 6- to 17-year-old youth matched to the US census on age, gender, race/ethnicity, years of education, and region. Confirmatory factor analyses and IRT analyses using the two-parameter graded response model were performed to evaluate the EDI's structure and psychometric properties. Results: Analyses supported the original two-factor structure of the EDI, reflecting factors for Reactivity and Dysphoria. Simulations of computerized adaptive testing supported use of the same items for a Reactivity short form as those that emerged as most informative in the original autism psychometric analyses. IRT co-calibration with commonly used measures of emotion regulation and irritability in child clinical or community samples indicated the EDI scales provide more information across a wider range of emotion dysregulation. Validity was supported by moderate correlations with measures of related constructs and expected known-group differences. Conclusions: The EDI is an efficient and precise measure of emotion dysregulation for use in general community and clinical samples as well as samples of youth with ASD. [ABSTRACT FROM AUTHOR]

Recent developments in the dimensional assessment of personality functioning have made the implementation of latent measurement models increasingly attractive. In this study, we applied item response theory (IRT) to a well-established personality functioning instrument (the OPD Structure Questionnaire) to identify a unidimensional latent trait and to evaluate the feasibility of computer adaptive testing (CAT). We hypothesized that the use of IRT could reduce the test burden – compared to a fixed short form – while maintaining high precision over a wide range of the latent trait. The OPD-SQ was collected from 1235 patients in a psychosomatic clinic. IRT assumptions were fulfilled. A 9-factor model yielded sufficient fit and unidimensionality in exploratory factor analysis with bifactor rotation. Items were iteratively reduced, and a graded-response IRT model was fitted to the data. Simulations showed that a CAT with approximately 7 items was able to capture an OPD-SQ global severity score with an accuracy similar to that of a fixed 12-item short form. The final item bank and CAT yielded satisfactory content validity. Strong correlations with depression and anxiety replicated previous results on the OPD-SQ. We concluded that IRT applications could be useful to reduce the test burden of personality functioning instruments. [ABSTRACT FROM AUTHOR]

Computer-adaptive tests use respondents' answers to previous questions to select the subsequent questions. They are gaining popularity for their increased measurement precision and decreased administration time compared to static questionnaires. The purpose of this study was to estimate the test-retest reliability and construct validity of the computer-adaptive test version of a participation measure, the Late Life Disability Instrument (LLDI-CAT) for people with COPD and to compare scores and administration time with those of the static LLDI. Among 76 older adults with COPD, scores on the LLDI-CAT were compared to scores on measures of related constructs, between groups based on symptom severity, prognosis and frailty phenotype, and to scores on the static LLDI. A subsample of 28 people completed the LLDI-CAT a second time within one week of the initial administration for test-retest reliability. The LLDI-CAT had very good test-retest reliability (ICC2,1 0.88; SEM 2.74 points), fair correlations with physical function (r = 0.37-0.50), anxiety (r=-0.42), and depression (r=-0.50), fair to moderately-strong correlations with quality of life (r = 0.48-0.63), and strong correlation with the static LLDI limitation domain (r = 0.80). The LLDI-CAT scores differed between people with different symptom severity, prognosis and frailty phenotype (p ≤ 0.004). The mean administration time for the LLDI-CAT was 3.3 (1.5) minutes, less than that of the static LLDI at 6.3 (2.8) minutes (p < 0.001). The LLDI-CAT demonstrates evidence of test-retest reliability and construct validity, and correlates well with the limitation domain of the static LLDI for people with COPD. The LLDI-CAT can be used to assess participation for this population. [ABSTRACT FROM AUTHOR]

In recent years, the interest in measuring growth in student ability in various subjects between different grades in school has increased. Therefore, good precision in the estimated growth is of importance. This paper aims to compare estimation methods and test designs when it comes to precision and bias of the estimated growth of mean ability between two groups of students that differ substantially. This is performed by a simulation study. One- and two-parameter item response models are assumed and the estimated abilities are vertically scaled using the non-equivalent anchor test design by estimating the abilities in one single run, so-called concurrent calibration. The connection between the test design and the Fisher information is also discussed. The results indicate that the expected a posteriori estimation method is preferred when estimating differences in mean ability between groups. Results also indicate that a test design with common items of medium difficulty leads to better precision, which coincides with previous results from horizontal equating. [ABSTRACT FROM AUTHOR]

In psychological and educational measurement, it is often of interest to assess change in an individual. The current study expanded on previous research by introducing methods that can evaluate individual change on multiple latent traits measured on multiple occasions. The four methods considered are the likelihood ratio test (LRT), the multivariate Wald test (MWT), the modified multivariate Wald test (MMWT), and the score test (ST). Simulation studies were conducted to examine the true positive rate (TPR) and the false positive rate (FPR) of the new methods under a conventional fixed-form test and a computerized adaptive test (CAT). Manipulated variables included the number of occasions, change magnitudes, patterns of change, and correlations between latent traits. Results revealed that, in terms of FPR, all methods except MWT had close adherence to the nominal significance level. Among the three methods, the LRT is recommended as it provided a balance between FPR and TPR. Larger change magnitude yielded higher TPR, regardless of the remaining factors. With the same test length, a CAT yielded higher TPR than a conventional test. Real-data examples are provided of identifying psychometrically significant change across two to four occasions using a multivariate adaptive self-report medical outcomes measure from hospitalized patients. The detection of significant change among the three methods agreed highly, and those patients identified as having significant change exhibited large profile differences, which provided support for the valid performance of the proposed methods. [ABSTRACT FROM AUTHOR]

The 1970s saw major transformations in how computerized databases were conceived, developed, and designed. Part of a broader shift in how software applications were developed, these transformations—sometimes referred to as "the database revolution"—introduced new and then-novel approaches to structuring and arranging digital data, optimizing them for usability and convenience. At the same time, however, the rhetoric of innovation and revolution surrounding this moment in database development obscures the ways it helped concentrate and extend particular kinds of racialized power and, in particular, whiteness (i.e., those norms and values congenial to the reproduction of white racial dominance and the subjugation of blackness). In this article, we revisit key works of the database revolution to show how they encoded whiteness as a kind of unremarked optimum, in both implicit and explicit ways. Finally, we argue that these developments helped to codify and extend a kind of "willful ignorance" that, as scholars of epistemology and justice have shown, is central to the preservation and reproduction of whiteness. [ABSTRACT FROM AUTHOR]

The Personality Assessment Inventory (PAI) is a reliable multidimensional psychometric inventory that is increasingly being used in the medical–legal context. To date, 18 language adaptations of the PAI exist, yet only the Spanish, Greek and German language versions have been examined psychometrically. This study evaluated the psychometric properties of the French-Canadian version of the PAI by comparing mean scale and subscale scores between the French-Canadian and English language versions, and analyzing the internal consistency and mean item inter-correlations (MICs) of each version in a sample of 50 bilingual university students. Cronbach's alphas ranged from −.57 to.80 in the French-Canadian version and from −1.10 to.83 in the English version, with most scales being below.70, indicating inadequate internal consistency. In addition, most of the MICs were below.20, indicating a lack of item homogeneity. Caution is given to this adaptation of the PAI in the medical–legal context. [ABSTRACT FROM AUTHOR]

Perinatal stroke is the most common form of stroke in childhood and is followed by a variety of outcomes, with many children experiencing specific functional and neuropsychological deficits. The association of these outcomes with the psychosocial impact caregivers face is not well documented. The goal of our pilot study was to examine caregivers' perception of executive behavior and functional abilities among children with perinatal stroke, and how these outcomes impact the caregivers. We administered three questionnaires to primary caregivers of children with perinatal stroke to obtain caregiver-reported measures of (1) executive behavior of their child (Behavior Rating Inventory of Executive Function, Second Edition), (2) the functional abilities of their child (Pediatric Evaluation of Disability Inventory Computer Adaptive Test), and (3) the psychosocial impact experienced by the caregiver themselves (Parental Outcome Measure). Participants included 20 children (mean age = 9.3 years, range = 6–16 years) with perinatal stroke and their primary caregivers. Functional abilities in the children were rated as clinically impaired in the domains of daily activities and mobility. Half of the children exhibited clinically impaired ratings on at least one executive behavior domain, but the mean scores for these domains did not reach clinically impaired levels. Greater ratings of problems in daily activities for the child was associated with greater caregiver guilt (r = −0.55, p = 0.02). Caregivers of children with perinatal stroke who experience limitations in performing daily activities should be more closely monitored for adverse impact and be provided the necessary support and education to alleviate the associated guilt. [ABSTRACT FROM AUTHOR]

The property of item parameter invariance in item response theory (IRT) plays a pivotal role in the applications of IRT such as test equating. The scope of parameter invariance when using estimates from finite biased samples in the applications of IRT does not appear to be clearly documented in the IRT literature. This article provides information on the extent to which item parameter invariance is observed in samples with the Rasch and 2-parameter model calibrations through simulations, where the behaviors of item parameter estimates were examined under 12 different types of convenient sampling scenarios. The results indicated that the property of item invariance in IRT for dichotomously scored data could hold for the sample item parameter estimates, regardless of biased samples, when the model holds in the data, the number of items in a test is not small, and the sample size is large. [ABSTRACT FROM AUTHOR]

Asian culture has been test-centric for over 1,400 years. U.S. education has been since the 1990s. Based on Kim's creative Climates, Attitudes, and Thinking skills (CATs), creativity indicators were developed using the 2015 PISA questionnaires. Study I examined (a) relationships between students' PISA science scores and CATs; (b) differences between Asian and U.S. students in PISA scores and CATs; and (c) relationships between teachers' years of teaching and nurturing CATs. Study I found (a) negative relationships between students' PISA scores and CATs, (b) Asian students demonstrating higher PISA scores but lower CATs than U.S. students, and (c) U.S. teachers decreasingly nurture CATs. Study II examined how U.S. CATs changed from 1990 through 2017 by analyzing TTCT norming data (N = 273,441) and found continued declines in CATs, especially in imagination and in young children. [ABSTRACT FROM AUTHOR]

The indirect effect of perceived test importance on test performance via examinee effort is often modeled using importance and effort scores measured after test completion, which does not align with their theoretical temporal ordering. These retrospectively measured scores may be influenced by examinees' test performance. To investigate the impact of timing of measurement, college students were randomly assigned to one of the three conditions: (a) importance and effort were measured retrospectively, (b) importance and effort were measured retrospectively and importance was measured prospectively, and (c) importance and effort were measured retrospectively and prospectively. The unstandardized indirect effect was invariant across conditions when modeling prospective and retrospective scores. Priming examinees via prospectively measuring importance and effort did not affect the interrelations among performance and retrospective importance and effort (i.e., invariant indirect effect). Priming did lead to higher average test performance. Thus, priming may provide a cheap intervention for increasing test performance. [ABSTRACT FROM AUTHOR]

The TOEFL iBT test was introduced in 2005 to better reflect the language demands of real-life academic tasks than did previous versions of the test. The task-based design of the test was intended to support the interpretation of its scores as a trustworthy measure of international students' ability to use English in an academic environment. Until recently, the TOEFL iBT test was exclusively administered online at secure test centers around the world. In response to the disruption caused by the pandemic in early 2020, the TOEFL iBT Home Edition was launched, which is identical to the version administered in test centers, but taken at home through the examinee's computer, in a proctored environment. We present key features of content delivery and the security of the test. We then discuss preliminary findings related to the comparability of scores across modes of delivery, and important implications for the design and score interpretation of at-home language proficiency tests. [ABSTRACT FROM AUTHOR]