Search

In this research, multistage adaptive tests (MST) were compared according to sample size, panel pattern and module length for top-down and bottom-up test assembly methods. Within the scope of the research, data from PISA 2015 were used and simulation studies were conducted according to the parameters estimated from these data. Analysis results for each condition were compared in terms of mean RMSE and bias. According to the results obtained from the MST simulation based on the top-down test assembly method, mean RMSE values reduced when the module length increased and when the panel pattern changed from 1-2 to 1-2-2 and 1-2-3 for MST applied to small and large samples. Within the scope of the research, data from PISA 2015 were used and simulation studies were conducted using the parameters estimated from these data. Analysis results for each condition were compared in terms of mean RMSE and bias.

The Centre for Evaluation and Monitoring (CEM), based in the North of England, recently celebrated its 40th birthday. Arising from an evaluation project at Newcastle University, and a subsequent move to Durham University, it rapidly grew in scope and influence, developing a series of highly regarded school assessments. For a relatively small organisation, its influence was seen across the world, resulting in outreach centres in New Zealand, Australia, and Hong Kong. Since being acquired by Cambridge University Press & Assessment in 2019, it has established itself in a unique role within the wider Cambridge organisation due mainly to its development of computer adaptive assessments for use in schools. This article documents the rise of CEM, from its early successes to its adoption of new ideas in educational assessment and supporting technology until the present day. However, CEM's development was not without its controversies, and these too make fascinating reading when set against the background of 40 years of ever changing educational policies.

Local item dependence (LID) refers to the situation where responses to items in a test or questionnaire are influenced by responses to other items in the test. This could be due to shared prompts, item content similarity, and deficiencies in item construction. LID due to a shared prompt is highly probable in cloze tests where items are nested within a passage. The purpose of this research is to examine the occurrence and magnitude of LID in a cloze test. A cloze test was analyzed with the Rasch model and locally dependent items were identified with the residual correlations. Findings showed that three pairs of items were locally dependent. When these items were removed from the analysis, test reliability dropped but item fit and unidimensionality improved. Removing the three locally dependent items did not affect person ability mean and standard deviation, though. The findings are discussed in terms of LID detection and modeling in the context of cloze test and language testing.

This research paper investigated the importance of conducting measurement invariance analysis in developing measurement tools for assessing differences between and among study variables. Most of the studies, which tended to develop an inventory to assess the existence of an attitude, behavior, belief, IQ, or an intuition in a person's characterological profile, ignored testing measurement invariance for equivalency between comparable variables. With this finding, measurers lack in true validity and reliability and suffer methodological bias and have little or no chance to figure out the true differences between variables being studied. This article, therefore, explains the necessity and use of measurement invariance analysis when a researcher wanted to develop a new measurement tool or adapt a tool from one source language to another target language. The types of measurement invariance levels mentioned in this study are configural-invariance, scalar invariance, metric invariance and structural invariance analysis. The approaches used to conduct those invariance models and the way they have been interpreted were all discussed in great detail with a robust collection of supportive literature.

The COVID-19 pandemic has had a significant impact on high-stakes testing, including the national benchmark tests in South Africa. Current linear testing formats have been criticized for their limitations, leading to a shift towards Computerized Adaptive Testing [CAT]. Assessments with CAT are more precise and take less time. Evaluation of CAT programs requires simulation studies. To assess the feasibility of implementing CAT in NBTs, SimulCAT, a simulation tool, was utilized. The SimulCAT simulation involved creating 10,000 examinees with a normal distribution characterized by a mean of 0 and a standard deviation of 1. A pool of 500 test items was employed, and specific parameters were established for the item selection algorithm, CAT administration rules, item exposure control, and termination criteria. The termination criteria required a standard error of less than 0.35 to ensure accurate abilities estimation. The findings from the simulation study demonstrated that fixed-length tests provided higher testing precision without any systematic error, as indicated by measurement statistics like CBIAS, CMAE, and CRMSE. However, fixed-length tests exhibited a higher item exposure rate, which could be mitigated by selecting items with fewer dependencies on specific item parameters (a-parameters). On the other hand, variable-length tests demonstrated increased redundancy. Based on these results, CAT is recommended as an alternative approach for conducting NBTs due to its capability to accurately measure individual abilities and reduce the testing duration. For high-stakes assessments like the NBTs, fixed-length tests are preferred as they offer superior testing precision while minimizing item exposure rates.

A test battery with two different levels of adaptation is presented: a within-subtest level for the selection of the items in the subtests and a between-subtest level to move from one subtest to the next. The battery runs on a two-level model consisting of a regular response model for each of the subtests extended with a second level for the joint distribution of their abilities. The presentation of the model is followed by an optimized MCMC algorithm to update the posterior distribution of each of its ability parameters, select the items to Bayesian optimality, and adaptively move from one subtest to the next. Thanks to extremely rapid convergence of the Markov chain and simple posterior calculations, the algorithm can be used in real-world applications without any noticeable latency. Finally, an empirical study with a battery of short diagnostic subtests is shown to yield score accuracies close to traditional one-level adaptive testing with subtests of double lengths.

To solve the attenuation paradox in computerized adaptive testing (CAT), this study proposes an item selection method, the integer programming approach based on real-time test data (IPRD), to improve test efficiency. The IPRD method turns information regarding the ability distribution of the population from real-time test data into feasible test constraints to reversely assembled shadow tests for item selection to prevent the attenuation paradox by integer programming. A simulation study was conducted to thoroughly investigate IPRD performance. The results indicate that the IPRD method can efficiently improve CAT performance in terms of the precision of trait estimation and satisfaction of all required test constraints, especially for conditions with stringent exposure control.

This paper evaluates the multistage adaptive test (MST) design of a large-scale academic language assessment (ACCESS) for Grades 1-12, with an aim to simplify the current MST design, using both operational and simulated test data. Study 1 explored the operational population data (1,456,287 test-takers) of the listening and reading tests of MST ACCESS in the 2018-2019 school year to evaluate the MST design in terms of measurement efficiency and precision. Study 2 is a simulation study conducted to find an optimal MST design with manipulation on the number of items per stage and panel structure. The results from operational test data showed that the test length for both the listening and reading tests could be shortened to six folders (i.e., 18 items), with final ability estimates and reliability coefficients comparable to those of the current test, with slight differences. The simulation study showed that all six proposed MST designs yielded slightly better measurement accuracy and efficiency than the current design, among which the 1-3-3 MST design with more items at earlier stages ranked first. The findings of this study provide implications for the evaluation of MST designs and ways to optimize MST designs in language assessment.

Adaptive models of word production hold that lexical processing is shaped by recent production episodes. In particular, the models proposed by Howard et al. (2006) and Oppenheim et al. (2010) assume that the connection strength between semantic and lexical representations is updated continuously, on each use of a word. These changes make semantically related competitor words to a selected target word (e.g., "cherry" or "grape" for the target word "apple") less accessible for future retrieval. We put this assumption to a novel test in two picture-word interference experiments. The experiments looked at semantic interference (longer picture naming latencies in the presence of a semantically related distractor word compared to an unrelated distractor word; the effect is assumed to reflect the activation of competitor words to a target word) and its development across repeated naming. Across two times of measurement, semantically related distractor words (and their unrelated controls) were either identical (fixed target-distractor mapping, e.g., either distractor "cherry" or distractor "grape" for the target word "apple" at both times of measurement) or different (variable target-distractor mapping, e.g., distractor "cherry" at first time of measurement and distractor "grape" at second time of measurement or vice versa). Semantic interference was reduced at the second time of measurement, and this reduction was found with both fixed and variable target-distractor mapping. This generalization to new target-distractor combinations provides strong novel support for the notion of production-dependent changes in lexical accessibility as captured by adaptive models of word production.

In an online learning environment, it is important to establish a suitable assessment approach that can be adapted on the fly to accommodate the varying learning paces of students. At the same time, it is essential that assessment criteria remain compliant with the expected learning outcomes of the relevant education standard which predominantly utilizes a competency-based curriculum such as in Indonesia. The aim of the research in this paper is to improve the adaptiveness of questions in the existing Computerized Adaptive Testing (CAT) model by taking into consideration multiple aspects of user context. We propose a context-based question selection model based on competency evaluation by merging four methods of Classical Test Theory, Rasch Model, Linear and Quadratic models, and the combination of branching and item adaptive methods to select questions of suitable difficulty for each individual student. To evaluate the proposed model, we conducted experiments based on a real dataset of 689 elementary school students in Indonesia. The experiment results prove the effectiveness of the proposed model in terms of accuracy in predicting the appropriateness of the questions in relation to the students' ability. This adaptive assessment method which accurately builds upon the students' competency level will support students' success in the online learning environment.

Between 2018-2021, eight European medical schools took part in a study to develop a medical knowledge Online Adaptive International Progress Test. Here we discuss participants' self-perception to evaluate the acceptability of adaptive vs non-adaptive testing. Study participants, students from across Europe at all stages of undergraduate medical education with varying levels of prior experience with progress testing, sat remotely invigilated tests using the online QuizOne® platform. Participants completed online feedback questionnaires on their experiences and perceptions of adaptive and non-adaptive tests. Overall satisfaction with the organisation and delivery of remote online tests was high regardless of previous experience with progress testing, differences in stages, programmes, and to some degree language. In statements probing the appropriateness of the level and the length of testing, differences were observed between adaptive and non-adaptive tests. There was a high level of agreement that the adaptive test was a good measure of personal knowledge and increased participants' motivation for study. Students' self-perception of the assessment is an important factor in evaluation of acceptability of the exam and its further development. In our study, the adaptive test algorithm adjusted the level of difficulty for the individual student in real-time, leading to positive perceptions of the length of the test and promoting students' engagement. The assessment increases student motivation for learning and in turn, has the potential to improve their performance.

Computational Thinking (CT) skill was a part of the global framework of reference on Digital Literacy for Indicator 4.4.2, widely developed in mathematics and science learning. This study aimed to promote an assessment tool using a two-tier Computerized Adaptive Test (CAT). The study used the Design and Development Research (DDR) method with four stages of data collection: expert judgment, usability test, simulation study, and student feedback. The two-tier questions have been tested empirically and fit with the Generalized Partial Credit Model (GPCM) or 2 Parameter Logistic (2PL). This study proved that the CAT could be used to estimate students' CT with a minimum different Standard Error of Measurement (different SEM) or close to zero ([approximately]0.001). According to the expert judgment, CAT was proper for measurement with a feasibility of 90.23%. The usability test proved that CAT was feasible to use, with satisfaction of 79.51%. Furthermore, the simulation study or empirical measurement explained that students' CT at the high school level were greater than at the middle and lower school levels. Thus, CAT were empirically proven to estimate students' CT accurately. Finally, students' feedback showed that the acceptance from students' feedback is 74.40% with a very satisfactory category.

This study has two main purposes. Firstly, to compare the different item selection methods and stopping rules used in Computerized Adaptive Testing (CAT) applications with simulative data generated based on the item parameters of the Vocational Maturity Scale. Secondly, to test the validity of CAT application scores. For the first purpose, simulative data produced based on Vocational Maturity Scale item parameters were analyzed under different item selection methods (Maximum Fisher Information [MFI], Maximum Likelihood Weighted Information [MLWI] Maximum Posterior Weighted Information [MPWI] Maximum Expected Information [MEI] Minimum Expected Posterior Variance [MEPV] Maximum Expected Posterior Weighted Information [MEPWI]) and stopping rules (Standard Error [SE]<0.30, SE<0.50, SE <0.70, Number of Item [NI]=10, NI=20) by calculating the average number of items, standard error averages, correlation coefficients, bias, and RMSE statistics. For all the conditions of the item selection methods, standard error averages, correlation coefficients, bias, and RMSE statistics showed similar results. When the average number of items is considered, MFI and SE<0.30 were found as most appropriate methods to be used in CAT application. For the second purpose of the study, the paper-pencil form of the Vocational Maturity scale and CAT version were administered to 33 students. A moderate, positive, and statistically significant relationship was found between the CAT application scores and the paper-pencil form scores on the vocational maturity scale. As a result, it can be said that the vocational maturity scale can be applied as a computerized adaptive test and can be used in career guidance processes.

Many examinations with thousands of participating students are organized worldwide every year. Usually, this large number of students sit the exams simultaneously and answer almost the same set of questions. This method of learning assessment requires tremendous effort and resources to prepare the venues, print question books and organize the whole process. Additional restrictions and obstacles may appear in conditions similar to those during the COVID-19 pandemic. One way to obviate the necessity of having all the students take an exam during the same period of time is to use a computer-assisted assessment with random item selection, so that every student receives an individual set of questions. The objective of this study is to investigate students' perceptions of using random item selection from item banks in order to apply this method in large-scale assessments. An analysis of the responses of more than 1000 surveyed students revealed that most of them agree or completely agree with using the proposed method of assessment. The students from natural science departments showed more tolerance of this method of assessment compared with students from other groups. Based on the findings of this study, the authors concluded that higher-education institutions could benefit from implementing the above mentioned assessment method.

The purpose of this study is to generate non-verbal items for a visual reasoning test using templated-based automatic item generation (AIG). The fundamental research method involved following the three stages of template-based AIG. An item from the 2016 4th-grade entrance exam of the Science and Art Center (known as BILSEM) was chosen as the parent item. A cognitive model and an item model were developed for non-verbal reasoning. Then, the items were generated using computer algorithms. For the first item model, 112 items were generated, and for the second item model, 1728 items were produced. The items were evaluated based on subject matter experts (SMEs). The SMEs indicated that the items met the criteria of one right answer, single content and behavior, not trivial content, and homogeneous choices. Additionally, SMEs' opinions determined that the items have varying item difficulty. The results obtained demonstrate the feasibility of AIG for creating an extensive item repository consisting of non-verbal visual reasoning items.

Testing programs have explored the use of technology-enhanced items alongside traditional item types (e.g., multiple-choice and constructed-response items) as measurement evidence of latent constructs modeled with item response theory (IRT). In this report, we discuss considerations in applying IRT models to a particular type of adaptive testlet referred to as a branching item. Under the branching format, all test takers are assigned to a common question, and the assignment of the next question relies on the response to the first question through deterministic rules. In addition, the items at both stages are scored together as one polytomous item. Real and simulated examples are provided to discuss challenges in applying IRT models to branching items. We find that model-data misfit is likely to occur when branching items are scored as polytomous items and modeled with the generalized partial credit model and that the relationship between the discrimination of the routing component and the discriminations of the subsequent components seemed to drive the misfit. We conclude with lessons learned and provide suggested guidelines and considerations for operationalizing the use of branching items in future assessments.

The digital transformation of education is greatly accelerating in various computer-supported applications. As a particularly prominent application of the human-machine interactive system, intelligent learning systems aim to capture users' current intentions and provide recommendations through real-time feedback. However, we have a limited understanding on what is the reality of user feedback for computer-supported adaptive quizzing. This present study proposes formalization for user feedback including engagement and interactions to understand and improve intelligent adaptive quizzing. This work starts by taking individual system's action transition patterns as a temporally evolving action graph and derives patterns in terms of informing future user engagement. Furthermore, considering the student knowledge preference for a certain moment is closely related to personal experience of past learning, a novel framework EICQ (Engagement and Interaction-based Computer-Supported Adaptive Quizzing) is proposed for learning and adapting to students' current preference based on engagement trends and interaction by collecting feedback data in real-time during a learning session. This established framework captures question characters, student dynamic interactions and engagement trends, which yield a better learning experience with adaptive practice question (PQ) recommendations. Finally, much more endeavors have been made to integrate the proposed EICQ into XXXU intelligent learning systems that are co-developed with the YYY Company.

Test-taking experience is a consequence of the interaction between students and assessment properties. We define a new notion, rapid-pacing behavior, to reflect two types of test-taking experience -- disengagement and speededness. To identify rapid-pacing behavior, we extend existing methods to develop response-time thresholds for individual items based on empirical patterns of response time and response accuracy. An analysis procedure is proposed to compare the extent of rapid-pacing behavior across testing formats. We also present a case study using datasets from an educational survey assessment of mathematics. A large-scale randomized experiment was conducted involving two testing formats (linear and multistage) for U.S. 8th-grade students. Results suggest that using multistage testing instead of linear testing has differential impact on test-taking experience across performance levels. The lower performers appear more engaged when presented with items better targeted to their proficiency, whereas the higher performers experience more time pressure when presented with more difficult items.

Digital transformations within educational systems are recurrently justified by their promise to enhance learning activities and outcomes. We examine this claim in a study of pupils' perception and experiences with computer-based adaptive tests in higher classes in Danish public primary schools. The study relies on survey and interview data and addresses whether pupils perceive their participation in computer-based adaptive tests as contributing to their learning and the degree to which the perception of teachers' attitude to the test influences their learning. A majority perceives their learning as low or non-existing. However, the perception of a more positive teacher attitude is related to more substantial learning. Theoretically, we attribute this relationship to the effect of self-fulfilling prophecies in the classroom. Learning also differs for subgroups related to gender and ability level. The findings contribute to discussions about educational policy, management, and the digital transformation of educational systems.

Motivation to read is crucial to improving reading skill. While there is extensive research examining reading motivation among elementary students, with respect to adolescents, research is limited. Employing a person-centered approach can aid in developing a better understanding of adolescent reading motivation and would help address possible barriers to engaging adolescent readers. The present study extracted reading motivation profiles in a sample of 367 high school students based on their responses on the Adaptive Reading Motivation Measure (ARMM). Three profiles emerged--high (HRM), ambivalent/neutral (ARM), and low reading motivation (LRM)--where students in the HRM profile performed better on the reading achievement assessment and reported reading more often compared to their peers in the other profiles. Results shed light on key facets of adolescent reading motivation, which have implications for addressing motivational barriers to engaging adolescent readers.

Although one of the challenges in college remedial mathematics courses involves dealing with negative emotions adversely impacting student achievement, little empirical research has been conducted to examine the factors contributing to the formation of such emotions in these settings. The purpose of this study was to examine whether background variables, motivation, and self-regulation predicted students' negative emotions, measured with boredom, frustration, and test anxiety. A total of 201 students taking a remedial mathematics course using an adaptive system - Assessment Learning in Knowledge Space (ALEKS) - participated in the study at a small public university in the midwestern United States. Hierarchical regression was used for the data analysis. Results indicate that students' age (a background variable) and motivation as shown in extrinsic goal orientation, task value, and self-efficacy were significantly associated with boredom, frustration, and test anxiety. In addition, students' self-regulation, specifically effort regulation and metacognitive regulation, was related to negative emotions. Discussion of the results and educational implications are included.

The present study compared the diagnostic accuracy of a single computer adaptive test (CAT), Star Reading or Star Math, and a combination of the two in a gated screening framework to predict end-of-year proficiency in reading and math. Participants included 13,009 students in Grades 3-8 who had at least one fall screening score and end-of-year state test score in reading and math. First, diagnostic accuracy statistics were evaluated for a single screening measure to predict proficiency on end-of-year tests. Second, a gated screening framework was simulated to examine the diagnostic accuracy of a combination of screening measures (i.e., scores from the CATs and the end-of-year test). The diagnostic accuracy of each screening method was compared. Results suggest that diagnostic accuracy did not improve for the gated screening method when compared to the single screening method. The gated screening method tended to yield low sensitivity values (M = 0.42, "range" = 0.35-0.48) and high specificity values (M = 0.97, "range" = 0.95-0.99). The only condition to reach acceptable sensitivity and specificity (>0.70) was a single reading screener predicting reading outcomes. Sample specific cut-scores from receiver operating curve (ROC) analyses led to improved diagnostic accuracy outcomes relative to all other methods.

Purpose: There is an increase globally of students using technology to support their learning. The purpose of this paper is to outline the technical aspects of adaptive learning and contribute to the development of pedagogy that incorporates this method in teaching and learning. Design/methodology/approach: This is a technical review article that summarises key guidance on the application of adaptive learning and then reflects on its application in a UK and Vietnamese context. Findings: Initial analysis demonstrates that learning can occur asynchronously because of students engaging with adaptive learning. Issues and recommendations were derived from the reflections and practice of both UK and Vietnamese practitioners. Recommendations focussed on the more practical elements of constructing and maintaining adaptive learning. Questions were then constructed to make the decision of whether to implement adaptive learning into teaching and learning practices. Originality/value: This academic commentary reflects on the implementation of asynchronous learning adaptive technologies in both the UK and Vietnam, specifically exploring the use of a "mastery path" and "computerised adaptive testing" to enhance student understanding.

Computerized adaptive testing (CAT) has gained deserved popularity in the administration of educational and professional assessments, but continues to face test security challenges. To ensure sustained quality assurance and testing integrity, it is imperative to establish and maintain multiple stable item pools that are consistent in terms of psychometric characteristics and content specifications. This study introduces the Honeycomb Pool Assembly (HPA) framework, an innovative solution for the construction of multiple parallel item pools for CAT that maximizes item utilization in the item bank. The HPA framework comprises two stages--cell assembly and pool assembly--and uses a mixed integer programming modeling approach. An empirical study demonstrated HPA's effectiveness in creating a large number of parallel pools using a real-world high-stakes CAT assessment item bank. The HPA framework offers several advantages, including (a) simultaneous creation of multiple parallel pools, (b) simplification of item pool maintenance, and (c) flexibility in establishing statistical and operational constraints. Moreover, it can help testing organizations efficiently manage and monitor the health of their item banks. Thus, the HPA framework is expected to be a valuable tool for testing professionals and organizations to address test security challenges and maintain the integrity of high-stakes CAT assessments.

This study evaluated the forecasting accuracy of trend estimation methods applied to time-series data from computer adaptive tests (CATs). Data were collected roughly once a month over the course of a school year. We evaluated the forecasting accuracy of two regression-based growth estimation methods (ordinary least squares and Theil-Sen). The precision, or accuracy, of predictions were heavily influenced by how far performance was forecasted into the future (1, 3, or 5 months) and the number of observations available to estimate growth (3, 4, or 5). When performance was forecasted further and growth estimates were based upon fewer observations, predictions were off by as much as two times the average conditional standard error of measurement for a grade level. No combinations of data collections schedule, forecasting length, or estimation method led to consistent bias in predictions. Educators should be cautious when using trend-line decision rules to predict future performance. Suggestions for alternate decision rules to explore with CATs are offered.

In today's environment of growing class sizes due to the prevalence of online and e-learning systems, providing one-to-one instruction and feedback has become a challenging task for teachers. Anyhow, the dialectical integration of instruction and assessment into a seamless and dynamic activity can provide a continuous flow of assessment information for teachers to boost and individualize learning. In this regard, adaptive learning technology is one way to facilitate teacher-supported learning and personalize curriculum and learning experiences. This study aimed to investigate the potential of an adaptive Computerized Dynamic Assessment (C-DA) tool applicable as a language diagnostician and assistant. The study tried to get insight into 75 Iranian EFL learners' listening development by focusing on the learning potential exhibited through learners' assessment and the degree of internalization of mediation. To achieve these, a C-DA tutor including two dynamic listening comprehension tests, each comprising 20 items, arranged in the order of difficulty was developed. The test takers unable to answer an item correctly were provided with graduated hints for different comprehension- and production-type items and the overall difficulty level of the test was adapted to the test takers' proficiency level. In order to have a full diagnosis of each individual's listening development, the adaptive C-DA automatically generated five test scores on each learner's performance: actual (unmediated) score, mediated score, gain score, Learning Potential Score (LPS), and transfer score. The results of paired-sample t-tests revealed a significant development from the actual to the mediated scores. Furthermore, the LPSs indicated that the tutor was capable of revealing learners' potential for learning. Moreover, learners with high LPS gained a higher mean for transfer scores followed by transfer scores of medium and low levels. The results of Mann-Whitney tests revealed a significant difference in the degree of internalization of mediation of learners with mid and low range of LPSs on the easy test and high and low range of LPSs on the difficult test. The findings of this research can have important theoretical and practical implications for researchers and educationalists. The instructional value of this adaptive C-DA tool lies in its unique opportunities for individualizing learning and developing individual learning plans in accordance with learners' needs.

Assessment of language skills for upper elementary and middle schoolers is important due to the strong link between language and reading comprehension. Yet, currently few practical, reliable, valid, and instructionally informative assessments of language exist. This study provides validation evidence for Monster, P.I., which is a gamified, standardized, computer-adaptive assessment (CAT) of language for fifth to eighth grade students. Creating Monster, P.I. involved an assessment of the dimensionality of morphology and vocabulary and an assessment of syntax. Results using multiple-group item response theory (IRT) with 3,214 fifth through eighth graders indicated morphology and vocabulary were best assessed via bifactor models and syntax unidimensionally. Therefore, Monster, P.I. provides scores on three component areas of language (multidimensional morphology and vocabulary and unidimensional syntax) with the goal of informing instruction. Validity results also suggest that Monster, P.I. scores show moderate correlations with each other and with standardized reading vocabulary and reading comprehension assessments. Furthermore, hierarchical regression results suggest an important link between Monster, P.I. and standardized reading comprehension, explaining between 56% and 75% of the variance. Such results indicate that Monster, P.I. can provide meaningful understandings of language performance which can guide instruction that can impact reading comprehension performance.

In the last decades, the development of computerized adaptive testing (CAT) has allowed more precise measurements with a smaller number of items. In this study, we develop an item bank (IB) to generate the adaptive algorithm and simulate the functioning of CAT to assess the domains of mathematical knowledge in Argentinian university students (N=773). Data were analyzed from the Rasch model. A simulation design created with the R software was used to determine the necessary items of the IB to estimate examinee ability. Our results indicate that the IB in the domains of mathematical knowledge is adequate to be applied in CAT, especially to estimate average ability levels. The use of CAT is recommended for rapidly generating indicators of the knowledge acquired by students and to design educational strategies that enhance student performance. Results, constrains, and implications of this study are discussed.

Computerized Adaptive Testing (CAT) is a beneficial test technique that decreases the number of items that need to be administered by taking items in accordance with individuals' own ability levels. After the CAT applications were constructed based on the unidimensional Item Response Theory (IRT), Multidimensional CAT (MCAT) applications have gained momentum with the improvement of multidimensional IRT (MIRT) models in recent years. Researchers often benefit from simulation studies in order to design the final adaptive testing application and to test the effectiveness of adaptive testing applications they developed with different methods. Recently, R has become one of the most widely used programming languages in Monte Carlo Simulation studies since it is a free and open-source software. The aims of this study are to present the MCAT simulation process step by step in the R environment, to examine the effects of the conditions that researchers can handle during the simulation process according to two different dimensional models, and to examine the effect of treating multidimensional structures as unidimensional structures on simulation results. In this direction, datasets generated in accordance with within-item dimensionality and between-item dimensionality models, MCAT simulation studies were constructed with different customizations, and MCAT simulation results were compared with unidimensional CAT simulation results. All commands required for each simulation example were explained and results were shared for each condition.

The first two decades of the twenty-first century witnessed the blossoming of frameworks to conceptualize "21st-century skills" both in school and career. However, there is still a lack of consensus about what we are talking about when using the term "21st-century skills," as the existing frameworks on "21st-century skills" are different from each other. Drawing upon 471 existing sources published in 2000-2017 regarding "21st-century skills," identified from six research databases, this study was intended to answer the following research question via a systematic review: What skills had been considered as "21st-century skills" in the existing educational research literature? Seven categories or domains of knowledge and skills emerged when investigating this literature. Implications and future research directions about how adaptive learning and adaptive testing in the new century can empower educational and vocational guidance were briefly discussed at the end of this paper.

Several forced-choice (FC) computerized adaptive tests (CATs) have emerged in the field of organizational psychology, all of them employing ideal-point items. However, despite most items developed historically follow dominance response models, research on FC CAT using dominance items is limited. Existing research is heavily dominated by simulations and lacking in empirical deployment. This empirical study trialed a FC CAT with dominance items described by the Thurstonian Item Response Theory model with research participants. This study investigated important practical issues such as the implications of adaptive item selection and social desirability balancing criteria on score distributions, measurement accuracy and participant perceptions. Moreover, nonadaptive but optimal tests of similar design were trialed alongside the CATs to provide a baseline for comparison, helping to quantify the return on investment when converting an otherwise-optimized static assessment into an adaptive one. Although the benefit of adaptive item selection in improving measurement precision was confirmed, results also indicated that at shorter test lengths CAT had no notable advantage compared with optimal static tests. Taking a holistic view incorporating both psychometric and operational considerations, implications for the design and deployment of FC assessments in research and practice are discussed.

Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of nonipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of (a) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled "on-the-fly" considering every possible pair of items), and (b) block selection rules (i.e., T, and Bayesian D and A-rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a nonadaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A-rule with questionnaires assembled "on-the-fly," the T-rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT.

This study explored the validity of growth on two computer adaptive tests, Star Reading and Star Math, in explaining performance on an end-of-year achievement test for a sample of students in Grades 3 through 6. Results from quantile regression analyses indicate that growth on Star Reading explained a statistically significant amount of variance in performance on end-of-year tests after controlling for baseline performance in all grades. In Grades 3 through 5, the relationship between growth on Star Reading and the end-of-year test was stronger among students who scored higher on the end-of-year test. In math, Star Math explained a statistically significant amount of variance in end-of-year scores after statistically controlling for baseline performance in all grades. The strength of the relationship did not differ among students who scored lower or higher on the end-of-year test across grades.

The article deals with the innovative, cutting age solution within the language testing realm, namely computer adaptive language testing (CALT) in accordance with the NATO Standardization Agreement 6001 (NATO STANAG 6001) requirements for further implementation in foreign language training of personnel of the Armed Forces of Ukraine (AF of Ukraine) in order to increase the quality of foreign language testing. The research provides the CALT method developed according to NATO STANAG 6001 requirements and the CALT algorithm that contains three blocks: "Starting point", "Item selection algorithm", "Scoring algorithm" and "Termination criterion". The CALT algorithm has an adaptive ability, changing a complexity level, sequence and the number of items according to the answers of a test taker. The comparative analysis of the results of the CALT method piloting and the paper-and-pencil testing (PPT) in reading and listening according to the NATO STANAG 6001 requirements justifies the effectiveness of the three-level CALT method. It allows us to determine the following important benefits of CALT: test length reduction, control of measurement accuracy, objective assessment, improved test security, generation of a unique set of items, adaptive ability of the CALT algorithm, high motivation of the test takers, immediate score reporting and test results management. CALT is a qualitative and effective tool to determine test takers' foreign language proficiency level in accordance with NATO STANAG 6001 requirements within the NATO Defence Educational Enhancement Programme. CALT acquires a special value and relevance in the context of the global COVID-19 pandemic.

Item response theory provides various important advantages for exams carried out or to be carried out digitally. For computerized adaptive tests to be able to make valid and reliable predictions supported by IRT, good quality item pools should be used. This study examines how adaptive test applications vary in item pools which consist of items with varying difficulty levels. Within the scope of the study, the impact of items was examined where the parameter b differentiates while the parameters a and c are kept in fixed range. To this end, eight different 2000- people item pools were designed in simulation which consist of 500 items with ability scores and varying difficulty levels. As a result of CAT simulations, RMSD, BIAS and test lengths were examined. At the end of the study, it was found that tests run by item pools with parameter b in the range that matches the ability level end up with fewer items and have a more accurate stimation. When parameter b takes value in a narrower range, estimation of ability for extreme ability values that are not consistent with parameter b required more items. It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.

E-authentication is one of the key topics in the field of online education and e-assessment. This study was aimed at investigating the user experiences of students with special educational needs and disabilities (SEND) while developing the accessible e-authentication system for higher education institutions. Altogether, 15 students tested the system (including instruments for face recognition, voice recognition, keystroke dynamics, text style analysis and anti-plagiarism), developed as part of the TeSLA project. Students also completed pre-questionnaires and post-questionnaires and attended individual interviews. The findings reveal positive expectations and experiences of e-authentication. Students believed that the e-authentication system increased trust and, thus, diversified their possibilities for studying online. Students found some challenges and emphasized that thea e-authentication system should be reliable and easy to use. The possibility to use different kinds of instruments was perceived as an important feature. Students' willingness to use these instruments and share their personal data for e-authentication varied due to their disabilities or individual preferences. The results suggest that students should have options for what kind of e-authentication they use. [Note: The publication year (2020) shown on the PDF is incorrect. The correct publication year for this article is 2021.]

To improve the calibration accuracy and calibration efficiency of cognitive diagnostic computerized adaptive testing (CD-CAT) for new items and, ultimately, contribute to the widespread application of CD-CAT in practice, the current article proposed a Gini-based online calibration method that can simultaneously calibrate the Q-matrix and item parameters of new items. Three simulation studies with simulated and real item banks were conducted to investigate the performance of the proposed method and compare it with the joint estimation algorithm (JEA) and the single-item estimation (SIE) methods. The results indicated that the proposed Gini-based online calibration method yielded higher calibration efficiency than those of the SIE method and outperformed the JEA method on item calibration tasks in terms of both accuracy and efficiency under most experimental conditions.

Computerized adaptive testing (CAT) offers an efficient and highly accurate method for estimating examinees' abilities. In this article, the free version of Concerto Software for CAT was reviewed, dividing our evaluation into three sections: software implementation, the Item Response Theory (IRT) features of CAT, and user experience. Overall, Concerto is an excellent tool for researchers seeking to create computerized adaptive (or non-adaptive) tests, providing a robust platform with comprehensive IRT capabilities and a user-friendly interface.

In low-stakes assessment settings, students' performance is not only influenced by students' ability level but also their test-taking engagement. In computerized adaptive tests (CATs), disengaged responses (e.g., rapid guesses) that fail to reflect students' true ability levels may lead to the selection of less informative items and thereby contaminate item selection and ability estimation procedures. To date, researchers have developed various approaches to detect and remove disengaged responses after test administration is completed to alleviate the negative impact of low test-taking engagement on test scores. This study proposes an alternative item selection method based on Maximum Fisher Information (MFI) that considers test-taking engagement as a secondary latent trait to select the most optimal items based on both ability and engagement. The results of post-hoc simulation studies indicated that the proposed method could optimize item selection and improve the accuracy of final ability estimates, especially for low-ability students. Overall, the proposed method showed great promise for tailoring CATs based on test-taking engagement. Practitioners are encouraged to consider incorporating engagement into the item selection algorithm to enhance the validity of inferences made from low-stakes CATs.

Online calibration is a key technology for item calibration in computerized adaptive testing (CAT) and has been widely used in various forms of CAT, including unidimensional CAT, multidimensional CAT (MCAT), CAT with polytomously scored items, and cognitive diagnostic CAT. However, as multidimensional and polytomous assessment data become more common, only a few published reports focus on online calibration in MCAT with polytomously scored items (P-MCAT). Therefore, standing on the shoulders of the existing online calibration methods/designs, this study proposes four new P-MCAT online calibration methods and two new P-MCAT online calibration designs and conducts two simulation studies to evaluate their performance under varying conditions (i.e., different calibration sample sizes and correlations between dimensions). Results show that all of the newly proposed methods can accurately recover item parameters, and the adaptive designs outperform the random design in most cases. In the end, this paper provides practical guidance based on simulation results.