Search

To compare unproctored and proctored online exams among chiropractic students. Pre-existing data of 234 students across 4 consecutive endocrinology classes were analyzed for this study. The course was comprised of 3 lectures (50 minutes per lecture) each week. Student performance was evaluated by midterm exam and summative exam (S1). The students from 3 classes were asked to take a voluntary second summative exam (S2) approximately 7 months after the S1. Since this study was partially conducted during the COVID pandemic, some classes took the midterm and the S1 proctored in the classroom while others took them unproctored from a remote location. The mean midterm exam (p <.001) and S1 scores (p =.01) for the unproctored group (93.6 ± 7.0 and 88.8 ± 8.2) were significantly higher than the proctored group (88.1 ± 8.2 and 83.9 ± 11.2). The mean time taken by students was much greater for the unproctored exams than for the proctored exams (midterm: 40.7 ± 10.2 versus 16.7 ± 7.0, p <.001; S1: 47.0 ± 8.7 versus 21.5 ± 9.0, p <.001). By contrast, the mean unproctored S2 scores were lower than the proctored group (60.2 ± 14.7 versus 88.1 ± 8.2, p <.001). A linear regression test showed that the final exam was a statistically significant predictor of the recall exam (p <.01, R2 = 28.3%). The findings suggest that student performance is significantly altered by test format. [ABSTRACT FROM AUTHOR]

Introduction: This study aimed at exploring the feasibility of applying a computer-based language test to young children aged 2–4 years. Methods: Thirty-two Cantonese-speaking children, aged 2–4 years, were recruited from local kindergartens. All participants underwent an assessment using both the computer-based and paper-pencil versions of the Macau Cantonese Language Screening Scale for Preschool Children, following a crossover study design. A short break of 15–30 min was provided between the two assessments. The data were analysed at three levels: the overall test, subcategory, and individual item levels. At the overall test and subcategory levels, data were analysed using the paired samples t-test and ICC. At the item level, the percentage of agreement and Cohen's kappa value were selected to assess the agreement of the two test formats. Results: Excellent agreement was found for the overall test level, and good agreement was observed for four of the five subcategories. At the individual item level, 28 of the 35 items showed more than 80% agreement, and 16 items showed substantial to almost perfect agreement. Conclusion: These results suggest that the two test formats give similar total scores and subcategory scores for children aged 2–4. For children older than 2 years and 6 months, the agreement for matching items is as high as 83.68% (1,318/1,575). The computer-based test is thus highly recommended for this group of children. For children younger than 2 years and 6 months, a modified computer-based test is suggested to accommodate their needs. [ABSTRACT FROM AUTHOR]

In various fields, including college admission, medical board certifications, and military recruitment, high-stakes decisions are frequently made based on scores obtained from large-scale assessments. These decisions necessitate precise and reliable scores that enable valid inferences to be drawn about test-takers. However, the ability of such tests to provide reliable, accurate inference on a test-taker's performance could be jeopardized by aberrant test-taking practices, for instance, practicing real items prior to the test. As a result, it is crucial for administrators of such assessments to develop strategies that detect potential aberrant test-takers after data collection. The aim of this study is to explore the implementation of machine learning methods in combination with multimodal data fusion strategies that integrate bio-information technology, such as eye-tracking, and psychometric measures, including response times and item responses, to detect aberrant test-taking behaviors in technology-assisted remote testing settings. [ABSTRACT FROM AUTHOR]

Background: Peer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop an effective automated grading model to analyse the data and predict learners' learning results. Objectives: The present study aimed to propose an automated grading model with reliability detection. Methods: A total of 109,327 instances of peer assessment from a large‐scale teacher online learning program were tested in the experiments. The reliability detection approach included three steps: recurrent convolutional neural networks (RCNN) was used to detect grade consistency, bidirectional encoder representations from transformers (BERT) was used to detect text originality, and long short‐term memory (LSTM) was used to detect grade‐text consistency. Furthermore, the automated grading was designed with the BERT‐RCNN model. Results and Conclusions: The effectiveness of the automated grading model with reliability detection was shown. For reliability detection, RCNN performed best in detecting grade consistency with an accuracy rate of 0.889, BERT performed best in detecting text originality with an improvement of 4.47% compared to the benchmark model, and LSTM performed best with an accuracy rate of 0.883. Moreover, the automated grading model with reliability detection achieved good performance, with an accuracy rate of 0.89. Compared to the absence of reliability detection, it increased by 12.1%. Implications: The results strongly suggest that the automated grading model with reliability detection for large‐scale peer assessment is effective, with the following implications: (1) The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results. (2) This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. (3) This solution could shift educational institutions from labour‐intensive grading procedures to a more efficient educational assessment pattern, allowing for more investment in supporting instructors and learners to improve the quality of peer feedback. Lay Description: What is already known about this topic: Peer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning.Issues such as disagreement between peer assessors, rough assessment, and plagiarism in large‐scale online learning can decrease peer assessment reliabilityIncorporating extensive data into a training model may result in grading uncertainties. What this paper adds: Detecting the peer assessment reliability before grading is essential in the context of large‐scale online learning.This study aimed to propose and validate an automated grading model with reliability detection for the large‐scale online peer assessment, which will help improve the effectiveness of automated grading, combining the advantages of computer technology and human expertise. Implications for practice and/or policy: The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results.This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. [ABSTRACT FROM AUTHOR]

Children with mathematical difficulties need to spend more time than typically achieving children on solving even simple equations. Since these tasks already require a larger share of their cognitive resources, additional demands imposed by the need to switch between tasks may lead to a greater decline of performance in children with mathematical difficulties. We explored differential task switch costs with respect to switching between addition versus subtraction with a tablet-based arithmetic verification task and additional standardized tests in German elementary school children in Grades 1 to 4. Two independent studies were conducted. In Study 1, we assessed the validity of a newly constructed tablet-based arithmetic verification task in a controlled classroom-setting (n = 165). Then, effects of switching between different types of arithmetic operations on accuracy and response latency were analyzed through generalized linear mixed models in an online-based testing (Study 2; n = 3,409). Children with mathematical difficulties needed more time and worked less accurately overall. They also exhibited a stronger performance decline when working in a task-switching condition, when working on subtraction (vs. addition) items and in operations with two-digit (vs. one-digit) operations. These results underline the value of process data in the context of assessing mathematical difficulties. [ABSTRACT FROM AUTHOR]

This study analyzed performance, process, and survey data of eighth graders with learning disabilities (LDs) who took the 2017 National Assessment of Educational Progress (NAEP) digital math test. Compared with students with LDs who did not receive extended time accommodations (ETAs), students with LDs who received and used ETA scored significantly higher on the test, whereas students with LDs who received but did not use ETA scored significantly lower on the test. In addition, students with LDs in the two ETA groups reported a lower level of perceived time pressure and a higher level of math interest and enjoyment than their peers who did not receive ETA. For students with LDs who received ETA, optimal performance was achieved with 50% additional time, while their peers who did not receive ETA typically performed best when utilizing most of their allotted time. The analysis of process data revealed that students with LDs who used ETA performed more actions, had a higher number of revisits, used universal design digital tools more frequently, and performed better on time-consuming items than their peers who did not receive ETA at the same level of math performance. [ABSTRACT FROM AUTHOR]

Purpose: Artificial intelligence, specifically large language models such as ChatGPT, offers valuable potential benefits in question (item) writing. This study aimed to determine the feasibility of generating case-based multiple-choice questions using ChatGPT in terms of item difficulty and discrimination levels. Methods: This study involved 99 fourth-year medical students who participated in a rational pharmacotherapy clerkship carried out based-on the WHO 6-Step Model. In response to a prompt that we provided, ChatGPT generated ten case-based multiple-choice questions on hypertension. Following an expert panel, two of these multiple-choice questions were incorporated into a medical school exam without making any changes in the questions. Based on the administration of the test, we evaluated their psychometric properties, including item difficulty, item discrimination (point-biserial correlation), and functionality of the options. Results: Both questions exhibited acceptable levels of point-biserial correlation, which is higher than the threshold of 0.30 (0.41 and 0.39). However, one question had three non-functional options (options chosen by fewer than 5% of the exam participants) while the other question had none. Conclusions: The findings showed that the questions can effectively differentiate between students who perform at high and low levels, which also point out the potential of ChatGPT as an artificial intelligence tool in test development. Future studies may use the prompt to generate items in order for enhancing the external validity of the results by gathering data from diverse institutions and settings. [ABSTRACT FROM AUTHOR]

Educational institutions are now concentrating to develop new initiative methods for recognizing the ability of the students through evaluation of the student's answer sheets. The manual evaluation of the answer sheet is a burdensome process for the tutors, which consumes more time and increases stress of the tutors. Hence, the advanced method is required for the automatic assessment of the student's mark sheet which saves the time of the tutors and meets the demand of the educational institution. In this research, the student performance is evaluated through the MCQ test using the neural networks in the VLab platform. The computational complexity and the overfitting issues are greatly reduced by the feature extraction process through Term Frequency-Inverse Document Frequency (TF-IDF) technique. The effectiveness of the proposed method is manifested through the comparative analysis. The accuracy, precision, and recall attained by the proposed student's performance prediction based on Neural Network while considering training percentage are found to be 0.9416, 0.9364, and 0.9502 respectively. The accuracy, precision, and recall attained by the proposed student's performance prediction based on Neural Network, while considering the K-fold value are found to be 0.9475, 0.9474, 0.9538, respectively. [ABSTRACT FROM AUTHOR]

Disengaged responding poses a severe threat to the validity of educational large-scale assessments, because item responses from unmotivated test-takers do not reflect their actual ability. Existing identification approaches rely primarily on item response times, which bears the risk of misclassifying fast engaged or slow disengaged responses. Process data with its rich pool of additional information on the test-taking process could thus be used to improve existing identification approaches. In this study, three process data variables—text reread, item revisit, and answer change—were introduced as potential indicators of response engagement for multiple-choice items in a reading comprehension test. An extended latent class item response model for disengaged responding was developed by including the three new indicators as additional predictors of response engagement. In a sample of 1,932 German university students, the extended model indicated a better model fit than the baseline model, which included item response time as only indicator of response engagement. In the extended model, both item response time and text reread were significant predictors of response engagement. However, graphical analyses revealed no systematic differences in the item and person parameter estimation or item response classification between the models. These results suggest only a marginal improvement of the identification of disengaged responding by the new indicators. Implications of these results for future research on disengaged responding with process data are discussed. [ABSTRACT FROM AUTHOR]

Purpose The purpose of this study was to clarify the perceptions of full-time faculty members regarding the efforts and challenges of quality assurance of pre-practicum midwifery students (hereafter referred to as "students") as well as the common midwifery achievement test consisting of Computer-based testing (CTB) and Objective Structured Clinical Examination (OSCE) at midwifery training schools by educational year. Subjects and Methods A web-based survey was conducted from October 2020 to January 2021 targeting 436 faculty members in total from 218 midwifery training schools nationwide that included an educational affairs chief who is responsible for training students and curriculum coordination and a full-time faculty member from each of the 218 training schools. We asked about their efforts to assure the quality of pre-practicum students and the issues still to be resolved, and their perceptions of the CTB and OSCE. We analyzed the data by dividing the respondents into three groups: one-year education course, two-year education course, and bachelor's program (elective course). Results We received 149 responses (valid response rate: 34.2%). Of the 143 respondents (96.0%) who answered that they were working to assure the quality of their students prior to practicum, 140 (94.0%) indicated that there were issues with their current efforts. The most frequently seen issue was "not enough time," with a significantly higher percentage of respondents in the 1-year education course and bachelor's program (elective course) compared to the 2-year education course (p=0.010). Regarding CTB and OSCE there was no difference depending on the program in the percentage (94.6%) who answered that they were "necessary/somewhat necessary" but the percentage of respondents who answered that an examination to assure the quality of students before practical training was "not very necessary/not necessary" was significantly higher for bachelor's program (elective course) compared to the one-year and two-year education courses (p=0.047). Conclusion Efforts to assure the quality of practicum students were already underway at many institutions, but the lack of time in the curriculum was a challenge. The common midwifery achievement test was recognized as a necessary system to assure the quality of students prior to practical training. [ABSTRACT FROM AUTHOR]

The rapid advancement of artificial intelligence (AI), particularly the development of Large Language Models (LLMs) such as Generative Pretrained Transformers (GPTs), has revolutionized numerous fields. The purpose of this study is to investigate the application of LLMs within the realm of orthopaedic in training examinations. Questions from the 2020–2022 Orthopaedic In-Service Training Exams (OITEs) were given to OpenAI's GPT-3.5 Turbo and GPT-4 LLMs, using a zero-shot inference approach. Each model was given a multiple-choice question, without prior exposure to similar queries, and their generated responses were compared to the correct answer within each OITE. The models were evaluated on overall accuracy, performance on questions with and without media, and performance on first- and higher-order questions. The GPT-4 model outperformed the GPT-3.5 Turbo model across all years and question categories (2022: 67.63% vs. 50.24%; 2021: 58.69% vs. 47.42%; 2020: 59.53% vs. 46.51%). Both models showcased better performance with questions devoid of associated media, with GPT-4 attaining accuracies of 68.80%, 65.14%, and 68.22% for 2022, 2021, and 2020, respectively. GPT-4 outscored GPT-3.5 Turbo on first-order questions across all years (2022: 63.83% vs. 38.30%; 2021: 57.45% vs. 50.00%; 2020: 65.74% vs. 53.70%). GPT-4 also outscored GPT-3.5 Turbo on higher-order questions across all years (2022: 68.75% vs. 53.75%; 2021: 59.66% vs. 45.38%; 2020: 53.27% vs. 39.25%). GPT-4 showed improved performance compared to GPT-3.5 Turbo in all tested categories. The results reflect the potential and limitations of AI in orthopaedics. GPT-4's performance is comparable to a second-to-third-year resident and GPT-3.5 Turbo's performance is comparable to a first-year resident, suggesting the application of current LLMs can neither pass the OITE nor substitute orthopaedic training. This study sets a precedent for future endeavors integrating GPT models into orthopaedic education and underlines the necessity for specialized training of these models for specific medical domains. [ABSTRACT FROM AUTHOR]

Introduction.: Feedback after computer-based assessments is essential to the learning process. Options include knowledge of results (KR), knowledge of correct results (KCR), and elaborated feedback (EF). This study aimed to identify whether specific or broader forms of postassessment feedback optimize physical therapist students' computer-based assessment scores. Review of Literature.: Health care educators have various types of feedback they can provide to students. Educators may consider the need for exam integrity and increasing student engagement when determining what type of feedback to deliver. Elaborated feedback seems to be most effective in higher-order learning objectives than KCR or KR to improve learning outcomes and can be delivered in clear and engaging ways for students while maintaining exam integrity. Subjects.: Forty-nine first-year Doctor of Physical Therapy students. Methods.: This study was a quasi-experimental design. Students were given a feedback form postassessment: secure exam review with note sheet (KCR) or Strengths and Opportunities Report (EF). Students repeated assessments on the same content (mix of repeat and related items). Exam scores and question performance were analyzed with linear and logistic regression. Students were surveyed about feedback preferences. Results.: After adjusting for differences in the initial exam scores and graduate Grade Point Average, KCR improved retake exam scores (confidence interval [CI95] = 0.07–1.82 percentage points). Knowledge of correct results was more effective than EF for repeated questions (RR = 1.53, CI95 = 1.12–2.09, P =.0031). Knowledge of correct results and EF were similarly effective for related questions (RR = 1.01, CI95 = 0.76–1.33, P =.9997). Most students (89.75%) preferred KCR. Discussion and Conclusion.: Compared with EF, KCR provided modest improvements on follow-up exams, specifically on repeated questions. Because students are unlikely to encounter repeated questions on licensure exams, EF may be an excellent choice to balance exam integrity and feedback quality. [ABSTRACT FROM AUTHOR]

Copyright of Gerokomos is the property of Indemm Farma SL and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background: Research into online exams in higher education has grown significantly, especially as they became common practice during the COVID‐19 pandemic. However, previous studies focused on understanding individual factors that relate to students' dispositions towards online exams in 'traditional' universities. Moreover, there is little knowledge on university distance learning students' experience of transitioning from in‐person to online exams. Objectives: This study investigates the acceptance and satisfaction of university distance learning students in their transitioning from in‐person to online exams, through multiple factors. Methods: We employed a mixed‐methods study to understand the relationship between assessment and online exam factors (e.g., revision and online exam satisfaction, assessment competencies, invigilation acceptance, exam anxiety and workspace satisfaction). Cluster analysis and interview data contributed to our understanding of students who are 'strongly positive' and 'less positive' towards online exams. Results and Conclusions: Our findings highlight the overall importance of increasing student confidence by building their assessment competencies throughout their studies and familiarising them early with the technologies and formats to be used in the actual exam. We also shed light on particular student characteristics that relate to reduced online exam acceptance, such as students with disabilities, caring responsibilities and mental health issues, or students who lack access to the necessary technology. Implications: The findings and recommendations of this research contribute to the wider agenda of designing fair and trustworthy online assessment, including exams, for the future. Lay Description: What is already known about this topic?: Studies focusing on students' perceptions of online exams have increased.The COVID‐19 pandemic has increased online exams at universities.Research show that students appreciate online exams.The factors that contribute to students' positive dispositions are not clear. What this paper adds?: A mixed‐methods study on online exams in distance learning higher education.Factors that determine students' positive dispositions towards online exams.Student characteristics that relate to reduced online exam acceptance. Implications for practice: Guidelines for designing online exam systems positively perceived by students.Online exams to be an integral part of teaching and learning design. [ABSTRACT FROM AUTHOR]

Issue: Automatic item generation is a method for creating medical items using an automated, technological solution. Automatic item generation is a contemporary method that can scale the item development process for production of large numbers of new items, support building of multiple forms, and allow rapid responses to changing medical content guidelines and threats to test security. The purpose of this analysis is to describe three sources of validation evidence that are required when producing high-quality medical licensure test items to ensure evidence for valid test score inferences, using the automatic item generation methodology for test development. Evidence: Generated items are used to make inferences about examinees' medical knowledge, skills, and competencies. We present three sources of evidence required to evaluate the quality of the generated items that is necessary to ensure the generated items measure the intended knowledge, skills, and competencies. The sources of evidence we present here relate to the item definition, the item development process, and the item quality review. An item is defined as an explicit set of properties that include the parameters, constraints, and instructions used to elicit a response from the examinee. This definition allows for a critique of the input used for automatic item generation. The item development process is evaluated using a validation table, whose purpose is to support verification of the assumptions related to model specification made by the subject-matter expert. This table provides a succinct summary of the content and constraints that were used to create new items. The item quality review is used to evaluate the statistical quality of the generated items, which often focuses on the difficulty and the discrimination of the correct and incorrect options. Implications: Automatic item generation is an increasingly popular item development method. The generated items from this process must be bolstered by evidence to ensure the items measure the intended knowledge, skills, and competencies. The purpose of this analysis is to describe these sources of evidence that can be used to evaluate the quality of the generated items. The important role of medical expertise in the development and evaluation of the generated items is highlighted as a crucial requirement for producing validation evidence. [ABSTRACT FROM AUTHOR]

Existing evidence suggested learners with differences in attention and cognition might respond to the same media in differential ways. The current study focused on one format of video design – instructor visibility and explored the moderating effects of working memory capacity on learning from such video design and if learners with high and low working memory capacity attended to the instructor's visuals differently. Participants watched a video either with or without the instructor's visuals on the screen, while their visual attention was recorded simultaneously. After the video, participants responded to a learning test that measured retention and transfer. Although the results did not show working memory capacity moderated the instructor visibility effects on learning or influenced learners' visual attention to the instructor's visuals, the findings did indicate working memory capacity was a positive predictor of retention performance regardless of the video design. Discussions and implications of the findings were provided. [ABSTRACT FROM AUTHOR]

Background: Artificial intelligence (AI) has the potential to revolutionize nursing education. This study compared NCLEX-RN questions generated by AI and those created by nurse educators. Method: Faculty of accredited baccalaureate programs were invited to participate. Likert-scale items for grammar and clarity of the item stem and distractors were compared using Mann-Whitney U, and yes/no questions about clinical relevance and complex terminology were analyzed using chi-square. A one-sample binomial test with confidence intervals evaluated participants' question preference (AI-generated or educator-written). Qualitative responses identified themes across faculty. Results: Item clarity, grammar, and difficulty were similar for AI and educator-created questions. Clinical relevance and use of complex terminology was similar for all question pairs. Of the four sets with preference for one item, three were generated by AI. Conclusion: AI can assist faculty with item generation to prepare nursing students for the NCLEX-RN examination. Faculty expertise is necessary to refine questions written using both methods. [J Nurs Educ. 2023;62(12):679–687.] [ABSTRACT FROM AUTHOR]

The assessment of attention-deficit/hyperactivity disorder (ADHD) typically relies on subjective observer reports from parents, teachers, or the young person, combined with clinical observation and history. Children and young people can often experience lengthy delays to assessment and medication initiation resulting from conflicting or missing observer reports and diagnostic uncertainty. However, more recently, a computerised test of attention, impulsivity and activity (QbTest) has been implemented as an adjunct to standard clinical practice, with the aim to provide a more objective measure of ADHD symptoms. Here, we discuss the evidence for the clinical utility of QbTest to aid in the assessment and monitoring of ADHD. Drawing on key literature and real-world case studies, we show the potential benefits that QbTest may have in creating service efficiencies for ADHD care, but also note limitations in diagnostic accuracy, importantly demonstrating that QbTest should supplement and not replace standard care. We review key barriers and facilitators to implementation, to aid decision making and planning in how to integrate QbTest in paediatric services. [ABSTRACT FROM AUTHOR]

Background: The use of e‐exams in higher education is increasing. However, the role of student diversity in the acceptance of e‐exams is an under‐researched topic. In the current study, we considered student diversity in terms of three sociodemographic characteristics (age, gender, and second language) and three dispositional student characteristics (computer anxiety, test anxiety, and technology openness). Objectives: The main objective of this study was to investigate the relationship between student diversity and acceptance of e‐exams. Methods: Our research combined cross‐sectional analyses (N = 1639) with data from a natural experiment on the introduction of e‐exams versus the established paper‐pencil exams (N = 626) and used both self‐report and institutional data. Sociodemographic and dispositional characteristics were indirectly related to pre‐exam acceptance via expectancy variables from the Technology Acceptance Model framework. Results and Conclusions: Comparisons of post‐exam acceptance showed that practical experience with the e‐exam led to a significant increase in e‐exam acceptance, and that students with low openness toward technology particularly benefited from this effect. Students' exam performance (i.e., grades) was unrelated to the exam format or their pre‐exam acceptance of the e‐exam format, and this was true across students' sociodemographic and dispositional characteristics. Takeaway: Student diversity plays a role in e‐exam acceptance, but its influence is mitigated by first‐hand experience with e‐exams. The practical implications for higher education institutions aiming to implement e‐exams are discussed. Lay Description: What is already known about this topic: The use of e‐exams in higher education is increasingThe role of student diversity for e‐exam acceptance is unclearTechnology acceptance is predicted by expectancies towards new system What this paper adds: We investigated students' sociodemographic and dispositional diversityDiversity predicted e‐exam acceptance via the expectanciesIn a natural experiment, first‐hand experience increased e‐exam acceptanceNo difference between performance in e‐exams and paper‐pencil‐exams found Implications of the study findings for practitioners: Higher education institutions implementing e‐exams should consider diversitySupport for older students and students with low technology openness neededNo student groups systematically disadvantaged by e‐exam implementationPractice rooms can increase experience with new system before exam [ABSTRACT FROM AUTHOR]

Background: The emergence of the COVID‐19 and the resulting global pandemic has ushered in far‐reaching changes for countries across the world, not least of which are changes to their education systems. With traditional location‐based exams no longer possible at universities, the uptake of online proctored exams (OPE) has occurred at a pace not seen prior to the pandemic. Students' experiences of online proctored exams during the pandemic are reasonably well‐understood in terms of digital access and ease of use of the technology. However, less is known about students' perceptions of digital confidence and competence to complete an online exam, both of which are important digital equity considerations. Objectives: This study investigates students' digital confidence and competence to undertake online proctored exams to determine whether issues of equity exist for students. Methods: This study reports the results of a survey (N = 761) of one university's students' experiences of end‐of‐semester online proctored exams in New Zealand. Results: Most students were positive about online exams, felt reasonably confident to complete an exam, and had the necessary digital access (i.e., devices, internet) and competence to succeed. However, digital inequalities were found between students' perceptions of digital competence based on ethnicity. While more time learning online equated to increasing self‐reported digital competence for most ethnic groups, this was not the case for Pacific learners. Takeaways: Perceptions of low digital competence can contribute to digital inequalities. Educators can support learners to develop positive perceptions of digital competence by teaching digital literacy skills. Lay Description: What is already known about this topic: Online proctored exams (OPE) have replaced many traditional exams because of the pandemic.Research on the use of online proctored exams during the pandemic suggests some acceptance among students.Apart from access related issues, little is known about digital inequalities associated with online exams. What this paper adds: Most participants were positive about OPE and felt reasonably confident to successfully complete them.Confident students were more likely to report that the exam software was easy to use.Perceptions of digital competence differed based on ethnicity.Marginalized groups were more likely to experience digital inequalities than other groups. Implications for practice and/or policy: Digital inequality is about more than issues of digital access (i.e., devices, internet).Provide differentiated technical support for students who express low digital competence.Teaching online exam digital literacy skills might increase perceptions of digital competence. [ABSTRACT FROM AUTHOR]

Chinese character size is the number of characters that a person can recognize and has been well documented as a critical measure of Chinese literacy. A variety of Chinese character size tests have been developed since the 1930s. However, systematic reviews have not yet been conducted on Chinese character size tests. The purpose of this article is to provide a comprehensive review of Chinese character size tests conducted between 1930 and 2021 among native Chinese-speaking children and Chinese language learners. There are three main findings. First, most character size tests were constructed using a frequency-based stratified-sampling method to select target characters, a mixed method focusing on both pronunciation and meaning to test target characters, and a holistic method to score the test-takers' responses. Second, the majority of tests used Classical Testing Theory (CTT) for checking the item quality, reliability, and validity, and only two tests employed both CTT and Item Response Theory. Third, most tests estimated character sizes using CTT, while only three tests constructed character size norms. It is suggested that future studies address cross-group investigation, determine the most robust construction and estimation methods, develop computer-assisted tests, and apply character size tests to classroom settings. [ABSTRACT FROM AUTHOR]

Background: Item response times in computerized assessments are frequently used to identify rapid guessing behaviour as a manifestation of response disengagement. However, non‐rapid responses (i.e., with longer response times) are not necessarily engaged, which means that response‐time‐based procedures could overlook disengaged responses. Therefore, the identification of disengaged responses could be improved by considering additional indicators of disengagement. We investigated the extent to which decreases in individuals' item solution probabilities over the course of a test reflect disengaged response behaviour. Objectives: To disentangle different types of possibly disengaged responses and better understand non‐effortful test‐taking behaviour, we augmented responses‐time‐based procedures for identifying rapid guessing with strategies for detecting disengaged responses on the basis of performance declines in non‐rapid responses. Methods: We combined item response theory (IRT) models for rapid guessing and test‐taking persistence to examine the capability of response times and item positions to capture response disengagement. We used a computerized assessment in which science items were randomly distributed across positions for each student. This allowed us to estimate individual differences in test‐taking persistence (i.e., the duration for which the initial level of performance is maintained) while accounting for rapid responses. Results and Conclusions: Response times did not fully explain disengagement; item responses reflected test‐taking persistence even when rapid responses were accounted for. This interpretation was supported by a strong correlation of test‐taking persistence with decreases in self‐reported test‐taking effort. Furthermore, our results suggest that IRT models for test‐taking persistence can effectively account for the undesirable impact of low test‐taking effort even when response times are unavailable. Practitioner Notes: Assessments of proficiencies that attempt to quantify what individuals know and can do lead to biased results when individuals provide disengaged responses.Item response times are frequently used to identify rapid guessing behaviour as a manifestation of response disengagement, but response‐time‐based procedures could overlook disengaged responses.To disentangle different types of possibly disengaged responses and better understand non‐effortful test‐taking behaviour, we augmented responses‐time‐based procedures for identifying rapid guessing with strategies for detecting disengaged responses on the basis of performance declines in non‐rapid responses.In a sample of fifth and sixth graders, we found that response times did not fully explain disengagement, as many students showed performance declines in non‐rapid item responses.Our results suggest that item response theory models for test‐taking persistence can effectively account for the undesirable impact of low test‐taking effort even when response times are unavailable. [ABSTRACT FROM AUTHOR]

Multiple choice questions (MCQs) suffer from cueing, item quality and factual knowledge testing. This study presents a novel multimodal test containing alternative item types in a computer-based assessment (CBA) format, designated as Proxy-CBA. The Proxy-CBA was compared to a standard MCQ-CBA, regarding validity, reliability, standard error of measurement, and cognitive load, using a quasi-experimental crossover design. Biomedical students were randomized into two groups to sit a 65-item formative exam starting with the MCQ-CBA followed by the Proxy-CBA (group 1, n = 38), or the reverse (group 2, n = 35). Subsequently, a questionnaire on perceived cognitive load was taken, answered by 71 participants. Both CBA formats were analyzed according to parameters of the Classical Test Theory and the Rasch model. Compared to the MCQ-CBA, the Proxy-CBA had lower raw scores (p < 0.001, η2 = 0.276), higher reliability estimates (p < 0.001, η2 = 0.498), lower SEM estimates (p < 0.001, η2 = 0.807), and lower theta ability scores (p < 0.001, η2 = 0.288). The questionnaire revealed no significant differences between both CBA tests regarding perceived cognitive load. Compared to the MCQ-CBA, the Proxy-CBA showed increased reliability and a higher degree of validity with similar cognitive load, suggesting its utility as an alternative assessment format. [ABSTRACT FROM AUTHOR]

Computer-based curriculum-based measurement (CBM) is a relatively common practice, but surprisingly few studies have examined the reliability of computer-based CBM. This study sought to examine the reliability of CBM administered via paper/pencil versus the computer. Twenty-one of 25 students in two third-grade classes (N = 21) participated in two generalizability studies. The primary facet of interest, format of assessment, was examined for two measures, a fact operation (fact family measure) and a more complex operation (multidigit addition). Researchers administered four alternate forms of each measure under both computer and paper/pencil conditions. Results indicated ideal reliability of measurement within all conditions and results were consistent across assessment formats for the fact operation. However, assessment format explained a large proportion of variance for the more complex skill (multidigit addition). Results indicate a need to consider the reliability of transitioning assessment from paper/pencil to computer-based response on a measure-by-measure basis. [ABSTRACT FROM AUTHOR]

OBJECTIVE: Objective structured clinical exams (OSCE) are used to train and assess resident foundational family-centered communication skills incorporating key stakeholders. In 2020 encounters were conducted virtually. We sought to compare standardized patient (SP) and family faculty (FF) assessment across OSCE and virtual OSCE (VOSCE) formats. METHODS: The intern classes of 2019 and 2020 completed a live OSCE and VOSCE respectively where they disclosed an error to an SP. The 10 minute encounter was observed by an FF and facilitator followed by a 20 minute debrief. The SP and FF completed a behavioral checklist to evaluate skill mastery. RESULTS: Eighteen (90%) of the 20 interns completed the encounter each year. Total mastery scores were significantly higher for SP assessment than for FF assessment in both OSCE [68% vs 46% (z = -3.005, P < .05)] and VOSCE formats [68% vs 53% (z = -2.105, P < .05)]. Total mastery scores of SPs across OSCE and VOSCE formats did not significantly differ, nor did FF scores based on evaluation format. CONCLUSIONS: Our current experience suggests VOSCEs are a viable alternative to in person sessions given the comparable assessment across the two modalities. The ease of participating in virtual sessions may provide a way to more easily partner with key stakeholders. [ABSTRACT FROM AUTHOR]

Objective: Large language models (LLMs) have demonstrated mixed results in their ability to pass various specialist medical examination and their performance within the field of emergency medicine remains unknown. Methods: We explored the performance of three prevalent LLMs (OpenAI's GPT series, Google's Bard, and Microsoft's Bing Chat) on a practice ACEM primary examination. Results: All LLMs achieved a passing score, with scores with GPT 4.0 outperforming the average candidate. Conclusion: Large language models, by passing the ACEM primary examination, show potential as tools for medical education and practice. However, limitations exist and are discussed. [ABSTRACT FROM AUTHOR]

Written assessments face challenges when administered repeatedly, including resource-intensive item development and the potential for performance improvement secondary to item recall as opposed to understanding. This study examines the efficacy of three-item development techniques in addressing these challenges. Learners at five training programs completed two 60-item repeated assessments. Items from the first test were randomized to one of three treatments for the second assessment: (1) Verbatim repetition, (2) Isomorphic changes, or (3) Total revisions. Primary outcomes were the stability of item psychometrics across test versions and evidence of item recall influencing performance as measured by the rate of items answered correctly and then incorrectly (correct-to-incorrect rate), which suggests guessing. Forty-six learners completed both tests. Item psychometrics were comparable across test versions. Correct-to-incorrect rates differed significantly between groups with the highest guessing rate (lowest recall effect) in the Total Revision group (0.15) and the lowest guessing rate (highest recall effect) in the Verbatim group (0.05), p = 0.01. Isomorphic and total revisions demonstrated superior performance in mitigating the effect of recall on repeated assessments. Given the high costs of total item revisions, there is promise in exploring isomorphic items as an efficient and effective approach to repeated written assessments. Practice points Item psychometrics were comparable across repeated assessments developed with three different techniques: verbatim repetition, isomorphic changes, and total revisions. Isomorphic changes and total item revisions demonstrate superior performance in mitigating the effect of recall on repeated assessments as compared to a verbatim repetition of test items. Given the high costs of total item revisions, there is promise in exploring isomorphic items as an efficient and effective approach to repeated written assessments. [ABSTRACT FROM AUTHOR]

Semantically related concepts are coactivated during spoken word comprehension. Two internet-mediated cursor-tracking experiments examined the spatiotemporal dynamics of this coactivation. Participants viewed visual arrays containing images of a target (e.g., accordion) and a semantically related (e.g., banjo) or unrelated (e.g., plum) distractor whilst hearing the target word (e.g., "accordion"). Participants were tasked with moving their cursor from the bottom of the visual array to the target in one of the upper corners. In contrast to Experiment 1, the onset of stimulus presentation was triggered by cursor movement in Experiment 2. Across both experiments, temporal (e.g., RT) and spatial (e.g., AUC) measures revealed significantly greater attraction to images of semantically related compared with unrelated distractors. These results reveal that online cursor-tracking methods are sensitive to semantic competition and suitable for studying the activation of semantic knowledge during language comprehension. [ABSTRACT FROM AUTHOR]

Aim: The purpose of this study was to explore the perceived benefits and challenges of online distance education and computer‐based testing (CBT) among registered nurses and midwives in a sample of government‐owned health facilities and health training institutions in Ghana. Design: Exploratory descriptive qualitative design. Methods: Individual semi‐structured interviews were conducted with 45 participants (i.e. 25 nurses and 20 midwives). The data were manually processed and analysed using Braun and Clarke's thematic analysis approach (November 2019‐February 2020). Results: Nurses and midwives prefer online distance education for three reasons: convenience, cost‐effectiveness and learning centre proximity to the workplace. The course schedule's flexibility allowed participants to work and study simultaneously. Others acknowledged online distance education as a viable option for overcoming the challenges of obtaining study leave. However, the lack of recognition of certificates by some employers, poor Internet connectivity and perceived excessive course load were noted as deterrents. Regarding the CBT, many of the participants said that it was useful. Among the advantages of CBT are: (1) a decrease in examination malpractices, (2) a decrease in examination costs and (3) a rise in students' interest in information, communication and technology (ICT). This finding emphasizes the necessity of integrating ICT into nursing and midwifery education and examinations, as well as maximizing its benefits. [ABSTRACT FROM AUTHOR]

Background: Online and blended learning need an appropriate assessment strategy which ensures academic integrity. During the pandemic, many universities have chosen for online proctoring. Although some earlier examples suggest that online proctoring may reduce cheating, the potential side‐effects of proctoring are largely unknown. Objectives: Therefore, this study aims to identify the effects of proctoring on students' self‐reported temptation to cheat and potentially undesirable side‐effects, including test anxiety, perceived exam difficulty, and performance. In addition, we examine which contextual and student characteristics affect test anxiety during online exams. Methods: For this, we collected four waves of survey data throughout a year of teaching during the pandemic at one faculty of a Dutch university, resulting in a total sample of 1760 students within 105 courses. Results and Conclusions: Multi‐level analyses showed that while proctoring had no effect on the temptation to cheat, exam difficulty or performance, students reported higher levels of test anxiety. Some learning strategies, internet literacy, access to a reliable technology and a dedicated study space as well as gender and financial stress affect their test anxiety. To conclude, the decision to use online proctoring needs to consider the undesirable side‐effect on test anxiety. We discuss practical implications for university administrators, educational designers and teachers to reduce test anxiety. Lay Description: What is already known about this topic: Proctoring is often used to ensure academic integrity in online exams.Some evidence shows that proctoring reduces cheating.Little is known about the potential negative side‐effects of proctoring. What this paper adds: The use of proctoring has no effect on the temptation to cheat.The use of proctoring results in increased test anxiety.Several student characteristics and contextual factors reduce text anxiety. Implications for practice: Consider potential negative side‐effects when choosing for proctoring.Time management and internet literacy training might reduce test anxiety.Simulating the exam environment might be used to reduce test anxiety. [ABSTRACT FROM AUTHOR]

Introduction: During the COVID‐19 lockdown in 2020, teachers had to shift their teaching and assessment to online. Formative assessment (FA) can help teachers to engage, guide and monitor students' (online) learning. However, more knowledge is needed of how teachers could use the full FA process online. Methods: In this study data from 50 secondary school teachers, who taught different grade levels and subjects and joined a FA learning network that started before and continued during the lockdown, were collected. The study investigates how they used online FA practices differently than face‐to‐face FA, what challenges and opportunities they experienced in online FA and what lessons they learned and intended to keep. This mixed methods study used data from a questionnaire, interviews and webinars that were segmented, coded and analysed. Results: Results showed that many teachers implemented new FA strategies and adopted, more often than in their face‐to‐face practice, all the five phases of the FA process in an aligned matter in online FA. Teachers indicated opportunities in stimulating student engagement and guiding and monitoring student learning more at an individual level in the online FA process, but also experienced challenges, mainly in lack of interaction online. Discussion: The sudden and necessary shift to online FA, due to the COVID‐19 lockdown, challenged teachers to more fundamentally reconsider their assessment practices and assumptions. Teachers intended to make use of these learned lessons to improve their future (blended) FA practice. Lay Description: Known about the subject matter: Online formative assessment (FA) can help teachers to engage, guide and monitor online learning.Most earlier research of online FA looked at parts of the FA process or tools.Teachers' former knowledge and skills in FA influences the online implementation.Online FA offers opportunities to promote learning, but is also challenging. What their paper adds to this: Experienced teachers were able to implement complete FA processes online.Teachers experienced that online FA could stimulate student engagement.Teachers made more time for individual help and feedback during online FA.Teachers intended to maintain these lessons learned in their future (blended) FA practice. The implications of study findings for practitioners: Combining online FA activities with face‐to‐face activities creates opportunities for future blended FA.The five phases of the FA cycle help teachers to implement FA online or in blended forms. [ABSTRACT FROM AUTHOR]

Medical students are faced with many unprecedented challenges, one of which is the large amount of material they are required to learn and retain. While effective learning strategies have been thoroughly researched, stress levels amongst medical students remain very high due to perceived failure to retain material, suggesting that novel methods of implementing such existing strategies are required. Aside from stress levels, additional challenges in medical education include the incorporation of inconsistent testing methods and the challenge of accommodating different learning styles and preferences. A more evidence-based approach that aims to cover many learning styles at once may be desirable. The aim of this commentary is to present some of the current learning and teaching strategies utilized within medical education across the world and to promote a novel supplementary approach to medical education involving a variable ratio incentive-based system of active recall and spaced repetition. This system aims to reward small achievements throughout the semester and complements formal structured examinations in order to motivate students. While this model has yet to be tested, we hope to motivate medical faculty to pilot a program with these evidence-based strategies in mind. [ABSTRACT FROM AUTHOR]

Issue: Technology is pervasive in medicine, but we too rarely examine how it shapes assessment, learning, knowledge, and performance. Cultures of assessment also shape identities, social relations, and the knowledge and behavior recognized as legitimate by a profession. Therefore, the combination of technology and assessment within medical education is worthy of review. Online proctoring services have become more prevalent during the Covid-19 pandemic, as a means of continuing high-stakes invigilated examinations online. With criticisms about increased surveillance, discrimination, and the outsourcing of control to commercial vendors, is this simply "moving exams online", or are there more serious implications? What can this extreme example tell us about how our technologies of assessment influence relationships between trainees and medical education institutions? Evidence: We combine postdigital and postphenomenology approaches to analyze the written component of the 2020 online proctored United Kingdom Royal College of Physicians (MRCP) membership exam. We examine the scripts, norms, and trust relations produced through this example of online proctoring, and then locate them in historical and economic contexts. We find that the proctoring service projects a false objectivity that is undermined by the tight script with which examinees must comply in an intensified norm of surveillance, and by the interpretation of digital data by unseen human proctors. Nonetheless, such proctoring services are promoted by an image of data-driven innovation, a rhetoric of necessity in response to a growing problem of online cheating, and an aversion, within medical education institutions, to changing assessment formats (and thus the need to accept different forms of knowledge as legitimate). Implications: The use of online proctoring technology by medical education institutions intensifies established norms, already present within examinations, of surveillance and distrust. Moreover, it exacerbates tensions between conflicting agendas of commercialization, accountability, and the education of trustworthy professionals. Our analysis provides an example of why it is important to stop and consider the holistic implications of introducing technological "solutions", and to interrogate the intersection of technology and assessment practices in relation to the wider goals of medical education. [ABSTRACT FROM AUTHOR]

The development of technology-enhanced innovative items calls for practical models that can describe polytomous testlet items. In this study, we evaluate four measurement models that can characterize polytomous items administered in testlets: (a) generalized partial credit model (GPCM), (b) testlet-as-a-polytomous-item model (TPIM), (c) random-effect testlet model (RTM), and (d) fixed-effect testlet model (FTM). Using data from GPCM, FTM, and RTM, we examine performance of the scoring models in multiple aspects: relative model fit, absolute item fit, significance of testlet effects, parameter recovery, and classification accuracy. The empirical analysis suggests that relative performance of the models varies substantially depending on the testlet-effect type, effect size, and trait estimator. When testlets had no or fixed effects, GPCM and FTM led to most desirable measurement outcomes. When testlets had random interaction effects, RTM demonstrated best model fit and yet showed substantially different performance in the trait recovery depending on the estimator. In particular, the advantage of RTM as a scoring model was discernable only when there existed strong random effects and the trait levels were estimated with Bayes priors. In other settings, the simpler models (i.e., GPCM, FTM) performed better or comparably. The study also revealed that polytomous scoring of testlet items has limited prospect as a functional scoring method. Based on the outcomes of the empirical evaluation, we provide practical guidelines for choosing a measurement model for polytomous innovative items that are administered in testlets. [ABSTRACT FROM AUTHOR]

Background: Underrepresented minorities in medicine (URMM) may face financial and social limitations when applying to medical schools. The computer-based assessment for sampling personal characteristics (CASPER) test is used by many medical schools to assess the nonacademic competencies of applicants. Performance on CASPER can be enhanced by coaching and mentorship, which URMMs often lack, for affordability reasons, when applying to medical schools. Methods: The CASPER Preparation Program (CPP) is a free, online, 4-week program to help URMM prepare for the CASPER test. CPP features free medical ethics resources, homework and practice tests, and feedback from tutors. Two of CPPs major objectives include relieving URMM of financial burdens and increasing their accessibility to mentorship during the COVID-19 pandemic. A program evaluation was conducted using anonymous, voluntary postprogram questionnaires to assess CPPs efficacy in achieving the aforementioned objectives. Results: Sixty URMMs completed the survey. The majority of the respondents strongly agreed or agreed that CPP relieves students of financial burden (97%), is beneficial for applicants with low-socioeconomic statuses (98%), provides students with resources they could not afford (n = 55; 92%), and enables access to mentors during the pandemic (90%). Discussion: Pathway coaching programs, such as the CASPER Preparation Program, have the potential to offer URMMs mentorship and financial relief, and increase their confidence and familiarity with standardized admission tests to help them matriculate into medical schools. [ABSTRACT FROM AUTHOR]

The multiple response structure can underlie several different technology-enhanced item types. With the increased use of computer-based testing, multiple response items are becoming more common. This response type holds the potential for being scored polytomously for partial credit. However, there are several possible methods for computing raw scores. This research will evaluate several approaches found in the literature using an approach that evaluates how the inclusion of scoring related to the selection/nonselection of both relevant and irrelevant information is incorporated extending Wilson's approach. Results indicated all methods have potential, but the plus/minus and true/false methods seemed the most promising for items using the "select all that apply" instruction set. Additionally, these methods showed a large increase in information per time unit over the dichotomous method. [ABSTRACT FROM AUTHOR]

In low-stakes assessments, some students may not reach the end of the test and leave some items unanswered due to various reasons (e.g., lack of test-taking motivation, poor time management, and test speededness). Not-reached items are often treated as incorrect or not-administered in the scoring process. However, when the proportion of not-reached items is high, these traditional approaches may yield biased scores and thereby threatening the validity of test results. In this study, we propose a polytomous scoring approach for handling not-reached items and compare its performance with those of the traditional scoring approaches. Real data from a low-stakes math assessment administered to second and third graders were used. The assessment consisted of 40 short-answer items focusing on addition and subtraction. The students were instructed to answer as many items as possible within 5 minutes. Using the traditional scoring approaches, students' responses for not-reached items were treated as either not-administered or incorrect in the scoring process. With the proposed scoring approach, students' nonmissing responses were scored polytomously based on how accurately and rapidly they responded to the items to reduce the impact of not-reached items on ability estimation. The traditional and polytomous scoring approaches were compared based on several evaluation criteria, such as model fit indices, test information function, and bias. The results indicated that the polytomous scoring approaches outperformed the traditional approaches. The complete case simulation corroborated our empirical findings that the scoring approach in which nonmissing items were scored polytomously and not-reached items were considered not-administered performed the best. Implications of the polytomous scoring approach for low-stakes assessments were discussed. [ABSTRACT FROM AUTHOR]

The response process of problem‐solving items contains rich information about respondents' behaviours and cognitive process in the digital tasks, while the information extraction is a big challenge. The aim of the study is to use a data‐driven approach to explore the latent states and state transitions underlying problem‐solving process to reflect test‐takers' behavioural patterns, and to investigate how these states and state transitions could be associated with test‐takers' performance. We employed the Hidden Markov Modelling approach to identify test takers' hidden states during the problem‐solving process and compared the frequency of states and/or state transitions between different performance groups. We conducted comparable studies in two problem‐solving items with a focus on the US sample that was collected in PIAAC 2012, and examined the correlation between those frequencies from two items. Latent states and transitions between them underlying the problem‐solving process were identified and found significantly different by performance groups. The groups with correct responses in both items were found more engaged in tasks and more often to use efficient tools to solve problems, while the group with incorrect responses was found more likely to use shorter action sequences and exhibit hesitative behaviours. Consistent behavioural patterns were identified across items. This study demonstrates the value of data‐driven based HMM approach to better understand respondents' behavioural patterns and cognitive transmissions underneath the observable action sequences in complex problem‐solving tasks. Lay Description: What is already known about this topic?: Process data of interactive problem‐solving tasks contains rich information about test‐takers.To extract underlying information from process data is a big challenge. ​ What this paper adds?: The hidden Markov model helps identify latent states underlying problem‐solving process.Different performance groups displayed different behaviour characteristics.A consistent behavioural pattern was observed across problem‐solving items. ​ Implications for practitioners: The use of HMM holds promise in better visualize and understand behaviour patterns in response process.HMM is potential in understanding cognitive transmissions underneath the observable action sequences. ​ [ABSTRACT FROM AUTHOR]

Simulation studies involving mixture models inevitably aggregate parameter estimates and other output across numerous replications. A primary issue that arises in these methodological investigations is label switching. The current study compares several label switching corrections that are commonly used when dealing with mixture models. A growth mixture model is used in this simulation study, and the design crosses three manipulated variables—number of latent classes, latent class probabilities, and class separation, yielding a total of 18 conditions. Within each of these conditions, the accuracy of a priori identifiability constraints, a priori training of the algorithm, and four post hoc algorithms developed by Tueller et al.; Cho; Stephens; and Rodriguez and Walker are tested to determine their classification accuracy. Findings reveal that, of all a priori methods, training of the algorithm leads to the most accurate classification under all conditions. In a case where an a priori algorithm is not selected, Rodriguez and Walker's algorithm is an excellent choice if interested specifically in aggregating class output without consideration as to whether the classes are accurately ordered. Using any of the post hoc algorithms tested yields improvement over baseline accuracy and is most effective under two-class models when class separation is high. This study found that if the class constraint algorithm was used a priori, it should be combined with a post hoc algorithm for accurate classification. [ABSTRACT FROM AUTHOR]

Background: Existing literature identifies a general positive benefit to students' examination scores when students change their answer on examinations. Current shifts toward computerized examinations and in test anxiety warrant a literature update on test-taking behaviors. Purpose: This study assessed answer changing on multiple-choice examinations and investigated faculty recommendations related to answer changing. Method: This study used a mixed-methods design using retrospective student data from ExamSoft reports and faculty responses from a national survey of nurse educators. Results: Findings indicated 55% of students improved their examination score, with an average improvement of 3%, and 24% of students lowered their examination score. However, faculty continue to incorrectly recommend that students should not change answers on examinations. Conclusion: The results of this study support prior literature that answer changing benefits students. However, data are lacking on the characteristics of students who improve or lower their examination scores by changing answers.[J Nurs Educ. 2021;60(6):324–328.]. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Medical Imaging & Radiation Sciences is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Purpose: In this study, we assessed whether there are independent associations between dietary total flavonoid intake and major flavonoid classes with retinal arteriolar and venular calibre. Methods: Blue Mountains Eye Study participants aged 49+ years who had complete data on diet and retinal vessel measures were analysed (n = 2821). Dietary intake was assessed using a semi-quantitative Food-Frequency Questionnaire (FFQ). Flavonoid content of foods in the FFQ was estimated using the US Department of Agriculture Flavonoid, Isoflavone and Proanthocyanidin databases. Fundus photographs were taken and retinal vascular calibre was measured using validated computer-assisted techniques. The associations of intake of dietary flavonoids with retinal vessel calibre were examined in linear regression models and general linear model. Results: The highest quartile of intake was compared with the lowest quartile using multivariable-adjustment models. Participants with the highest proanthocyanidin intake had narrower retinal venules (223.9 ± 0.62 versus 226.5 ± 0.63, respectively; Ptrend = 0.01); and the highest isoflavone intake was associated with wider retinal arterioles (188.1 ± 0.55 versus 186.3 ± 0.56, respectively; Ptrend = 0.01). The highest apple/pear consumption (a dietary source of catechin) was associated with narrower retinal venules (223.8 ± 0.57 versus 226.1 ± 0.52; Ptrend = 0.01) and wider retinal arterioles (187.9 ± 0.51 versus 186.2 ± 0.51; Ptrend = 0.02). Further, participants who were in the highest versus lowest quartile of chocolate consumption had ~ 2.1 μm narrower retinal venules (multivariable-adjusted P = 0.03). Conclusions: This study shows that higher intakes of specific flavonoid subclasses are associated with a favourable retinal microvascular profile. Greater consumption of flavonoid-rich apples/pears and chocolate was also associated with beneficial variations in retinal vascular calibre. [ABSTRACT FROM AUTHOR]

Phenomenon: Detection of visual and auditory clinical findings is part of medical students' core clinical performance abilities that a medical education curriculum should teach, assess, and remediate. However, there is a limited understanding of how students develop these skills. While training physical exam technical skills has received significant attention and emphasis, teaching and assessing medical students' ability to detect and interpret visual and auditory clinical findings skills has been less systematic. Therefore, the purpose of this study is to investigate how medical students' visual and auditory clinical findings skills progress and develop over their four years of undergraduate medical education. This study will provide educators insights that can guide curriculum refinements that lead to improving students' abilities in this area. Approach: A computer-based progress exam was created to measure the longitudinal development of students' abilities to detect and interpret visual and auditory findings. After pilot testing, sixty test items were developed in collaboration with six clinical faculty members and two medical education researchers. The exam includes detection and description of ECG, x-ray, heart sounds, breath sounds, skin lesions, and movement findings. The exam was administered to students at the beginning of each training year since 2014. Additionally, the exam was administered to the Class of 2017 prior to their graduation. Measurement validity and reliability tests were conducted. Descriptive statistics and ANOVA were used to determine progress. Findings: More than 98% of students in four years of training completed the exam each year. The exam instrument had high reliabilities and demonstrated acceptable concurrent validity when compared with other academic performance data. Findings showed that students' visual and auditory clinical findings skills increased each training year until their fourth year. There was no performance improvement between incoming Year 4 students and graduating Year 4 students. While group means increased, class performance did not become more homogeneous across four years. Longitudinal data showed the same performance patterns as the cross-sectional data. Performance of the bottom quartile of graduating fourth-year students was not significantly higher than the performance of the top quartile of incoming first-year students who had not had formal medical training. Insights: A longitudinal study to follow learners' performance in detecting and interpreting visual and auditory clinical findings can provide meaningful insights regarding the effects of medical training programs on performance growth. The present study suggests that our medical curriculum is not effective in bringing all students to a higher level of performance in detecting and interpreting visual and auditory clinical findings. This study calls for further investigation how medical students can develop visual and auditory detection and interpretation skills in undergraduate medical education. There is a need for planned curriculum and assessment of medical students' skills in detecting and interpreting visual and auditory clinical findings. [ABSTRACT FROM AUTHOR]

Objective: The aim of the study was to evaluate a longitudinal assessment program for physical medicine and rehabilitation (LA-PM&R) as a replacement for the American Board of Physical Medicine and Rehabilitation maintenance of certification examination. Design: In this quality improvement study, randomly selected American Board of Physical Medicine and Rehabilitation diplomates were invited to participate in the LA-PM&R. Participants' maintenance of certification scaled scores were compared with LA-PM&R nonparticipants. We examined the association between LA-PM&R scores and maintenance of certification scaled scores and performance on clone items placed on both examinations. Results: There were 2222 American Board of Physical Medicine and Rehabilitation diplomates eligible to take the maintenance of certification examination from February 2018 to February 2019. Invitations were sent to 1050 randomly selected diplomates, of whom 448 initially agreed. Three hundred eighteen individuals completed more than two quarters of LA-PM&R participation and took the maintenance of certification examination. Two hundred sixty-six diplomates took the maintenance of certification examination in the same period who did not participate in LA-PM&R. The LA-PM&R group scored higher on the maintenance of certification examination than the control group (P < 0.05). Performance on the two examinations were highly correlated, r =.50, P < 0.0001. On clone items, LA-PM&R participants had 74% correct on the LA-PM&R but 86% correct on the maintenance of certification examination (P < 0.01). Conclusions: The LA-PM&R program leads to better learning and retention of information than the 10-yr maintenance of certification examination. [ABSTRACT FROM AUTHOR]

This article outlines the development and validation of the Computer-Delivered Test (CDT) Acceptance Questionnaire (CTAQ). The CTAQ was designed to be a practical measure of CDT acceptance of Singapore secondary and high school students (Grades 7-12) toward taking tests within an e-assessment system. The stages of test (questionnaire item) content, response processes, and internal structure under Messick's unified concept of validity suggested that the CTAQ had sound psychometric properties. Exploratory factor analysis (EFA; n = 485) and confirmatory factor analysis (CFA; n = 484) yielded a three-factor model (ease of use, involvement and experience) with a reduction of 21 to 13 items for CDT acceptance. Practical applications and limitations of the CTAQ are discussed. [ABSTRACT FROM AUTHOR]

AIM The purpose of this study was to analyze uses of the Graduate Record Examinations® (GRE) General Test in doctoral nursing education. BACKGROUND Doctoral study is a costly (time, money, resources, effort) investment for both students and institutions alike, and it is imperative students who are likely to succeed are admitted. Admission criteria such as the GRE General Test must be examined for appropriate use and necessity. METHOD Literature regarding uses of the GRE General Test in doctoral nursing education was analyzed versus GRE General Test publisher evidence-based uses and guidelines while considering test psychometrics. RESULTS Although the GRE General Test is widely used in doctoral nursing program admissions, it appears it is often not used according to evidence-based guidelines or uses. Notably, crucial self-validations are infrequent. CONCLUSION Self-validations are essential when determining if doctoral nursing programs should start, stop, continue, or modify their use of the GRE General Test. Admission criteria analyses may help streamline admission processes and remove some unnecessary costs in doctoral education. [ABSTRACT FROM AUTHOR]

Although the use of technology in the K12 classroom has been shown to have a positive impact, research on the use of open education resources (OER) is relatively limited, especially research focusing on low‐achieving students. The present study examines the relationship between usage of Algebra Nation, a self‐guided system that provided instructional videos and practice problems, and the performance of students who had failed the state‐administered Algebra I end‐of‐course (EOC) assessment the previous year. Indicators of usage of Algebra Nation consisted of logins, video views, and practice questions answered. Path analyses and logistic regressions were used to evaluate relationships between usage indicators and algebra scores, controlling for number of absences, free/reduced lunch eligibility, Hispanic/Latino origin, race, and gender. The results indicate that higher levels of logins, video views, and practice questions answered were related to higher scores when the students re‐took the assessment. Logins and practice questions were also related to increases in odds of passing the Algebra I EOC assessment, but not video views. The results suggest that there may be benefits to technology use in the form of an OER adopted by students and teachers on an informal basis and link self‐regulated learning strategies to student achievement. Lay Description: What is already known about this topic: Algebra courses typically have a high failure rate.Repeating algebra is costly for the education system and for the student.Open education resources could potentially be an effective and cost‐efficient approach to algebra instruction. What this paper adds: This study shows that logins, video views, and practice questions answered in an open education resource were positively related to scores on a state‐administered,high‐stakes algebra assessment.The study indicates that the relationship between a self‐regulated learning strategy (i.e., practice questions answered) and student algebra scores is stronger than that between video views and algebra scores.This study shows that student‐driven ad hoc use of online tutoring programme is associated with higher passing rates on the algebra exam. Implications for practice and/or policy: This study demonstrates that informal use of an open education resource with videos and a practice question system can be effective in increasing student scores on algebra assessments.Future research is needed to connect diverse types of student self‐regulated use of open education resources to academic achievement and on how to design resources that encourage effective self‐regulation. [ABSTRACT FROM AUTHOR]