Search

The coronavirus disease (Covid-19) pandemic raised challenges for everyday life. Development of new diagnostic tests was necessary, but under such enormous pressure risking inadequate evaluation. Against a background of concern about standards applied to the evaluation of in vitro diagnostic tests (IVDs), clear statistical thinking was needed on the principles of diagnostic testing in general, and their application in a pandemic. Therefore, in July 2020, the Royal Statistical Society convened a Working Group of six biostatisticians to review the statistical evidence needed to ensure the performance of new tests, especially IVDs for infectious diseases—for regulators, decision-makers, and the public. The Working Group's review was undertaken when the Covid-19 pandemic shone an unforgiving light on current processes for evaluating and regulating IVDs for infectious diseases. The report's findings apply more broadly than to the pandemic and IVDs, to diagnostic test evaluations in general. A section of the report focussed on lessons learned during the pandemic and aimed to contribute to the UK Covid-19 Inquiry's examination of the response to, and impact of, the Covid-19 pandemic to learn lessons for the future. The review made 22 recommendations on what matters for study design, transparency, and regulation. [ABSTRACT FROM AUTHOR]

Evaluating text-based answers obtained in educational settings or behavioral studies is time-consuming and resource-intensive. Applying novel artificial intelligence tools such as ChatGPT might support the process. Still, currently available implementations do not allow for automated and case-specific evaluations of large numbers of student answers. To counter this limitation, we developed a flexible software and user-friendly web application that enables researchers and educators to use cutting-edge artificial intelligence technologies by providing an interface that combines large language models with options to specify questions of interest, sample solutions, and evaluation instructions for automated answer scoring. We validated the method in an empirical study and found the software with expert ratings to have high reliability. Hence, the present software constitutes a valuable tool to facilitate and enhance text-based answer evaluation. • Generative AI-enhanced software for customizable, case-specific, and automized grading of large amounts of text-based answers. • Open-source software and web application for direct implementation and adaptation.

For centuries, chemists have researched acid-base reactions and developing numerous different acid-base concepts. Consequentially, it is difficult to teach about acid-base reactions in a clear and conceptually coherent manner. We aim at contributing to teaching about acid-base chemistry by an intervention introducing upper secondary school chemistry students to an acid-base concept that is scientifically appropriate and learner-centred. This modernized version of the Brønsted-Lowry acid-base concept appears sufficiently complex for the target group. Additionally, it emphasizes the donor-acceptor concept and the reaction mechanism. In order to evaluate the gain in declarative knowledge during our intervention, we developed and piloted a multiple-choice knowledge test with Austrian upper secondary students (N = 136). By estimating a Rasch model (WLE-Rel. = 0.63, Infitmin = 0.90, Infitmax = 1.02, Outfitmin = 0.89, Outfitmax = 1.00, d = −0.44–1.8), we constructed a pre-test (11 MC items, 6 open-ended questions) and a post-test (11 MC items, 6 open-ended questions) based on key concepts of the topic for our intervention study (N = 57). Overall, the test proved a reliable tool to estimate learners’ declarative knowledge about acid-base reactions in the course of the intervention study, showing a significant knowledge increase.

Precipitation datasets derived from reanalysis products play a crucial role in weather forecasting and hydrological applications. This paper aims to evaluate the performance of two distinct reanalysis precipitation products, i.e., the first-generation Chinese global land-surface reanalysis precipitation product (CRA40) and the fifth-generation European reanalysis precipitation product (ERA5), over mainland China. The evaluation is based on continuous and categorical statistical indicators with daily-scale gridded-point rain gauge data obtained from Chinese surface meteorological stations. The findings of this study can be summarized as follows: (1) CRA40 demonstrates a clear superiority over ERA5 in terms of the 13-year daily mean precipitation and seasonal daily precipitation. CRA40 exhibits better correlation coefficients (0.97), relative biases (5.25%), root mean square errors (0.34 mm), and fractional standard errors (0.05). (2) Both reanalyzed precipitation products generally exhibit an overestimation of precipitation in mainland China. The degree of overestimation is particularly pronounced in dry climatic regions (e.g., QZ, XJ), while wet regions (e.g., CJ, HN) demonstrate relatively less overestimation. (3) ERA5 shows better performance in the detection of daily precipitation. Neither CRA40 nor ERA5 can capture heavy precipitation events well. These findings are expected to advance our understanding of the strengths and limitations of the reanalysis precipitation products, CRA40 and ERA5, over China. [ABSTRACT FROM AUTHOR]

Copyright of Lernen und Lernstörungen is the property of Hogrefe AG and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Fire Safety Science is the property of Fire Safety Science Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Precipitation datasets derived from reanalysis products play a crucial role in weather forecasting and hydrological applications. This paper aims to evaluate the performance of two distinct reanalysis precipitation products, i.e., the first-generation Chinese global land-surface reanalysis precipitation product (CRA40) and the fifth-generation European reanalysis precipitation product (ERA5), over mainland China. The evaluation is based on continuous and categorical statistical indicators with daily-scale gridded-point rain gauge data obtained from Chinese surface meteorological stations. The findings of this study can be summarized as follows: (1) CRA40 demonstrates a clear superiority over ERA5 in terms of the 13-year daily mean precipitation and seasonal daily precipitation. CRA40 exhibits better correlation coefficients (0.97), relative biases (5.25%), root mean square errors (0.34 mm), and fractional standard errors (0.05). (2) Both reanalyzed precipitation products generally exhibit an overestimation of precipitation in mainland China. The degree of overestimation is particularly pronounced in dry climatic regions (e.g., QZ, XJ), while wet regions (e.g., CJ, HN) demonstrate relatively less overestimation. (3) ERA5 shows better performance in the detection of daily precipitation. Neither CRA40 nor ERA5 can capture heavy precipitation events well. These findings are expected to advance our understanding of the strengths and limitations of the reanalysis precipitation products, CRA40 and ERA5, over China.

Abstract Background This study aimed to screen the sera of goats and sheep from flocks suspected of Mycobacterium avium subsp. paratuberculosis (MAP) infection by a newly standardized Mce-truncated ELISA (Mt-ELISA) kit for the detection of antibodies against MAP. Four diagnostic applied tests were evaluated including Indigenous plate-ELISA (IP-ELISA), Mt-ELISA, fecal Polymerase Chain Reaction (f-PCR) and fecal culture (FC). Materials and methods Assuming the absence of a gold standard, latent-class models in a Bayesian framework were used to estimate the diagnostic accuracy of the four tests for MAP. Results Mt-ELISA had higher Sensitivity (Se) in sheep (posterior median: 0.68 (95% Probability Interval (PI): 0.43–0.95), while IP-ELISA recorded the highest Se in goats as 0.83 (95% PI, 0.61–0.97). The f-PCR Se estimate slightly differed between species [sheep 0.36 (0.19–0.58), goats 0.19 (0.08–0.35)], while the Se of FC was similar between species [sheep 0.29 (0.15–0.51), goats 0.27 (0.13–0.45)]. The specificity estimates for all tests were high, close to unity, and similar between species. Conclusion Overall, the results showed that the Mt-ELISA method can be used for MAP detection in small ruminants’ flocks.

As one of the most significant protein post-translational modifications (PTMs) in eukaryotes, ubiquitylation plays an essential role in regulating diverse cellular functions, such as apoptosis, cell division, DNA repair and replication, intracellular transport and immune reactions. Traditional experimental methods have the defect of being time-consuming, costly and labor-intensive. Therefore, it is highly desired to develop automated computational methods that can recognize potential ubiquitylation sites rapidly and accurately. In this study, we propose a novel predictor, named UPFPSR, for predicting lysine ubiquitylation sites in plant. UPFPSR is developed using multiple physicochemical properties of amino acids and sequence-based statistical information. In order to find a suitable classification algorithm, four traditional algorithms and two deep learning networks are compared, and the random forest with superior performance is selected ultimately. An extensive benchmarking shows that UPFPSR outperforms the most advanced ubiquitylation prediction tool on each measurement indicator, with the accuracy of 77.3%, precision of 75%, recall of 81.7%, F1-score of 0.7824, and AUC of 0.84 on the independent test dataset. The results indicate that UPFPSR can provide new guidance for further experimental study on ubiquitylation. The data sets and source code used in this study are freely available at https://github.com/ysw-sunshine/UPFPSR.

Since the 1980's, scientific interest in applicants' reactions to admission procedures has been growing. Several theoretical frameworks and evaluation questionnaires were developed. Many researchers have asked potential participants about their attitudes—what is missing, however, are studies of applicants who have actually taken part in the selection procedures. In addition, applicants' reactions to student selection procedures receive less attention than applicants' reactions to personnel selection procedures. Furthermore, tests and testing conditions continue to develop, e.g., through online-based testing and test supervision at home (“proctoring”). Therefore, we used a standardized questionnaire for measuring the overall test evaluation and different dimensions of acceptance including face validity, controllability, and the absence of strain to examine six scholastic aptitude tests. We added items to get deeper insights into situational aspects. According to the results of 2,052 test participants, applicants prefer specific tests and shorter tests. Situational aspects, such as privacy, working conditions, and prior information, have an influence on the perceptions of applicants. Proctored testing is evaluated positively, but taking the test at a test center is still rated more favorably. This study discusses the practical implications.

ABSTRACT Bacteria can be associated with male infertility. Antibacterial substances (e.g., zinc-containing proteins, antimicrobial peptides) in ejaculates might impair the growth of bacteria in culture. We therefore wanted to test if removing antibacterial substances by washing the ejaculate could improve the detection of bacteria in culture. All ejaculates from patients ≥18 years old, which were obtained for routine diagnostics to assess male infertility were included in this study (no exclusion criteria were applied). Test samples were diluted with 2 mL sterile 0.45% saline, vortexed, and centrifuged (5 min; 7.5 × g). After the removal of 2 mL of the supernatant and resuspension, 10 μL of the pellet was used for aerobic and anaerobic culture. Control samples were cultured identically but without washing. Species identification was done with matrix-assisted laser desorption ionization–time of flight mass spectrometry. A total of 186 samples were included. The data set was stratified into five groups (Gram-negative rods [GNR], anaerobes [AN], Enterococcus spp. [EC], coagulase-negative staphylococci [CNS], and viridans streptococci [VS]). Compared to the control arm, the test arm revealed significant lower proportions for CNS (59.1% versus 44.6%, P < 0.01) and VS (53.8% versus 41.9%, P = 0.03). Similarly, slightly lower proportions of GNR (16.1% versus 15.1%, P = 0.89), AN (19.9% versus 17.2%, P = 0.5), and EC (25.3% versus 23.1%, P = 0.63) were observed. The medians of CFU were lower in test samples compared to the control samples (6.5 × 103 versus 2.5 × 103, P < 0.01) for any bacterial growth. Lower colony counts were also observed for individual bacterial groups. In conclusion, preculture washing of ejaculates results in a decrease in total bacteria count and culture-positive samples. IMPORTANCE This study compares two methods for processing ejaculate samples from men undergoing investigations for infertility. The method of sample washing and centrifugation was compared to the standard method of direct inoculation and culture. The study hypothesis was that preprocessing of samples may increase bacterial yield by removing bactericidal substances from semen. However, we found that washing ejaculate samples before microbiological culture did not improve the detection of bacteria and led to a reduction in colony counts.

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Prior to the implementation of new diagnostic techniques, a thorough evaluation is mandatory in order to ensure diagnostic reliability. If positive samples are scarcely available, however, such evaluations can be difficult to perform. Here, we evaluated four SeeGene Allplex real-time PCR assays amplifying a total of 28 bacteria, microsporidal and parasitic nucleic acid sequence targets in human stool samples in a multicentric approach. In the assessments with strongly positive samples, sensitivity values ranging between 13% and 100% were recorded for bacteria, between 0% and 100% for protozoa and between 7% and 100% for helminths and microsporidia; for the weakly positive samples, the recorded sensitivity values for bacteria ranged from 0% to 100%; for protozoa, from 0% to 40%; and for helminths and microsporidia, from 0% to 53%. For bacteria, the recorded specificity was in the range between 87% and 100%, while a specificity of 100% was recorded for all assessed PCRs targeting parasites and microsporidia. The intra- and inter-assay variations were generally low. Specifically for some helminth species, the sensitivity could be drastically increased by applying manual nucleic acid extraction instead of the manufacturer-recommended automatic procedure, while such effects were less obvious for the bacteria and protozoa. In summary, the testing with the chosen positive control samples showed varying degrees of discordance between the evaluated Allplex assays and the applied in-house reference assays associated with higher cycle threshold values in the Allplex assays, suggesting that samples with very low pathogen densities might be missed. As the targeted species can occur as harmless colonizers in the gut of individuals in high-endemicity settings as well, future studies should aim at assessing the clinical relevance of the latter hint. [ABSTRACT FROM AUTHOR]

Developing scientific evidence showing the impact of intimate partner violence (IPV) on companies' productivity is an effective way to involve them in IPV prevention. However, there are no suitable and brief self-report instruments available that measure this impact on labor settings. This study develops and assesses the measurement properties of lost days of labor productivity scale based on tardiness, absenteeism, and presenteeism which may be due to IPV. Fourteen items have been developed and tested for 2,017 employees in 306 companies in Ghana, Pakistan, and South Sudan. Descriptive statistics, confirmatory factor analysis, heterotrait–monotrait matrix, and reliability coefficients have been conducted to assess the reliability of the scores. Confirmatory factor analysis indicates a two-factor second-order solution, stable by sex and countries. All subscales demonstrate good reliability, construct and discriminant validity, showing that the scale is a valid and reliable self-report questionnaire, which may measure the impact of IPV on businesses. [ABSTRACT FROM AUTHOR]

Background: This study aimed to screen the sera of goats and sheep from flocks suspected of Mycobacterium avium subsp. paratuberculosis (MAP) infection by a newly standardized Mce-truncated ELISA (Mt-ELISA) kit for the detection of antibodies against MAP. Four diagnostic applied tests were evaluated including Indigenous plate-ELISA (IP-ELISA), Mt-ELISA, fecal Polymerase Chain Reaction (f-PCR) and fecal culture (FC). Materials and methods: Assuming the absence of a gold standard, latent-class models in a Bayesian framework were used to estimate the diagnostic accuracy of the four tests for MAP. Results: Mt-ELISA had higher Sensitivity (Se) in sheep (posterior median: 0.68 (95% Probability Interval (PI): 0.43–0.95), while IP-ELISA recorded the highest Se in goats as 0.83 (95% PI, 0.61–0.97). The f-PCR Se estimate slightly differed between species [sheep 0.36 (0.19–0.58), goats 0.19 (0.08–0.35)], while the Se of FC was similar between species [sheep 0.29 (0.15–0.51), goats 0.27 (0.13–0.45)]. The specificity estimates for all tests were high, close to unity, and similar between species. Conclusion: Overall, the results showed that the Mt-ELISA method can be used for MAP detection in small ruminants' flocks. [ABSTRACT FROM AUTHOR]

Abstract Introduction Test evaluation trials present different challenges for trial managers compared to intervention trials. There has been very little research on the management of test evaluation trials and how this impacts on trial success, in comparison with intervention trials. Evaluations of medical tests present specific challenges, because they are a pivot point bridging the complexities of pathways prompting testing with treatment decision-making. We systematically explored key differences in the trial design and management of test evaluation trials compared to intervention trials at the different stages of study design and delivery. We identified challenges in test evaluation trials that were more pronounced than in intervention trials, based on experience from 10 test evaluation trials. Methods We formed a focus group of 7 trial managers and a statistician who had been involved in the day-to-day management of both test evaluation trials and intervention trials. We used discussion and content analysis to group challenges from 10 trials into a structured thematic format. The trials covered a range of medical conditions, diagnostic tests, clinical pathways and conditions including chronic kidney disease, chronic pelvic pain, colitis, detrusor over-activity, group B streptococcal colonisation, tuberculosis and colorectal, lung, ovarian and thyroid cancers. Results We identified 10 common themes underlying challenges that are more pronounced in test evaluation compared to intervention trials. We illustrate these themes with examples from 10 trials, including with 31 specific challenges we experienced. The themes were ethics/governance; accessing patient populations; recruitment; patient preference; test processes, clinical pathways and samples storage; uncertainty of diagnostic results; verifying diagnosis (reference standard); follow-up; adverse effects; and diagnostic impact. Conclusion We present 10 common themes, including 31 challenges, in test evaluation trials that will be helpful to others designing and managing future test evaluation trials. Proactive identification of potential challenges at the design and planning stages of test evaluation trials will enable strategies to improve trial design and management that may be different from standard strategies used for intervention trials. Future work could extend this topic to include challenges for other trial stakeholders including participants, clinicians, statisticians and funders. Trial registration All trials reviewed in this project were registered and are provided in Table 1.

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

On-farm culture (OFC) systems facilitate pathogen-based mastitis management and can facilitate antimicrobial stewardship on dairy farms. Interpretation of the results, however, may present a challenge for those with limited microbiology experience. Here, we compared results of 3 OFC systems interpreted by trained and untrained observers against results of a standard laboratory reference method (aerobic culture and mass spectrometry). Milk samples (280 quarter and 60 composite) were selected from submissions for routine diagnostic testing to Quality Milk Production Services (Cornell University, Ithaca, NY) between August 2017 and January 2018. Samples were cultured simultaneously using the standard laboratory reference method and 3 commercially available OFC systems that varied in detail of pathogen identification (provided in parentheses) as follows: (1) Minnesota Easy Culture System II Bi-plate (University of Minnesota Laboratory for Udder Health, St. Paul; gram-positive, gram-negative), (2) Minnesota Easy Culture System II Tri-plate (gram-positive, gram-negative, some genus level), and (3) FERA Diagnostics and Biologicals AccuMast plate (Ithaca, NY; genus level, some species level). After 18 to 24 h of incubation, OFC plates were interpreted by 1 trained observer (>10 yr of experience in milk microbiology) and 6 untrained observers with no previous milk microbiology training, using only the manufacturers' instructions for guidance. Strength of agreement (κ) between observer groups and the reference method was determined for the available outcomes of each system. Interpreted by the trained observer, agreement was moderate for identifying gram-positive organisms (Bi-plate, κ = 0.56) and substantial for Streptococcus spp. (Tri-plate, κ = 0.64, AccuMast κ = 0.61). Interpretation by untrained observers resulted in fair agreement (κ = 0.29–0.37) for these organisms. Moderate agreement (κ = 0.43–0.59) was found across all 3 OFC for the identification of gram-negative organisms (Bi-plate), non- aureus staphylococci (Tri-plate and AccuMast), Lactococcus spp., and Enterococcus spp. (AccuMast) when interpreted by the trained observer, and fair to moderate agreement was found (κ = 0.31–0.53) among untrained observers. Across all 3 OFC, agreement was almost perfect (κ = 0.80–0.89) for Staphylococcus aureus for the trained observer, and moderate to substantial (κ = 0.56–0.61) for untrained observers. We concluded that all 3 OFC appeared suitable to support pathogen-based mastitis management when operated by trained observers. Training beyond the instruction manual is a prerequisite to make OFC systems useful for pathogen-based mastitis management. [ABSTRACT FROM AUTHOR]

Prior to the implementation of new diagnostic techniques, a thorough evaluation is mandatory in order to ensure diagnostic reliability. If positive samples are scarcely available, however, such evaluations can be difficult to perform. Here, we evaluated four SeeGene Allplex real-time PCR assays amplifying a total of 28 bacteria, microsporidal and parasitic nucleic acid sequence targets in human stool samples in a multicentric approach. In the assessments with strongly positive samples, sensitivity values ranging between 13% and 100% were recorded for bacteria, between 0% and 100% for protozoa and between 7% and 100% for helminths and microsporidia; for the weakly positive samples, the recorded sensitivity values for bacteria ranged from 0% to 100%; for protozoa, from 0% to 40%; and for helminths and microsporidia, from 0% to 53%. For bacteria, the recorded specificity was in the range between 87% and 100%, while a specificity of 100% was recorded for all assessed PCRs targeting parasites and microsporidia. The intra- and inter-assay variations were generally low. Specifically for some helminth species, the sensitivity could be drastically increased by applying manual nucleic acid extraction instead of the manufacturer-recommended automatic procedure, while such effects were less obvious for the bacteria and protozoa. In summary, the testing with the chosen positive control samples showed varying degrees of discordance between the evaluated Allplex assays and the applied in-house reference assays associated with higher cycle threshold values in the Allplex assays, suggesting that samples with very low pathogen densities might be missed. As the targeted species can occur as harmless colonizers in the gut of individuals in high-endemicity settings as well, future studies should aim at assessing the clinical relevance of the latter hint.

Unmanned swarm test evaluation is an important bridge that promotes unmanned swarm technology from theory to practice. Compared with traditional combat systems, unmanned swarm systems have the characteristics of autonomous coordination, emergence of capabilities, and intelligent countermeasures. These have greatly exceeded the scope of traditional combat system test evaluation. In response to the abovementioned practical needs， this paper summarizes the groundbreaking work of advancing swarm test evaluation in the United States and China, and discusses the concept and development status of unmanned swarm test evaluation. Based on our previous research on aviation swarm operations, we also explore the main content and feasible technical route of unmanned swarm test evaluation, which will lay the foundation for the information upgrade and intelligent transformation of military equipment. [ABSTRACT FROM AUTHOR]

Quantum key distribution (QKD) based quantum secure communication (QSC) is one of the promising solutions for improving information security.With the development of experimental network construction and demonstrational application,the demand of QSC test evaluation and standardization is becoming more urgent.Based on the introduction of technology and architecture of QSC system,the latest progress of QSC test evaluation and standardization was reviewed,typical questions in test evaluation and standardization were analyzed,and development trends and prospects were also discussed.

In resource-limited settings, where pneumocystosis in immunocompromised patients is infrequently observed, cost-efficient, reliable, and sensitive approaches for the diagnostic identification of Pneumocystis jirovecii in human tissue samples are desirable. Here, an in-house fluorescence in situ hybridization assay was comparatively evaluated against Grocott’s staining as a reference standard with 30 paraffin-embedded tissue samples as well as against in-house real-time PCR with 30 respiratory secretions from immunocompromised patients with clinical suspicion of pneumocystosis. All pneumocystosis patients included in the study suffered from HIV/AIDS. Compared with Grocott’s staining as the reference standard, sensitivity of the FISH assay was 100% (13/13), specificity was 41% (7/17), and the overall concordance was 66.7% with tissue samples. With respiratory specimens, sensitivity was 83.3% (10/12), specificity was 100% (18/18), and the overall concordance was 93.3% as compared with real-time PCR. It remained unresolved to which proportions sensitivity limitations of Grocott’s staining or autofluorescence phenomena affecting the FISH assay accounted for the recorded reduced specificity with the tissue samples. The assessment confirmed Pneumocystis FISH in lung tissue as a highly sensitive screening approach; however, dissatisfying specificity in paraffin-embedded biopsies calls for confirmatory testing with other techniques in case of positive FISH screening results. In respiratory secretions, acceptable sensitivity and excellent specificity were demonstrated for the diagnostic application of the P. jirovecii-specific FISH assay.

Ensuring the quality of a circuit implies ensuring the quality of test. Despite the fact that performance-based testing has been the golden standard for Analog, Mixed-Signal and RF test for decades, high-reliability markets like automotive have found that functional test leaves some potential defects undetected that can produce in-field failure. There is thus a push towards defect-oriented testing which, in turn, calls for an efficient defect simulation framework. This paper presents a statistical adaptive defect simulation based on likelihood-weighted random sampling to evaluate the quality of AMS-RF tests in terms of defect coverage and fault escape. The adaptive loop takes a decision at each new defect simulation on whether it is more efficient to assess the defect coverage or the fault escape rate of the test under evaluation, as a function of the desired targets for these two metrics. Several decision criteria are proposed and validated by simulation of a complete IC for different tests. • Adaptive simulation process for AMS-RF test evaluation. • Complete vision of test quality, optimal evaluation of Defect Coverage and Fault Escape. • Comparative of different adaptive criteria with several Defect-oriented Tests. [ABSTRACT FROM AUTHOR]

While hybridization probe-based real-time PCR assays targeting highly repetitive multi-copy genome sequences for the diagnosis of S. mansoni complex or S. haematobium complex from human serum are well established, reports on the evaluation of respective assays for the identification of S. japonicum complex DNA in human serum are scarce. Here, we assessed the potential use of the retrotransposon sequences SjR2 and SjCHGCS19 from S. japonicum, S. mekongi and S. malayensis for the diagnosis of Asian Schistosoma infections. Based on available S. japonicum sequences and newly provided S. mekongi and S. malayensis sequences, hybridization probe-based real-time PCRs targeting SjR2 and SjCHGCS19 of the S. japonicum complex were designed both as consensus primer assays as well as multi-primer assays for the coverage of multiple variants of the target sequences. The assays were established using plasmids and S. mekongi DNA. While the consensus primer assays failed to detect S. mekongi DNA in human serum samples, the multi-primer assays showed positive or borderline positive results but only in 9.8% (6/61) of serum samples from patients with confirmed S. mekongi infections. Some cross-reactions with samples positive for S. mansoni or S. haematobium were observed but with the SjCHGCS19-PCR only. In spite of the low sensitivity, the presented experience may guide future evaluations of S. japonicum-complex-specific PCRs from human serum.

Abstract Background The most rigorous method for evaluating the effectiveness of diagnostic tests is through randomised trials that compare test-treatment interventions: complex interventions comprising episodes of testing, decision-making and treatment. The multi-staged nature of these interventions, combined with the need to relay diagnostic decision-making and treatment planning, has led researchers to hypothesise that test-treatment strategies may be very challenging to document. However, no reviews have yet examined the reporting quality of interventions used in test-treatment RCTs. In this study we evaluate the completeness of intervention descriptions in a systematically identified cohort of test-treatment RCTs. Methods We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL. Included trials randomized patients to diagnostic tests and measured patient outcomes after treatment. Two raters examined the completeness of test-treatment intervention descriptions in four components: 1) the test, 2) diagnostic decision-making, 3) management decision-making, 4) treatments. Results One hundred and three trials compared 105 control with 119 experimental interventions, most commonly in cardiovascular medicine (35, 34%), obstetrics and gynecology (17%), gastroenterology (14%) or orthopedics (10%). A broad range of tests were evaluated, including imaging (50, 42%), biochemical assays (21%) and clinical assessment (12%). Only five (5%) trials detailed all four components of experimental and control interventions, none of which also provided a complete care pathway diagram. Experimental arms were missing descriptions of tests, diagnostic-decision making, management planning and treatments (36%, 51%, 55% and 79% of trials respectively); control arms were missing the same details in 61%, 66%, 67% and 84% of trials. Conclusion Reporting of test-treatment interventions is very poor, inadequate for understanding the results of these trials, and for comparing or translating results into clinical practice. Reporting needs to improve, with greater emphasis on describing the decision-making components of care pathways in both pragmatic and explanatory trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0287-z .

Abstract Background There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. Methods We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. Results One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. Conclusion Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0 .

Evaluating text-based answers obtained in educational settings or behavioral studies is time-consuming and resource-intensive. Applying novel artificial intelligence tools such as ChatGPT might support the process. Still, currently available implementations do not allow for automated and case-specific evaluations of large numbers of student answers. To counter this limitation, we developed a flexible software and user-friendly web application that enables researchers and educators to use cutting-edge artificial intelligence technologies by providing an interface that combines large language models with options to specify questions of interest, sample solutions, and evaluation instructions for automated answer scoring. We validated the method in an empirical study and found the software with expert ratings to have high reliability. Hence, the present software constitutes a valuable tool to facilitate and enhance text-based answer evaluation.•Generative AI-enhanced software for customizable, case-specific, and automized grading of large amounts of text-based answers.•Open-source software and web application for direct implementation and adaptation., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2023 The Author(s).)

Objectives: We evaluated the six-item Inmate Security Assessment (ISA) tool used among detained youth in Manitoba, Canada. Method: Two hundred and forty-one recorded self-harm incidents among all incarcerated youth occurred between January 1, 2005 and December 31, 2010 (N=5102). The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR) and negative likelihood ratio (NLR) for three categories of suicide risk (high, medium, and low) as well as each of the six suicide risk evaluation indicators were determined. Receiver operating characteristic (ROC) curves and area under the curve (AUC) calculations for the three suicide risk levels and the six indicators were created. Results: Having at least a low suicide risk level (93.8%) or at least one suicide risk factor (94.6%) provided high sensitivity. Specificity was high if an individual had at least a medium suicide risk level (94.2%) or at least three suicide risk indicators (96.7%). The PPV was low (8.9-16.2%) and the NPV was high (94.9-99.3%) for all suicide risk levels. The most sensitive risk factor for self-harm was a prior history of suicidal behavior or a family history of suicide (94.6%). All risk indicators had a low PPV (7.4-23.1%) and a high NPV (95.4-99.5%). A very low NLR was found for those without prior suicidal behavior or a family history of suicide (0.107). The AUC was 0.719 (95%CI = 0.692-0.746), indicating a fair test. Conclusion: The ISA is a moderately accurate tool for identifying risk for self-harm in detained youth. [ABSTRACT FROM AUTHOR]

Background: Biomarker discovery studies often claim 'promising' findings, motivating further studies and marketing as medical tests. Unfortunately, the patient benefits promised are often inadequately explained to guide further evaluation, and few biomarkers have translated to improved patient care. We present a practical guide for setting minimum clinical performance specifications to strengthen clinical performance study design and interpretation. Methods: We developed a step-by-step approach using test evaluation and decision-analytic frameworks and present with illustrative examples. Results: We define clinical performance specifications as a set of criteria that quantify the clinical performance a new test must attain to allow better health outcomes than current practice. We classify the proposed patient benefits of a new test into three broad groups and describe how to set minimum clinical performance at the level where the potential harm of false-positive and false-negative results does not outweigh the benefits. (1) For add-on tests proposed to improve disease outcomes by improving detection, define an acceptable trade-off for false-positive versus true-positive results; (2) for triage tests proposed to reduce unnecessary tests and treatment by ruling out disease, define an acceptable risk of false-negatives as a safety threshold; (3) for replacement tests proposed to provide other benefits, or reduce costs, without compromising accuracy, use existing tests to benchmark minimum accuracy levels. Conclusions: Researchers can follow these guidelines to focus their study objectives and to define statistical hypotheses and sample size requirements. This way, clinical performance studies will allow conclusions about whether test performance is sufficient for intended use. [ABSTRACT FROM AUTHOR]

Short measures of psychological constructs are routinely used to save assessment time and cost. The downside is a trade-off between resource savings and psychometric quality. When evaluating tests, a pragmatic strategy is frequently applied that neglects the assessment objective, which may result in unfair rejection or unmindfully acceptance of short scales. Our main aim is to demonstrate the consequences of applying a pragmatic test evaluation strategy. We used two tests that measure the same construct—obsessive-compulsive symptomatology—but differ considerably in test length (1:3) and evaluated the measures by taking the assessment objective into account. The two scale scores showed distinct profiles of psychometric qualities. Whereas routinely evaluated reliability, factorial validity, and convergent/discriminant validity did not differ, rendering both tests useful for research purposes, substantial differences were found for qualities that are rarely focused on—measurement precision and diagnostic validity—which are highly relevant for accurate decisions in clinical practice. [ABSTRACT FROM AUTHOR]

Copyright of Dixi is the property of Universidad Cooperativa de Colombia and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The objective of this article is to describe the possible elements of comprehension that affect the response to new items on a verbal reasoning test. The starting point was the verbal report of a group of thirteen students obtaining high and low scores on an academic aptitude test at Universidad de Costa Rica. An exploratory qualitative study identified five irrelevant elements for comprehension: lexicon, rhetoric, length, very specific context, and use of previous knowledge, which affect the response to items of reasoning in the verbal context. The recognition of these aspects was a result of the research project, which aimed at determining the elements of the sentence, instructions, or detractors that hindered comprehension, and defining their relationship to the difficulty assigned by the test-taker. The suggestion was made to use the verbal report to generate evidence of validity for an admissions test. [ABSTRACT FROM AUTHOR]

Copyright of China Petroleum Machinery is the property of China Petroleum Machinery Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The present paper has three objectives. First, methods for comparing alternative tests for the purpose of replacement of one test with a second presumably superior test are described. Second, problems in the interpretation of the relevance of different diagnostic thresholds (thresholds of positivity) that define who is and who is not a disordered gambler are examined and a potential solution offered in the form of a common quantitative measure of the risk of being a disordered gambler. Third, alternative methodologies are described as potential solutions to the lack of a gold or reference standard in the evaluation of new tests. [ABSTRACT FROM AUTHOR]