Search

Automatic assessment of the operation ability of operators based on computers is an essential approach for improving the effectiveness of equipment operation training and enhancing equipment safety. Present methods primarily focus on the operation results but pay less attention to the operation procedure. One reason is that there is a lack of a model that has the ability to describe all probable paths to accomplish the same task. Therefore, an operations chain model is put forward for the first time to describe the standard operation procedure and relationships among operations based on the decomposition of operational tasks and the relationships among the various operations required to fulfill the task. A specific operation task corresponds to an operations chain, which will form one or multiple standard operation sequences that will allow trainees to complete the same task through different paths. The Needleman–Wunsch sequence alignment algorithm is introduced to match the trainees' operation sequence with all standard sequences. The maximum alignment result is the score of the trainees' operations. An example shows that the operations chain model can accurately describe the complex structure of the standard operating procedures. The Needleman–Wunsch sequence alignment algorithm can objectively evaluate the trainee's operation capabilities. The combination of the operations chain model and sequence alignment algorithm can form a complete operation procedure assessment method that is friendlier to trainees and has more objective evaluation results. The method will help to improve the effectiveness of the competency assessment of equipment operators. [ABSTRACT FROM AUTHOR]

This study investigated the automatic segmentation and classification of mitral regurgitation (MR) and tricuspid regurgitation (TR) using a deep learning-based method, aiming to improve the efficiency and accuracy of diagnosis of valvular regurgitations. A VABC-UNet model was proposed consisting of VGG16 encoder, U-Net decoder, batch normalization, attention block and deepened convolution layer based on the U-Net backbone. Then, a VABC-UNet-based assessment framework was established for automatic segmentation, classification, and evaluation of valvular regurgitations. A total of 315 color Doppler echocardiography images of MR and/or TR in an apical four-chamber view were collected, including 35 images in the test dataset and 280 images in the training dataset. In comparison with the classic U-Net and VGG16-UNet models, the segmentation performance of the VABC-UNet model was evaluated via four metrics: Dice, Jaccard, Precision, and Recall. According to the features of regurgitation jet and atrium, the regurgitation could automatically be classified into MR or TR, and evaluated to mild, moderate, moderate–severe, or severe grade by the framework. The results show that the VABC-UNet model has a superior performance in the segmentation of valvular regurgitation jets and atria to the other two models and consequently a higher accuracy of classification and evaluation. There were fewer pseudo- and over-segmentations by the VABC-UNet model and the values of the metrics significantly improved (p < 0.05). The proposed VABC-UNet-based framework achieves automatic segmentation, classification, and evaluation of MR and TR, having potential to assist radiologists in clinical decision making of the regurgitations in valvular heart diseases. [ABSTRACT FROM AUTHOR]

Background: Right heart catheterization is the gold standard for evaluating hemodynamic parameters of pulmonary circulation, especially pulmonary artery pressure (PAP) for diagnosis of pulmonary hypertension (PH). However, the invasive and costly nature of RHC limits its widespread application in daily practice. Purpose: To develop a fully automatic framework for PAP assessment via machine learning based on computed tomography pulmonary angiography (CTPA). Materials and Methods: A machine learning model was developed to automatically extract morphological features of pulmonary artery and the heart on CTPA cases collected between June 2017 and July 2021 based on a single center experience. Patients with PH received CTPA and RHC examinations within 1 week. The eight substructures of pulmonary artery and heart were automatically segmented through our proposed segmentation framework. Eighty percent of patients were used for the training data set and twenty percent for the independent testing data set. PAP parameters, including mPAP, sPAP, dPAP, and TPR, were defined as ground-truth. A regression model was built to predict PAP parameters and a classification model to separate patients through mPAP and sPAP with cut-off values of 40 mm Hg and 55 mm Hg in PH patients, respectively. The performances of the regression model and the classification model were evaluated by analyzing the intraclass correlation coefficient (ICC) and the area under the receiver operating characteristic curve (AUC). Results: Study participants included 55 patients with PH (men 13; age 47.75 ± 14.87 years). The average dice score for segmentation increased from 87.3% ± 2.9 to 88.2% ± 2.9 through proposed segmentation framework. After features extraction, some of the AI automatic extractions (AAd, RVd, LAd, and RPAd) achieved good consistency with the manual measurements. The differences between them were not statistically significant (t = 1.222, p = 0.227; t = −0.347, p = 0.730; t = 0.484, p = 0.630; t = −0.320, p = 0.750, respectively). The Spearman test was used to find key features which are highly correlated with PAP parameters. Correlations between pulmonary artery pressure and CTPA features show a high correlation between mPAP and LAd, LVd, LAa (r = 0.333, p = 0.012; r = −0.400, p = 0.002; r = −0.208, p = 0.123; r = −0.470, p = 0.000; respectively). The ICC between the output of the regression model and the ground-truth from RHC of mPAP, sPAP, and dPAP were 0.934, 0.903, and 0.981, respectively. The AUC of the receiver operating characteristic curve of the classification model of mPAP and sPAP were 0.911 and 0.833. Conclusions: The proposed machine learning framework on CTPA enables accurate segmentation of pulmonary artery and heart and automatic assessment of the PAP parameters and has the ability to accurately distinguish different PH patients with mPAP and sPAP. Results of this study may provide additional risk stratification indicators in the future with non-invasive CTPA data. [ABSTRACT FROM AUTHOR]

This paper presents the development and evaluation of a method for enabling quantitative and automatic scoring of alternating tapping performance of patients with Parkinson's disease (PD). Ten healthy elderly subjects and 95 patients in different clinical stages of PD have utilized a touch-pad handheld computer to perform alternate tapping tests in their home environments. First, a neurologist used a web-based system to visually assess impairments in four tapping dimensions ('speed', 'accuracy', 'fatigue' and 'arrhythmia') and a global tapping severity (GTS). Second, tapping signals were processed with time series analysis and statistical methods to derive 24 quantitative parameters. Third, principal component analysis was used to reduce the dimensions of these parameters and to obtain scores for the four dimensions. Finally, a logistic regression classifier was trained using a 10-fold stratified cross-validation to map the reduced parameters to the corresponding visually assessed GTS scores. Results showed that the computed scores correlated well to visually assessed scores and were significantly different across Unified Parkinson's Disease Rating Scale scores of upper limb motor performance. In addition, they had good internal consistency, had good ability to discriminate between healthy elderly and patients in different disease stages, had good sensitivity to treatment interventions and could reflect the natural disease progression over time. In conclusion, the automatic method can be useful to objectively assess the tapping performance of PD patients and can be included in telemedicine tools for remote monitoring of tapping. [ABSTRACT FROM AUTHOR]

E-learning tools are gaining increasing relevance as facilitators in the task of learning how to program. This is mainly a result of the pandemic situation and consequent lockdown in several countries, which forced distance learning. Instant and relevant feedback to students, particularly if coupled with gamification, plays a pivotal role in this process and has already been demonstrated as an effective solution in this regard. However, teachers still struggle with the lack of tools that can adequately support the creation and management of online gamified programming courses. Until now, there was no software platform that would be simultaneously open-source and general-purpose (i.e., not integrated with a specific course on a specific programming language) while featuring a meaningful selection of gamification components. Such a solution has been developed as a part of the Framework for Gamified Programming Education (FGPE) project. In this paper, we present its two front-end components: FGPE AuthorKit and FGPE PLE, explain how they can be used by teachers to prepare and manage gamified programming courses, and report the results of the usability evaluation by the teachers using the platform in their classes. [ABSTRACT FROM AUTHOR]

The assessment of foot pad dermatitis at slaughter is a suitable method to assess and monitor the welfare of broilers. The goals of this study were to define and validate a camera-based score that could identify macroscopic lesions of the foot pads, to identify errors, and to assess possible external factors that could influence the assessment. In the first phase 200 feet of broilers and in the second phase 500 feet were collected at slaughter, assessed visually, hung back into the evisceration line, and assessed by an automatic system. The camera score cut-off values were defined in the first (=calibration) phase. In the second (=validation) phase, the performance of diagnosis for these cut-off values was evaluated, and possible errors in the assessment of reference surface area and foot pad lesions were analyzed. The results showed that, in particular, Macro Scores 0, 2, and 3 could be identified with sufficiently high sensitivity. For Macro Score 1, the sensitivity of diagnosis was not sufficiently high in the two evaluated software versions. The current automatic assessment systems at slaughter could be adjusted to the cut-off values in order to classify foot pad dermatitis lesions. Furthermore, software updates can enhance the performance measures and lower the probability of errors. [ABSTRACT FROM AUTHOR]

The COVID-19 pandemic has evidenced a need for tools and methodologies to support students' autonomous learning and the formative assessment practices in distance education contexts, especially for students from challenging backgrounds. This paper proposes a conceptualization of Interactive Feedback (IF) for Mathematics, which is a step-by-step interactive process that guides the learner in the resolution of a task after one or more autonomous tentative. This conceptualization is grounded on theories and models of automatic assessment, formative assessment, and feedback. We discuss the effectiveness of the IF for engaging students from low socio-economic contexts in closing the gap between current and reference performance through a didactic experimentation involving 299 Italian students in grade 8. Using quantitative analyses on data from the automatic assessment, we compared the results of the first and last attempts in activities with and without IF, based on algorithmic parameters so that the task changes at every attempt. We found that IF was more effective than other kinds of activities to engage learners in actions aimed at improving their results, and the effects are stronger in low socio-economic contexts. [ABSTRACT FROM AUTHOR]

The continuous development of machine learning procedures and the development of new ways of mapping based on the integration of spatial data from heterogeneous sources have resulted in the automation of many processes associated with cartographic production such as positional accuracy assessment (PAA). The automation of the PAA of spatial data is based on automated matching procedures between corresponding spatial objects (usually building polygons) from two geospatial databases (GDB), which in turn are related to the quantification of the similarity between these objects. Therefore, assessing the capabilities of these automated matching procedures is key to making automation a fully operational solution in PAA processes. The present study has been developed in response to the need to explore the scope of these capabilities by means of a comparison with human capabilities. Thus, using a genetic algorithm (GA) and a group of human experts, two experiments have been carried out: (i) to compare the similarity values between building polygons assigned by both and (ii) to compare the matching procedure developed in both cases. The results obtained showed that the GA—experts agreement was very high, with a mean agreement percentage of 93.3% (for the experiment 1) and 98.8% (for the experiment 2). These results confirm the capability of the machine-based procedures, and specifically of GAs, to carry out matching tasks. [ABSTRACT FROM AUTHOR]

Recent outstanding results of supervised object detection in competitions and challenges are often associated with specific metrics and datasets. The evaluation of such methods applied in different contexts have increased the demand for annotated datasets. Annotation tools represent the location and size of objects in distinct formats, leading to a lack of consensus on the representation. Such a scenario often complicates the comparison of object detection methods. This work alleviates this problem along the following lines: (i) It provides an overview of the most relevant evaluation methods used in object detection competitions, highlighting their peculiarities, differences, and advantages; (ii) it examines the most used annotation formats, showing how different implementations may influence the assessment results; and (iii) it provides a novel open-source toolkit supporting different annotation formats and 15 performance metrics, making it easy for researchers to evaluate the performance of their detection algorithms in most known datasets. In addition, this work proposes a new metric, also included in the toolkit, for evaluating object detection in videos that is based on the spatio-temporal overlap between the ground-truth and detected bounding boxes. [ABSTRACT FROM AUTHOR]

Loss of motivation is one of the most prominent concerns in programming education as it negatively impacts time dedicated to practice, which is crucial for novice programmers. Of the distinct techniques introduced in the literature to engage students, gamification, is likely the most widely explored and fruitful. Game elements that intrinsically motivate students, such as graphical feedback and game-thinking, reveal more reliable long-term positive effects, but those involve significant development effort. This paper proposes a game-based assessment environment for programming challenges, built on top of a specialized framework, in which students develop a program to control the player, henceforth called Software Agent (SA). During the coding phase, students can resort to the graphical feedback demonstrating how the game unfolds to improve their programs and complete the proposed tasks. This environment also promotes competition through competitive evaluation and tournaments among SAs, optionally organized at the end by the teacher. Moreover, the validation of the effectiveness of Asura in increasing undergraduate students' motivation and, consequently, the practice of programming is reported. [ABSTRACT FROM AUTHOR]

Point-based standard methodologies (PBSM) suggest using 'at least 20' check points in order to assess the positional accuracy of a certain spatial dataset. However, the reason for decreasing the number of checkpoints to 20 is not elaborated upon in the original documents provided by the mapping agencies which develop these methodologies. By means of theoretical analysis and experimental tests, several authors and studies have demonstrated that this limited number of points is clearly insufficient. Using the point-based methodology for the automatic positional accuracy assessment of spatial data developed in our previous study Ruiz-Lendínez, et al (2017) and specifically, a subset of check points obtained from the application of this methodology to two urban spatial datasets, the variability of National Standard for Spatial Data Accuracy (NSSDA) estimations has been analyzed according to sample size. The results show that the variability of NSSDA estimations decreases when the number of check points increases, and also that these estimations have a tendency to underestimate accuracy. Finally, the graphical representation of the results can be employed in order to give some guidance on the recommended sample size when PBSMs are used. [ABSTRACT FROM AUTHOR]