Search

In the context of formative assessment in classrooms, the incorporation of automated evaluation (AE) systems and teachers' interactions with them hold significant importance. This study aimed to investigate the cognitive processes of pre-service teachers as they engaged with an AE system. We developed an unsupervised learning-based AE system, the Scoring Assistant using Artificial Intelligence (SAAI). SAAI calculates scores without relying on predefined labels and generates scientific keywords from student responses to provide constructive feedback. We collected a substantial number of constructed responses from students, and four pre-service teachers evaluated these responses initially without any external assistance and then re-evaluated them using SAAI scores as a reference point. Employing a mixed-methods approach, this study demonstrated a strong level of consistency between human raters and SAAI scores. Pre-service teachers also reflectively recalibrated their assessments and adjusted their rubrics to identify students' learning more accurately. This study highlights the practical application of AE in real classroom settings and demonstrates how AE can enhance efficiency and accuracy in K-12 science assessments, thus supporting teachers. [ABSTRACT FROM AUTHOR]

The objective of the research is to enhance the functionality of the Evaluator software agent within the Distributed eLearning Center (DeLC), a platform providing extensive support for e-learning activities. This paper is focused on presenting the latest step in the evolution of one software agent (Evaluator Agent), which helps teachers during evaluating students' exams, being responsible for evaluation of short free text answers, provided by the students. To accomplish its job, the agent has its own dictionary, created from a set of words and expressions, provided by test creators for each question of that type, and from the words of each student's answer, marked as successful by the teacher. Now this agent's effectiveness could be increased by extending its dictionary with much larger knowledge base or so-called Large Language Models, made accessible through third party AI, as ChatGPT. Experiments and results are provided to compare the change in agent's behavior after the integration with the AI system. The results presented in this work are from first experiments and are deliberately limited in number to enable manual verification. [ABSTRACT FROM AUTHOR]

The globally accepted evaluation method for facial palsy is the House–Brackmann facial grading system; however, it does not reflect minute changes. Several methods have been attempted, but there is no universally accepted evaluation method that is non-time-consuming and quantitative. Recently, Emotrics, a two-dimensional analysis that incorporates machine-learning techniques, has been used in various clinical fields. However, its reliability and validity have not yet been determined. Therefore, this study aimed to examine and establish the reliability and validity of Emotrics. All patients had previously received speech therapy for facial palsy at our hospital between January and November 2022. In speech therapy at our hospital, Emotrics was routinely used to measure the state of the patient's facial palsy. A frame was created to standardize and overcome the limitation of the two-dimensional analysis. Interrater, intrarater, and intrasubject reliability were evaluated with intraclass correlation coefficients (ICC) by measuring the indicators that reflect eye and mouth functions. Validity was evaluated using Spearman's correlation for each Emotrics parameter and the House–Brackmann facial grading system. A total of 23 patients were included in this study. For all parameters, there was significant interrater and intrarater reliability (ICC, 0.61 to 0.99). Intrasubject reliability showed significant reliability in most parameters (ICC, 0.68 to 0.88). Validity showed a significant correlation in two parameters (p-value < 0.001). This single-center study suggests that Emotrics could be a quantitative and efficient facial-palsy evaluation method with good reliability. Therefore, Emotrics is expected to play a key role in assessing facial palsy and in monitoring treatment effects more accurately and precisely. [ABSTRACT FROM AUTHOR]

Architectural spatial layout configurations are the result of a complex design process aiming at managing the most advantageous location of all spaces with certain objectives and constraints. In the presence of broadening the scope of understanding spatial configuration, its role, and its composition in scope of various programmatic, ideological, formal, and engineering aspects, embedded in a complex theoretical background. The aspect of evaluation was the least apparent and there were significant limitations to what can be achieved there. This paper is aiming at evaluating functionality (such as quality of space, use, Access, adjacency, non-adjacency, proximity etc.), build quality (such as performance, efficiency, cost, etc.), impact (such as user performance, internal quality, urban, social, and cultural integration, etc.), indication of the quality of the architectural space design and the quality of the layout configuration. To achieve this research goal, we devised all possible spatial relation, affecting these qualities, as inputs for a given architectural space using descriptive rule blocks. We defined this fuzzy logic system for residential spaces that has been blended into a layout to evaluate the layout configuration. We defined all input variables, output variables, and fuzzy sets, and present space-space relations using membership functions. The paper proposes a framework based on fuzzy logic approach for automated evaluation of architectural spatial layout configurations. [ABSTRACT FROM AUTHOR]

• We conducted probability of detection and false positive analyses to quantitatively evaluate and compare the reliability of the automated and human-based evaluations. • We compared three image-processing-based segmentation and three sequential-signal-based deep learning algorithms for automatic data analysis. • We demonstrated the potential of using advanced deep-learning techniques to solve complex non-destructive examination data interpretation tasks. Nowadays, the reliability of data analysis and decision-making based on deep learning (DL) remains a primary concern in promoting DL technology for industrial non-destructive testing and evaluation (NDT&E). This study focuses on the quantitative assessment of the reliability of various automated data analysis techniques in NDT&E. To achieve this, optical pulsed thermography was employed to inspect three non-planar carbon-fiber-reinforced polymer (CFRP) samples, each containing embedded Teflon to simulate debonding defects. After applying thermographic signal reconstruction and first-order derivation processing to the raw thermal data, automated analysis was performed using three image-processing-based segmentation methods and three sequential-signal-based DL algorithms. Additionally, an experienced inspector manually analyzed the data for comparison purposes. Subsequently, the probability of detection and false positive analyses were conducted to quantitatively evaluate and compare the reliability of the automated and human-based evaluations. The comparison results demonstrated that optimal DL classification and advanced image-processing-based segmentation techniques could achieve performance levels close to that of human inspectors in defect detection, even for challenging non-planar CFRP samples. [ABSTRACT FROM AUTHOR]

The compaction quality of a highway subgrade is influenced by the compaction parameters and soil properties. Current practice lacks reliable models to assess this multivariate situation. This paper describes a monitoring and evaluation method based on intelligent compaction (IC) and artificial neural networks (ANNs) to assess the compaction quality of a highway subgrade. Field compaction tests were conducted at various roller speeds and vibratory modes. The compactness was predicted using a particle-swarm-optimization optimized backpropagation neural network (PSO-BP-NN) model. The model was integrated into the IC system to automatically evaluate compaction quality. It also demonstrated the advantage of high prediction accuracy with multiple input variables of operation parameters and soil gradation. By efficiently controlling subgrade compaction, the model can be employed for engineering practice and facilitate IC applications. • An advanced ANN model is proposed to evaluate the compaction quality. • Four compaction modes are identified to characterize different compaction parameters. • The particle gradation is an influential factor and is integrated into the correlation model. • The proposed evaluation model demonstrates a good generalization for different scenarios. [ABSTRACT FROM AUTHOR]

5-Bromo-2-deoxyuridine (BrdU) staining is often used to evaluate cortical layer formation during mammalian brain development. This method allows the quantification of newly generated cells and therefore the study of the effects of xenobiotics or genetic factors on proliferation, cell death and migration behavior in a quantitative manner. However, these endpoints are generally assessed by time-consuming manual evaluation. In the present work, we introduce a novel procedure to identify and quantify BrdU+ cells within cortical layers, using the commercially available vHCS-Scan V.6.3.1 software to identify BrdU+ cell coordinates and the novel program 'BrdeLuxe' to define cortical layers and quantitatively assign BrdU+ cells to them. This procedure is compared to BrdU+ cell counting with the freeware 'ImageJ' in respect to the manual evaluation, all by two different researchers. BrdeLuxe shows high accuracy and precision for the determination of total number of BrdU+ cells compared to the manual counting, while ImageJ does not reach such results. Accuracy and precision are also higher for employing the BrdeLuxe program to evaluate the percentage of BrdU+ cells per brain layer compared to ImageJ. In terms of running time, BrdeLuxe is the fastest method of the three making it more suitable for multiple brain slices analyses. [ABSTRACT FROM AUTHOR]

Modern Web sites use several techniques (e.g. DOM manipulation) that allow for the injection of new content into their Web pages (e.g. AJAX), as well as manipulation of the HTML DOM tree. This has the consequence that the Web pages that are presented to users (i.e. after browser processing) are different from the original structure and content that is transmitted through HTTP communication (i.e. after browser processing). This poses a series of challenges for Web accessibility evaluation, especially on automated evaluation software. This article details an experimental study designed to understand the differences posed by accessibility evaluation after Web browser processing. We implemented a Javascript-based evaluator, Qual Web, that can perform WCAG 2.0 based accessibility evaluations in the two phases of browser processing. Our study shows that, in fact, there are considerable differences between the HTML DOM trees in both phases, which have the consequence of having distinct evaluation results. We discuss the impact of these results in the light of the potential problems that these differences can pose to designers and developers that use accessibility evaluators that function before browser processing. [ABSTRACT FROM AUTHOR]

The Web Science framework poses fundamental questions on the analysis of the Web, by focusing on how microscopic properties (e.g. at the level of a Web page or Web site) emerge into macroscopic properties and phenomena. One research topic on the analysis of the Web is Web accessibility evaluation, which centres on understanding how accessible a Web page is for people with disabilities. However, when framing Web accessibility evaluation on Web Science, we have found that existing research stays at the microscopic level. This article presents an experimental study on framing Web accessibility evaluation into Web Science's goals. This study resulted in novel accessibility properties of the Web not found at microscopic levels, as well as of Web accessibility evaluation processes themselves. We observed at large scale some of the empirical knowledge on how accessibility is perceived by designers and developers, such as the disparity of interpretations of accessibility evaluation tools warnings. We also found a direct relation between accessibility quality and Web page complexity. We provide a set of guidelines for designing Web pages, education on Web accessibility, as well as on the computational limits of large-scale Web accessibility evaluations. [ABSTRACT FROM AUTHOR]

Abstract: Within the field of incident management split second decisions have to be made, usually on the basis of incomplete and partially incorrect information. As a result of these conditions, errors occur in such decision processes. In order to avoid repetition of such errors, historic cases, disaster plans, and training logs need to be thoroughly analysed. This paper presents a formal approach for such an analysis that pays special attention to spatial and temporal aspects, to information exchange, and to organisational structure. The formal nature of the approach enables automation of analysis, which is illustrated by case studies of two disasters. [Copyright &y& Elsevier]

• A support to reduce effort of pedagogues in large-scale to evaluate novelty in mass examination. • To identify features for evaluating novelty in mass examination of design education. • To design an expert system to evaluate novelty from image-based solutions. • To evaluate performance of the model to optimize trust in it. An inherent criterion of evaluation in Design education is novelty. Novelty is a measure of newness in solutions which is evaluated based on relative comparison with its frame of reference. Evaluating novelty is subjective and generally depends on expert's referential metrics based on their knowledge and persuasion. Pedagogues compare and contrast solution for cohort of students in mass examination aspiring admission to Design schools. Large number of students participate in mass examinations, and in situations like this, examiners are confronted with multiple challenges in subjective evaluation such as- 1) Errors encountered in evaluation due to stipulated timeline, 2) Errors encountered due to prolonged working hours, 3) Errors encountered due to stress in performing repeated task on a large-scale. Pedagogues remain ever-inquisitive and vigilant about the evaluation process being consistent and accurate due to monotony of repeated task. To mitigate these challenges, a computational model is proposed for automating evaluation of novelty in image-based solutions. This model is developed by mixed-method research, where features for evaluating novelty are investigated by conducting a survey study. Further, these features were utilized to evaluate novelty and generate score for image-based solutions using Computer Vision (CV) and Deep Learning (DL) techniques. The performance metric of the model when measured reveals a negligible difference between scores of experts and scores of proposed model. These comparative analysis of the proposed model with human experts' confirm the competence of the devised model and would go a long way to establish trust of pedagogues by ensuring reduced error and stress during the evaluation process. [ABSTRACT FROM AUTHOR]