Search

Ecological suitability evaluation is a critical component of regional sustainable development and construction, serving as a foundation for optimizing spatial patterns of regional growth. This is particularly pertinent in karst mountainous regions characterized by limited land resources and heightened ecosystem vulnerability, where a quantitative assessment of ecological suitability for land development is both crucial and urgent. Based on the fundamental principles of structural and functional dynamics in landscape ecology, this study focuses on Gui'an New Area, a designated urban development zone situated in the karst landscape of Guizhou Province. An index system was established encompassing three dimensions: ecological elements, ecological significance, and ecological resilience, utilizing the integrated ecological resistance (IER) model to evaluate the suitability of regional development and construction. The results reveal that the eastern region exhibits higher suitability compared to the central and western regions, with the northwest region demonstrating the lowest suitability overall. Relatively speaking, the evaluation of geological environment suitability and the comprehensive ecological constraints associated with development and construction indicates that the areas currently planned and ongoing reflect flat terrain and low ecological risk. Furthermore, within the scope of ecosystem dynamic adaptation, developmental activities in these regions exert minimal impact on the natural ecosystem, thereby demonstrating a high suitability for development and construction. In terms of future key development zones, areas with gentle slopes ranging from 8 to 15 degrees are recommended, aligning with the actual requirements for cultivated land protection. The total area designated as prohibited development zones constitutes the smallest proportion, representing only 9.45%, which is significantly lower than that of priority development zones (38.75%) and moderate development zones (22.45%). From the perspective of landscape ecology, this paper provides a comprehensive investigation into the ecological suitability evaluation system for development and construction in the karst regions of Southwest China, offering valuable insights for assessing ecological suitability in similar areas. [ABSTRACT FROM AUTHOR]

This study aims to identify an appropriate conceptual framework to evaluate crowdsourcing platforms from an open innovation perspective employing a combination of qualitative and quantitative methods. The initial indices of the performance evaluation framework in the crowdsourcing platforms are obtained through the Delphi method and interviews with experts. Then, using these factors, a statistical questionnaire is designed and distributed among users of crowdsourcing platforms to confirm or reject the factors. Finally, the aspects of the performance evaluation framework of crowdsourcing platforms are specified from the perspective of open innovation. Using fuzzy hierarchical analysis, these aspects are prioritized in order of importance: Collaboration, Project design, Moderation, Terms and conditions, UI/UX (user interface and user experience), and Key statistics. Concerning the principle of crowdsourcing, which is based on crowd participation and crowd intelligence of users, Collaboration and Project design turned out to be the significant factors in evaluating a crowdsourcing platform. [ABSTRACT FROM AUTHOR]

Local government is underpinned by the principle that finding sustainable ways to meet the needs of communities through service delivery matters. Post 1994, the South African government opted for a 'developmental' local government system as a way of addressing the socio-economic challenges of historically disadvantaged communities. However, the scholarly discourse on service delivery in South Africa repeatedly highlights disappointing municipal performance. This study examined the need for performance evaluation of municipal service delivery in South Africa. Using a mixed methods approach and purposive sampling to collect data from 36 municipal managers of dysfunctional municipalities, the findings revealed that municipal performance was not evaluated satisfactorily; service delivery backlogs persisted and the impacts of deficient services on communities were not evaluated by either municipalities or communities. These findings justify a need for an effective evaluation framework for measuring municipal performance to assist municipalities to identify service delivery challenges and promote necessary change and improvement. [ABSTRACT FROM AUTHOR]

The domain of Enterprise Information Systems Engineering uses many different conceptual modelling languages and methods to specify the requirements of a system under development. The complexity of the systems under development may require addressing different perspectives with different models, such as the data and process perspectives. The modeller will thus have to choose the appropriate (set of) modelling languages according to their specific modelling goal. Given that the different aspects relate to a single system, ideally, the models that capture the different perspectives should be aligned and consistent to ensure their integration. Each candidate (set of) modelling languages comes with advantages and disadvantages. To make an informed choice in this matter, the modeller should select a number of criteria relevant to their problem domain and compare candidate modelling languages based on these criteria. A comprehensive evaluation framework for integrated modelling approaches, that considers more general aspects such as understandability, ease of use, model quality, etc. besides the ability to model the desired aspects, does not yet exist and is therefore the focus of this paper. In recent years, several combinations of modelling languages have been investigated. Amongst these combinations, data + process modelling has attracted a lot of interest, and, interestingly, evaluation frameworks for this combination have been proposed as well. Therefore, this paper will primarily focus on the integrated multi-modelling of data and processes, including the process-related viewpoints of users and authorisations. The contribution of this paper is two-fold: on a theoretical level, the paper provides an overview of existing evaluation frameworks in the literature, builds a more complete set of evaluation criteria and proposes a unified taxonomy for the classification of these evaluation criteria (TEC-MAP); on a practical level, the paper provides guidance and support to the modeller for selecting the appropriate evaluation criteria for their problem domain and presents three examples of the application of TEC-MAP. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive review of the use of Artificial Intelligence (AI) in Systematic Literature Reviews (SLRs). A SLR is a rigorous and organised methodology that assesses and integrates prior research on a given topic. Numerous tools have been developed to assist and partially automate the SLR process. The increasing role of AI in this field shows great potential in providing more effective support for researchers, moving towards the semi-automatic creation of literature reviews. Our study focuses on how AI techniques are applied in the semi-automation of SLRs, specifically in the screening and extraction phases. We examine 21 leading SLR tools using a framework that combines 23 traditional features with 11 AI features. We also analyse 11 recent tools that leverage large language models for searching the literature and assisting academic writing. Finally, the paper discusses current trends in the field, outlines key research challenges, and suggests directions for future research. We highlight three primary research challenges: integrating advanced AI solutions, such as large language models and knowledge graphs, improving usability, and developing a standardised evaluation framework. We also propose best practices to ensure more robust evaluations in terms of performance, usability, and transparency. Overall, this review offers a detailed overview of AI-enhanced SLR tools for researchers and practitioners, providing a foundation for the development of next-generation AI solutions in this field. [ABSTRACT FROM AUTHOR]

With the growing demand for synthetic data to address contemporary issues in machine learning, such as data scarcity, data fairness, and data privacy, having robust tools for assessing the utility and potential privacy risks of such data becomes crucial. SynthEval, a novel open-source evaluation framework distinguishes itself from existing tools by treating categorical and numerical attributes with equal care, without assuming any special kind of preprocessing steps. This makes it applicable to virtually any synthetic dataset of tabular records. Our tool leverages statistical and machine learning techniques to comprehensively evaluate synthetic data fidelity and privacy-preserving integrity. SynthEval integrates a wide selection of metrics that can be used independently or in highly customisable benchmark configurations, and can easily be extended with additional metrics. In this paper, we describe SynthEval and illustrate its versatility with examples. The framework facilitates better benchmarking and more consistent comparisons of model capabilities. [ABSTRACT FROM AUTHOR]

This study mainly aims to validate the framework of the entrepreneurship ecosystem of science and technology parks in Iran. The purpose of this study is to improve the infrastructures and achieve the vision of the science and technology parks, managing and directing the interactions among different factors involved for creating sustainable value, entrepreneurial opportunities and business scalability. In terms of purpose, the current research is applied and has a descriptive approach of the correlational type, with a qualitative strategy of a sequential exploratory type. The statistical sample in the qualitative section was purposeful and theoretical in the form of interviews with 15 experts in the field of entrepreneurship and science and technology parks in Iran. In the quantitative part, 181 managers of parks and technology units, and other key people involved in this ecosystem participated. The reliability and validity of the researcher-made questionnaire was measured using Cronbach’s alpha and the average extracted variance were above 0.845 and 0.604, respectively. The findings show that causal factors (characteristics of local structures, entrepreneurship policies and local entrepreneurship indicators), contextual factors (financial, human resources, physical and intangible infrastructure, capacity building, promotion, enablers) and intervening factors (adaptability, tension removal, resilience) predict 0.487, 0.541 and 0.634 of changes in strategies (interactions within and outside the ecosystem), respectively. The axial category predicts 0.798 of changes in strategies. Strategies also predict 0.782 of changes in outcomes (wealth creation, workforce, and businesses). The science and technology parks are suggested to consider the aforementioned

The integration of smart buildings (SBs) into smart cities (SCs) is critical to urban development, with the potential to improve SCs' performance. Artificial intelligence (AI) applications have emerged as a promising tool to enhance SB and SC development. The authors apply an AI-based methodology, particularly Large Language Models of OpenAI ChatGPT-3 and Google Bard as AI experts, to uniquely evaluate 26 criteria that represent SB services across five SC infrastructure domains (energy, mobility, water, waste management, and security), emphasizing their contributions to the integration of SB into SC and quantifying their impact on the efficiency, resilience, and environmental sustainability of SC. The framework was then validated through two rounds of the Delphi method, leveraging human expert knowledge and an iterative consensus-building process. The framework's efficiency in analyzing complicated information and generating important insights is demonstrated via five case studies. These findings contribute to a deeper understanding of the effects of SB services on SC infrastructure domains, highlighting the intricate nature of SC, as well as revealing areas that require further integration to realize the SC performance objectives. [ABSTRACT FROM AUTHOR]

Objective: Propose a framework for developing and evaluating Total Worker Health ® (TWH) education and training efforts by implementing institutions. Methods: This is a review of TWH information from symposia, workshops, academic offerings, and publications, along with a review of education and training development and evaluation resources applicable across various disciplines. Results: Examples of knowledge, skills, and abilities (KSAs) are provided for each TWH core competency, and a framework for developing and evaluating a TWH competency-based education or training program. Conclusions: The proposed set of KSAs and framework for developing and evaluating TWH education or training programs may inform future pilot testing of KSAs and framework by implementing institutions and help standardize practices across the discipline. Academic, business, community, labor, and government stakeholders are encouraged to provide further input to assist in its maturation and uptake. [ABSTRACT FROM AUTHOR]

This paper presents an enhanced evaluation framework for teacher professional development programmes, which is based on one originally proposed by Huber in 2011. This paper draws on a study on the Professional Up-skilling of English Language Teachers (ProELT) programme in Sabah (Borneo), Malaysia. The study adopted a mixed methods exploratory sequential design using a questionnaire survey, individual interviews and a focus group discussion. Based on the findings from the study, four new components have been added to Huber's original framework, namely (1) selection of participants, (2) incorporation of the Adult Learning principles, (3) follow-up support, and (4) assessment of programme impact. This enhanced framework has significant contributions to make to programme designers and programme providers, in providing them with additional guidelines to consider when designing the pre-, ongoing and post-phases of a teacher professional development programme. [ABSTRACT FROM AUTHOR]

The growing array of English Language Learning (ELL) websites and their users suggests that the ease of getting online material exceeds the process of reading books when it comes to achieving individuals' language learning objectives. However, the task of locating high-quality websites that cater to the needs of language teachers and students has gotten increasingly challenging. The evaluation of ELL websites is highly worthwhile because it facilitates the identification of appropriate resources for educators and students. Thus, this study attempts to assess the relevance of a compilation of ELL websites for primary learners. Besides, it aims to analyse the suitability of one selected ELL website in the Malaysian primary education context. The research methodology employed two evaluation stages: the general evaluation and the context-specific evaluation which were conducted by four experienced primary school teachers using a checklist and an evaluation framework. The results of this study suggest that ELL websites are useful as supplementary resources for ESL and EFL primary learners. The only concern is the availability of technological facilities within the selected context, which will definitely affect the implementation and integration of ELL websites as a whole. The aforementioned findings provide website hosts with valuable insights into their website's limitations and the need to enhance its relevance and suitability. The findings shed light on the effectiveness of the evaluation framework in assessing ELL websites. This study emphasises the need to investigate the suitability of ELL websites in other contexts, which will undoubtedly highlight the potential of these resources. [ABSTRACT FROM AUTHOR]

Generative AI (GenAI) has transformed how businesses operate and innovate and how individuals learn, live, and work. Large language models (LLMs), a specific type of GenAI, focus on generating human-like text based on user instructions. Like other types of GenAI, LLMs have received wide recognition for their potential to augment human intelligence, but several challenges hinder efforts to realize their full potential in practice. Some notable challenges include not adequately exploring LLM applications beyond chatbots and/or text generation, the difficulty in categorizing various LLM adaptation strategies (particularly regarding human interactions), and the lack of a reference framework for evaluating and selecting LLM adaptation strategies from a human-centered perspective. To address these challenges, we propose a categorization framework for LLM adaptation that features two human-centered dimensions and stage LLM adaptation with respect to when it interacts with human intelligence. Additionally, we introduce an evaluation framework that incorporates a human-centered perspective that goes beyond the common machine-centered measures. Our empirical investigations, in which we use text classification as use cases, not only demonstrate the application of these frameworks but also compare various adaptation strategies. These artifacts and findings provide fresh insights and practical recommendations for selecting effective adaptation strategies to improve the efficacy of LLMs for intelligence augmentation. We further identify future research issues to address current limitations and suggest improvements for the proposed frameworks. [ABSTRACT FROM AUTHOR]

In language teaching and learning domains, evaluation plays a prominent role in visualizing the scope of progress and achievements. Therefore, evaluation occurs constantly in all teaching aspects (materials, content, pedagogical practices, and other related issues). However, evaluating materials remains complex owing to their distinctiveness. This complexity is attributed to the excessive application of Web-based resources in teaching and learning settings to create authentic learning opportunities. Consequently, evaluating materials' suitability requires guidance and practical frameworks that constitute common ground for evaluation. As technology offers a tremendous solution to a particular learning/teaching context, including Computer-Assisted Language Dedicated Apps, the question of how these apps fit into specific teaching/learning contexts remains controversial. However, the evaluation frameworks that Hubbard, Chapelle, Richards, and Rodgers developed have paved the way for more effective evaluation of CALL resources and applications. In light of this, the study attempts to take part in revealing the myth of CDAPPS evaluation by adopting the conceptual research methodology in association with a systematic review of the previous models for evaluating Computer-Assisted Language Learning Dedicated Applications where a conceptual and principled framework entitled Mudawe and Maslamani Framework is proposed. The proposed framework embraces four levels of analysis for evaluation: Learner/user fit, language professional Fit, Technology fit, and institutional administrators Fit. Each consideration contains several criteria associated with the main level of the analysis that can be used through judgmental or empirical evaluation. [ABSTRACT FROM AUTHOR]

Knowledge transfer initiatives aim to improve farmers’ knowledge and skills for tackling the growing economic, environmental and social challenges in agriculture. However, their evaluation is often inadequate due to institutional barriers and a lack of comprehensive methodology for assessing normative and behavioural shifts within this social group. To address this gap, we developed an extended evaluation framework for the knowledge transfer activities within the European Union's Common Agricultural Policy (CAP), including three indicators levels: output, result and impact. The indicators and survey instruments were based on the Theory of Planned Behaviour and piloted in a selected EU Member State for evaluating knowledge transfer measures in the programming period 2014–2022. The results indicate a promising potential of the proposed framework for capturing the multifaceted outcomes of knowledge transfer activities in agriculture. The usefulness and feasibility of the evaluation framework were further assessed through interviews with AKIS actors. They highlighted that evaluation activities at the Member State level may be hindered by inconsistent long-term data collection, insufficient response and involvement of farmers, deviations from the agreed protocols and a lack of dedicated resources. To ensure the uptake of the proposed evaluation framework, evaluation culture within the managing authorities and among stakeholders should be systematically fostered at the Member State level. This process can be enhanced by adopting a more elaborated CAP monitoring framework and methodology guidelines and by providing targeted training on evidence-driven policy-making.

The construction industry faces challenges in evaluating success due to inconsistent methods and a lack of uniformity. Traditional evaluation methods often result in subjective and biased results. Professionals, stakeholders, and researchers disagree on assessment techniques. The paper aims to identify stakeholder-responsive evaluation criteria, establish an evaluation criteria model, develop a quantitative evaluation algorithm, and provide an evaluation process framework. This method and framework fill traditional evaluation gaps, reduce subjectivity and bias, enhance accuracy, simplify evaluation, and are suitable for project evaluation. Data was gathered from experts and project stakeholders through focus group discussions, interviews, archive documents, and site observations. First, identify criteria and subcriteria and develop a criteria-relation model that achieves fitness indices and passes validity and reliability tests. Next, quantitative evaluation weighted coefficients were determined, and an algorithm was developed. A quantitative evaluation of a case study revealed that project seven has very high success; projects three, six, eight, and nine have high success; project two has medium success; projects one, four, and five have low success; and there is no project with a very low success. Projects achieved average success rates of 86.42, 79.6, 66.5, 71.17, 83.63, 81.27, and 89.14% in quality, cost, time, safety, impact, satisfaction, and lesson learning, respectively. This study is novel and original due to its criteria structural equation model, quantitative evaluation algorithm with inclusive criteria index, and unique evaluation processes framework. This enhances evaluation science, providing new insights for stakeholders, professionals, academics, and policymakers, and enhancing stakeholder communication in future project success evaluations.

Local government is underpinned by the principle that finding sustainable ways to meet the needs of communities through service delivery matters. Post 1994, the South African government opted for a ‘developmental’ local government system as a way of addressing the socio-economic challenges of historically disadvantaged communities. However, the scholarly discourse on service delivery in South Africa repeatedly highlights disappointing municipal performance. This study examined the need for performance evaluation of municipal service delivery in South Africa. Using a mixed methods approach and purposive sampling to collect data from 36 municipal managers of dysfunctional municipalities, the findings revealed that municipal performance was not evaluated satisfactorily; service delivery backlogs persisted and the impacts of deficient services on communities were not evaluated by either municipalities or communities. These findings justify a need for an effective evaluation framework for measuring municipal performance to assist municipalities to identify service delivery challenges and promote necessary change and improvement.

Despite the existence of many interventions to mitigate or adapt to the health effects of climate change, their effectiveness remains unclear. Here, we introduce the Comprehensive Evaluation Framework for Intervention on Health Effects of Ambient Temperature to evaluate study designs and effects of intervention studies. The framework comprises three types of interventions: proactive, indirect, and direct, and four categories of indicators: classification, methods, scope, and effects. We trialed the framework by an evaluation of existing intervention studies. The evaluation revealed that each intervention has its own applicable characteristics in terms of effectiveness, feasibility, and generalizability scores. We expanded the framework's potential by offering a list of intervention recommendations in different scenarios. Future applications are then explored to establish models of the relationship between study designs and intervention effects, facilitating effective interventions to address the health effects of ambient temperature under climate change.

The digital transformation of our health care system will require not only digitization of existing tools but also a redesign of our care delivery system and collaboration with digital partners. Traditional patient journeys are reactive to symptom presentation and delayed by health care system-centric scheduling, leading to poor experience and avoidable adverse outcomes. Patient journeys will be reimagined to a digital health pathway that seamlessly integrates various care experiences from telemedicine, remote monitoring, to in-person clinic visits. Through centering the care delivery around the patients, they can have more delightful experiences and enjoy the quality of standardized condition pathways and outcomes. To design and implement digital health pathways at scale, enterprise health care systems need to develop capabilities and partnerships in human-centered design, operational workflow, clinical content management, communication channels and mechanisms, reporting and analytics, standards-based integration, security and data management, and scalability. Using a human-centered design methodology, care pathways will be built upon an understanding of the unmet needs of the patients to have a more enjoyable experience of care with improved clinical outcomes. To power this digital care pathway, enterprises will choose to build or partner for clinical content management to operationalize up-to-date, best-in-class pathways. With this clinical engine, this digital solution will engage with patients through multimodal communication modalities, including written, audio, photo, or video, throughout the patient journey. Leadership teams will review reporting and analytics functions to track that the digital care pathways will be iterated to improve patient experience, clinical metrics, and operational efficiency. On the backend, standards-based integration will allow this system to be built in conjunction with the electronic medical record and other data systems to provide safe and efficient use of the digital care solution. For protecting patient information and compliance, a security and data management strategy is critical to derisking breeches and preserving privacy. Finally, a framework of technical scalability will allow digital care pathways to proliferate throughout the enterprise and support the entire patient population. This framework empowers enterprise health care systems to avoid collecting a fragmented series of one-off solutions but develop a sustainable concerted roadmap to the future of proactive intelligent patient care.

The advancement of Augmented Reality (AR) technology has been remarkable, enabling the augmentation of user perception with timely information. This progress holds great promise in the field of interaction design. However, the mere advancement of technology is not enough to ensure widespread adoption. The user dimension has been somewhat overlooked in AR research due to a lack of attention to user motivations, needs, usability, and perceived value. The critical aspects of AR technology tend to be overshadowed by the technology itself. To ensure appropriate future assessments, it is necessary to thoroughly examine and categorize all the methods used for AR technology validation. By identifying and classifying these evaluation methods, researchers and practitioners will be better equipped to develop and validate new AR techniques and applications. Therefore, comprehensive and systematic evaluations are critical to the advancement and sustainability of AR technology. This paper presents a theoretical framework derived from a cluster analysis of the most efficient evaluation methods for AR extracted from 399 papers. Evaluation methods were clustered according to the application domains and the human–computer interaction aspects to be investigated. This framework should facilitate rapid development cycles prioritizing user requirements, ultimately leading to groundbreaking interaction methods accessible to a broader audience beyond research and development centers. [ABSTRACT FROM AUTHOR]

To date, the primary challenge in the field of information and communication technologies-mediated rehabilitative interventions for autism is the lack of evidence regarding efficacy and effectiveness. Although such interventions, particularly those realised with Immersive Virtual Reality-based Serious Games, show promise, clinicians are hesitant to adopt them due to minimal evidence supporting their efficiency and effectiveness. Efficacy refers to whether an intervention produces the expected result under ideal circumstances, while effectiveness measures the degree of beneficial effect in real-world clinical settings. The absence of efficacy and effectiveness evidence undermines the reliability and generalisability of such interventions, which are crucial for real-life settings, making accurate evaluation pivotal. Evaluating the efficacy and effectiveness of these interventions poses a significant challenge due to the absence of evaluation guidelines. A previous study systematically reviewed the evaluation of Immersive Virtual Reality-based Serious Games for autism, revealing incomplete or methodologically problematic evaluation processes. This evidence underpinned the aim of the present study: to propose an Evaluation Framework encompassing all necessary methodological criteria for evaluating the efficacy and effectiveness of such interventions. Disseminating this Evaluation Framework as a pocket guide could facilitate the development of reliable future studies, thereby advancing evidence-based interventions to improve the quality of life for individuals with autism. [ABSTRACT FROM AUTHOR]

Background: Fire simulators are increasingly used to predict fire spread. Australian fire agencies have been concerned at not having an objective basis to choose simulators for this purpose. Aims: We evaluated wildland fire simulators currently used in Australia: Australis, Phoenix, Prometheus and Spark. The evaluation results are outlined here, together with the evaluation framework. Methods: Spatial metrics and visual aids were designed in consultation with simulator end-users to assess simulator performance. Simulations were compared against observations of fire progression data from 10 Australian historical fire case studies. For each case, baseline simulations were produced using as inputs fire ignition and fuel data together with gridded weather forecasts available at the time of the fire. Perturbed simulations supplemented baseline simulations to explore simulator sensitivity to input uncertainty. Key results: Each simulator showed strengths and weaknesses. Some simulators displayed greater sensitivity to different parameters under certain conditions. Conclusions: No simulator was clearly superior to others. The evaluation framework developed can facilitate future assessment of Australian fire simulators. Implications: Collection of fire behaviour observations for routine simulator evaluation using this framework would benefit future simulator development. Australian fire and land management agencies use fire simulators to predict characteristics of fires in the landscape, and require evaluation of simulator accuracy. Simulations of fire spread were assessed against observations. Inputs were perturbed to evaluate simulator sensitivity. Against the evaluation measures, no simulators proved universally superior to the others. [ABSTRACT FROM AUTHOR]

Transportation-related activity scheduling is becoming more complex due to the growing number of potential locations and extensive opportunities to visit various places. Throughout the years, in the field of transportation several attempts were made to optimize travelers’ activity chains with different parameters to set, but there is a lack of comprehensive solutions. In this research, the activity chain optimization algorithm is applied, which requires high computational efforts. To provide an adequate calibration of the parameters, a sensitivity analysis is conducted. The aim of the analysis is to reveal how changes in the attribute values modify the final outcomes. The relevant parameters, activity chains, transport modes, optimization algorithms, and fitness functions, are identified and considered. For each parameter, an investigation is conducted to reveal its behavior throughout the runs. For example, changes in the population size and crossover function lead to more reliable results, while alteration in the number of generations and the mutation function have no effects on the outcomes. The analysis presents a peculiar behavior of the parameters related to the activity chains. The results can be useful for transportation planners and service providers in the adaptation of the existing network and transportation services to the travelers’ mobility patterns.

Accurately predicting the concentration of fine particulate matter (PM2.5) is crucial for evaluating air pollution levels and public exposure. Recent advancements have seen a significant rise in using deep learning (DL) models for forecasting PM2.5 concentrations. Nonetheless, there is a lack of unified and standardized frameworks for assessing the performance of DL-based PM2.5 prediction models. Here we extensively reviewed those DL-based hybrid models for forecasting PM2.5 levels according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We examined the similarities and differences among various DL models in predicting PM2.5 by comparing their complexity and effectiveness. We categorized PM2.5 DL methodologies into seven types based on performance and application conditions, including four types of DL-based models and three types of hybrid learning models. Our research indicates that established deep learning architectures are commonly used and respected for their efficiency. However, many of these models often fall short in terms of innovation and interpretability. Conversely, models hybrid with traditional approaches, like deterministic and statistical models, exhibit high interpretability but compromise on accuracy and speed. Besides, hybrid DL models, representing the pinnacle of innovation among the studied models, encounter issues with interpretability. We introduce a novel three-dimensional evaluation framework, i.e., Dataset-Method-Experiment Standard (DMES) to unify and standardize the evaluation for PM2.5 predictions using DL models. This review provides a framework for future evaluations of DL-based models, which could inspire researchers to standardize DL model usage in PM2.5 prediction and improve the quality of related studies.

Water security is a multi-faceted concept that encompasses dimensions such as water quantity, quality, human health, well-being, water hazards, and governance. The evaluation of water security is an important step towards understanding and improving it, particularly in urban settings where disparities resulting from unequal distribution of population and resources are present and often evade citywide assessments. To address the diversity of the urban space, we propose a multi-level assessment approach based on downscaling the spatial dimension. Using a comprehensive indicator-based framework, we evaluate the city of Campinas in Brazil across citywide and intra-city scales. Employing the Theil index to measure inequality, the results reveal nuanced disparities less apparent at broader scales. Despite an overall favourable water security condition, spatial heterogeneity is still noticeable in the urban area of Campinas. The methodology highlights different aspects, such as vegetation cover, social green areas, and wastewater collection, which are inequitably distributed in the urban area. This integrated approach, linking inequality and water security assessment, has the potential to unveil specific needs within urban areas, helping guide targeted interventions to improve water security levels for all. HIGHLIGHTS A multi-disciplinary urban security assessment framework is applied through three scales in the urban area: citywide, regions and sector.; Theil index is used to measure inequality in terms of water security.; Downscaling the urban water security analysis facilitates the exposure of disparities, providing useful information for decision-makers on local needs.;

This paper presents a systematic approach to enhancing ISO 26262, a widely adopted standard for automotive functional safety, by integrating Machine Learning (ML)-specific life cycle phases and testing methods for Automotive Safety Integrity Level (ASIL) A/B. With the increasing incorporation of ML techniques in automotive systems, the current ISO 26262 framework reveals significant gaps in addressing ML-specific safety requirements. While ISO/DPAS 8800 provides an approach for developing AI systems that meet some safety properties, it does not provide a mapping concept for ASIL classification of ML systems. Furthermore, given the complexity of ML techniques in automotive systems, issues such as interpretability—critical for transparency and accountability—along with robustness and uncertainty handling, pose significant challenges that are not fully addressed by ISO 26262 and ISO/DPAS 8800. This study identifies and addresses these gaps by defining three additional life cycle phases: prepare data, train ML model, and deploy ML model. For each life cycle phase, we establish desired properties such as robustness, uncertainty handling, and interpretability, and propose suitable methods to achieve these properties. We adopt a rigorous evaluation framework inspired by IEC 61508 to assess the effectiveness of these methods. Since the method recommendations of ISO 26262 for ML-based products are incomplete, the approach presented in this paper provides critical guidance and room for expert assessment and independent certification, ensuring solid and reliable recommendations. This systematic, clear, uniform development procedure not only supports product teams in achieving their safety goals but also facilitates the certification process, reducing ambiguity and enhancing the overall safety and reliability of ML-based automotive systems.

Background Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated. Methods SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation. Results This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings. Conclusions The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

The concept of immersion has been widely used for the design and evaluation of user experiences. Augmented, virtual and mixed-reality environments have further sparked the discussion of immersive user experiences and underlying requirements. However, a clear definition and agreement on design criteria of immersive experiences remains debated, creating challenges to advancing our understanding of immersive experiences and how these can be designed. Based on a multidisciplinary Delphi approach, this study provides a uniform definition of immersive experiences and identifies key criteria for the design and staging thereof. Thematic analysis revealed five key themes – transition into/out of the environment, in-experience user control, environment design, user context relatedness, and user openness and motivation, that emphasise the coherency in the user-environment interaction in the immersive experience. The study proposes an immersive experience framework as a guideline for industry practitioners, outlining key design criteria for four distinct facilitators of immersive experiences – systems, spatial, empathic/social, and narrative/sequential immersion. Further research is proposed using the immersive experience framework to investigate the hierarchy of user senses to optimise experiences that blend physical and digital environments and to study triggered, desired and undesired effects on user attitude and behaviour. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Capital Institute of Physical Education is the property of Shoudu Tiyu Xueyuan and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The use of Deep Learning models to forecast geomagnetic storms is achieving great results. However, the evaluation of these models is mainly supported on generic regression metrics (such as the Root Mean Squared Error or the Coefficient of Determination), which are not able to properly capture the specific particularities of geomagnetic storms forecasting. Particularly, they do not provide insights during the high activity periods. To overcome this issue, we introduce the Binned Forecasting Error to provide a more accurate assessment of models' performance across the different intensity levels of a geomagnetic storm. This metric facilitates a robust comparison of different forecasting models, presenting a true representation of a model's predictive capabilities while being resilient to different storms duration. In this direction, for enabling fair comparison among models, it is important to standardize the sets of geomagnetic storms for model training, validation and testing. To do this, we have started from the current sets used in the literature for forecasting the SYM‐H, enriching them with newer storms not considered previously, focusing not only on disturbances caused by Coronal Mass Ejections but also addressing High‐Speed Streams. To operationalize the evaluation framework, a comparative study is conducted between a baseline neural network model and a persistence model, showcasing the effectiveness of the new metric in evaluating forecasting performance during intense geomagnetic storms. Finally, we propose the use of preliminary measurements from ACE to evaluate the model performance in settings closer to an operational real‐time scenario, where the forecasting models are expected to operate. Plain Language Summary: Using Machine Learning (ML) for forecasting geomagnetic indices has seen a notable increase in popularity, especially with the constantly increasing available data from space probes and ground‐based magnetometers. One critical aspect that defines the success of ML models in this domain is the quality and representativeness of the data. The evaluation of previous models in the field often relied in conventional metrics that did not give a complete evaluation of the particularities of the geomagnetic storms. To address this, we introduce the Binned Forecasting Error, a metric designed to provide a more rigorous and complete evaluation of model performance, taking into account the particular characteristics of geomagnetic storms and the challenges inherent in predicting them. Considering that, we have reviewed and expanded the selected storms for the SYM‐H index to include a broader spectrum of geomagnetic events. Plus, we suggest using preliminary data from ACE to test these models in more realistic scenarios, similar to how they would be used in an operational scenario. Key Points: A common framework for evaluating geomagnetic index forecasting models improves the quality of research in a growing fieldWe introduce the Binned Forecasting Error for better capturing the relevance of large geomagnetic storms in forecasting modelsWe expand current assessed geomagnetic storms sets to improve training and testing of geomagnetic forecasting models [ABSTRACT FROM AUTHOR]

Bridges are critical components of transportation systems and are susceptible to various natural and man-made disasters throughout their lifecycle. With the rapid development of the transportation industry, the frequency of vehicle-induced disasters has been steadily increasing. These incidents not only result in structural damage to bridges but also have the potential to cause traffic interruptions, weaken social service functions, and impose significant economic losses. In recent years, research on resilience has become a new focus in civil engineering disaster prevention and mitigation. This study proposes a concept of generalized bridge resilience and presents an evaluation framework for cable-stayed bridges under disasters. The framework includes a resilience evaluation indicator system from multiple dimensions, including safety, society, environment, and economy, which facilitates the dynamic and comprehensive control of bridge resilience throughout its entire lifecycle with the ultimate goals of enhancing structural safety and economic efficiency while promoting the development of environmentally friendly structural ecosystems. Furthermore, considering the influence of recovery speed, the study evaluates various repair strategies through resilience assessment, revealing the applicable environments and conditions for different repair strategies. This methodology offers significant implications for enhancing the safety, efficiency, and environmental sustainability of infrastructure systems, providing valuable guidance for future research in this field. [ABSTRACT FROM AUTHOR]

The uptake of complex technologies and platforms has resulted in several challenges in the pricing and reimbursement of innovative pharmaceuticals. To address these challenges, plenty of concepts have already been described in the scientific literature about innovative value judgment or payment models, which are either (1) remaining theoretical; or (2) applied only in pilots with limited impact on patient access; or (3) applied so heterogeneously in many different countries that it prevents the health care industry from meeting expectations of HTA bodies and health care payers in the evidence requirements or offerings in different jurisdictions. This paper provides perspectives on how to reduce the heterogeneity of pharmaceutical payment models across European countries in five areas, including 1) extended evaluation frameworks, 2) performance-based risk-sharing agreements, 3) pooled procurement for low volume or urgent technologies, 4) alternative access schemes, and 5) delayed payment models for technologies with high upfront costs. Whilst pricing and reimbursement decisions will remain a competence of EU member states, there is a need for alignment of European pharmaceutical payment model components in critical areas with the ultimate objective of improving the equitable access of European patients to increasingly complex pharmaceutical technologies. [ABSTRACT FROM AUTHOR]

As a significant manifestation of human activities influencing natural environment, rapid urbanization has enhanced economic prosperity while simultaneously posing threats to ecological quality. Beijing–Tianjin–Hebei (BTH), the core region of the Yangtze River Delta (CYRD), and the Pearl River Delta (PRD) stand as three major economic centers characterized by the highest level of urbanization in China, encompassing areas of heightened ecological sensitivity. Nevertheless, the ecological quality at the scale of urban agglomerations remains ambiguous, with many studies failing to develop a comprehensive and effective method for comparing diverse urban agglomerations. Consequently, this study integrates multi-source remote sensing data, including information on land cover and other socio-economic parameters, to construct the Ecological Quality Index (EQI) based on the "Function–Interaction–Pressure–Stability" (FIPS) framework. Through a stratified determination of indicator weights grounded in both objective importance and empirical knowledge, we mapped the spatiotemporal changes of EQI and analyzed the impact of urbanization on ecological quality in three urban agglomerations from 2001 to 2020. We determined the following: (1) The calculated EQI can further capture the nuanced details with better performance at both underlining the discrepancy of highs and lows of EQI and describing the spatial detail of urban agglomerations' characteristics. (2) Substantial disparities in EQI and its changes are evident across different urban agglomerations. Notably, only the average EQI improves in PRD, while ecological degradation is prominent in specific regions, such as the southeastern plains of BTH area, along the Yangtze River, and around Shanghai in CYRD and central PRD. The CYRD exhibits the largest affected area. (3) Urbanization predominantly influences ecological quality through land cover transitions. In expansion areas, ecological deterioration is significantly more pronounced, constituting approximately 90% of the total area. (4) Despite significant urbanization, city-level analysis in CYRD reveals a better coordination between urban expansion and ecological protection, with a lower intensity of ecological degradation compared to urban expansion rates. Conversely, some cities in the BTH, despite modest urban expansion, exhibit substantial declines in ecological quality, highlighting the need for targeted policy interventions. In conclusion, this study elucidates the intricate relationship between urbanization and ecological quality, offering valuable insights for the development of targeted protection strategies and sustainable urban planning. [ABSTRACT FROM AUTHOR]

The construction industry faces challenges in evaluating success due to inconsistent methods and a lack of uniformity. Traditional evaluation methods often result in subjective and biased results. Professionals, stakeholders, and researchers disagree on assessment techniques. The paper aims to identify stakeholder-responsive evaluation criteria, establish an evaluation criteria model, develop a quantitative evaluation algorithm, and provide an evaluation process framework. This method and framework fill traditional evaluation gaps, reduce subjectivity and bias, enhance accuracy, simplify evaluation, and are suitable for project evaluation. Data was gathered from experts and project stakeholders through focus group discussions, interviews, archive documents, and site observations. First, identify criteria and subcriteria and develop a criteria-relation model that achieves fitness indices and passes validity and reliability tests. Next, quantitative evaluation weighted coefficients were determined, and an algorithm was developed. A quantitative evaluation of a case study revealed that project seven has very high success; projects three, six, eight, and nine have high success; project two has medium success; projects one, four, and five have low success; and there is no project with a very low success. Projects achieved average success rates of 86.42, 79.6, 66.5, 71.17, 83.63, 81.27, and 89.14% in quality, cost, time, safety, impact, satisfaction, and lesson learning, respectively. This study is novel and original due to its criteria structural equation model, quantitative evaluation algorithm with inclusive criteria index, and unique evaluation processes framework. This enhances evaluation science, providing new insights for stakeholders, professionals, academics, and policymakers, and enhancing stakeholder communication in future project success evaluations. [ABSTRACT FROM AUTHOR]

European cities are motivated to act towards the achievement of climate-neutral mobility solutions. Often, though, they are facing many challenges when bringing (innovative) sustainable mobility solutions forward. Capacity building that fills the skills gaps and/or enables the acquisition of new ones related to the planning and implementation of such solutions can empower local/regional authorities to identify them, adopt them and eventually deliver them properly. The aim of this paper is to present the Key Performance Indicator (KPI) framework that has been used for the assessment of the effectiveness of the Learning and Exchange Programme applied in an EU-funded project. It presents the methodological steps for the adoption of the KPIs, as well as the tools used for the selection of the KPI data and the KPI monitoring at the project level. It also presents the results from the application of the framework for assessing the knowledge performance towards the deployment of sustainable mobility solutions. It finally reflects on recommendations for applying the KPI framework to other cases and thematic contents. [ABSTRACT FROM AUTHOR]

Background Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated. Methods SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation. Results This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings. Conclusions The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

The integration of wastewater heat recovery (WWHR) and wastewater reuse offers a numerous advantage, making its application possible in various sectors. Nevertheless, this concept faced challenges to the identification of appropriate location. Existing research lacks comprehensive evaluation methods that encompass a various factor for effective decision-making. This study introduces a new evaluation framework that involves different aspects, including thermal energy potential and spatial distribution analysis. The novelty of this research lies in its unique focus on the combination of WWHR and wastewater reuse. Moreover, it introduces a structured evaluation framework that considers multiple criteria and expert opinions, enhancing decision-making precision. Multi-criteria decision analysis (MCDA) was applied to select assessment criteria, which were categorized into three aspects: water–energy supplier, water–energy consumers, and water–energy station. The relative importance of criteria was determined using the analytical hierarchical process (AHP). The results of the AHP highlight significance of factors: treated wastewater flow rate; treated wastewater temperature; water–energy supply distance, and type of water–energy consumer. These factors were assigned weight values of 0.297, 0.186, 0.123, and 0.096, respectively. It is emphasizing their influence in the decision-making process that potential locations depend on the water–energy supplier and water–energy consumer as supply and demand sources. HIGHLIGHTS The integration of heat recovery with treated wastewater reuse creates a powerful nexus between essential resources.; An evaluation framework is a tool, empowering thoughtful water–energy planning and management.; Water–energy supplier, consumer, and station stand as the key factor, shaping the future of water–energy integration.; The analysis framework offers a comprehensive evaluation of optimal site selection.;