Search

To uncover the evaluation information on the academic contribution of research papers cited by peers based on the content cited by citing papers, and to provide an evidence-based tool for evaluating the academic value of cited papers.

Objectives: This study aimed to evaluate a microfluidic paper-based analytical device (DEN-NS1-PAD) based on a rapid NS1 antigen test for diagnosing dengue at the point of care. Methods: 219 serum samples from suspected dengue cases were tested with the developed DEN-NS1-PAD and commercial RDT by SD BIOLINE. The results were compared with the nested-PCR results. Results: The limit of detection of DEN-NS1-PAD was 0.78 ng mL−1. It showed 88.89% sensitivity, 86.67% specificity, and a substantial agreement correlation (κ = 0.7522) compared with nested-PCR. In contrast, SD BIOLINE for NS1 (SD-NS1) detection showed 87.88% sensitivity, 90.00% specificity, and had a substantial agreement correlation with nested-PCR (κ = 0.7788). Conclusions: DEN-NS1-PAD is a valuable tool for diagnosing DENV infections, especially for diagnosed patients with early acute phase samples with high viral load. DEN-NS1-PAD has better sensitivity than SD-NS1 but less specificity.

Background: Only a few studies with small sample sizes have compared electronic Objective Structured Clinical Examination (OSCE) rating checklists with traditional paper-based OSCE rating checklists. In this study, the examiner-perceived usability and preference for type of OSCE checklist (electronic vs. paper based) were compared, and the influence of OSCE checklist type on missed ratings was determined, for the Swiss Federal Licensing Examination in clinical skills for human medicine.Methods: All examiners in the Swiss Federal Licensing Examination in clinical skills for human medicine were invited over two subsequent years to evaluate the OSCE checklist type they had worked with during the examination. This was based on a questionnaire with 14 closed questions (i.e., demographic, checklist-type experience, perceived usability, checklist type preference). Furthermore, the numbers of missed ratings for the paper-based checklist were recorded.Results: The data from these examiners (=377) with experience of both OSCE checklist types were analyzed. The electronic OSCE checklist was rated significantly higher on all usability aspects (i.e., ease of use, candidate rating and error correction, clarity, distraction using the checklist, overall satisfaction), except for the speed of registering comments (no significant difference). The majority of the examiners in both years (2014: 54.5%, =60, 2015: 89.8%, =230) reported preference for working with the electronic OSCE checklist in the future. Missed ratings were seen for 14.2% of the paper-based OSCE checklists, which were prevented with the electronic OSCE checklists.Conclusions: Electronic OSCE checklists were rated significantly more user-friendly and were preferred over paper-based OSCE checklists by a broad national sample of examiners, supporting previous results from faculty-level examinations. Furthermore, missed ratings were prevented with the electronic OSCE checklists. Overall, the use of electronic OSCE checklists is therefore advisable.

In the wake of local initiatives and developmental funding programs, interprofessionality is now included in national curricula in the German-speaking countries. Based on the 3P model (presage, process, product), this position paper presents the development of interprofessional education in recent years in Germany, Austria and Switzerland and places it in an international context. Core aspects as legal frameworks, including amendments to occupational regulations as well as the formation of networks and faculty development are basic requirements for interprofessional education. New topics and educational settings take shape in the process of interprofessional education: patient perspectives and teaching formats, such as online courses, become more important or are newly established. The influence of the COVID-19 pandemic on interprofessional education is explored as well. Among many new interprofessional courses, particularly the implementation of interprofessional training wards in Germany and Switzerland are positive examples of successful interprofessional education. The objective of interprofessional education continues to be the acquisition of interprofessional competencies. The main focus is now centered on evaluating this educational format and testing for the corresponding competencies. In the future, more capacities will be required for interprofessional continuing education and post-graduate education. Structured research programs are essential to ascertain the effects of interprofessional education in the German-speaking countries.In this position paper the GMA committee on interprofessional education encourages further advancement of this topic and expresses the aim to continue cooperating with other networks to strengthen and intensify interprofessional education and collaboration in healthcare.

The goal of medical education is to produce the physician we would like to see if we are sick - Melinkoff In all educational developments, one of the most troublesome questions is that of evaluation. Written examination is the most widely used tool in evaluation and assessment of the competency of the medical students. At present, questions are prepared casually just before the examination and are not subjected to any quality check, which may lead to confusion or wrong understanding of the questions by the students. This article is aimed at providing guidelines and a scientific method to frame good question papers to improve the quality of evaluation and assessment of medical students. J-GMC-N | Volume 11 | Issue 01 | January-June 2018, Page: 64-67

The administration of A/B exams usually involves the use of items. Issues arise when the pre-establishment of a question bank is necessary and the inconsistency in the knowledge points to be tested (in the two exams) reduces the exams ‘fairness’. These are critical for a large multi-teacher course wherein the teachers are changed such that the course and examination content are altered every few years. However, a fair test with randomly participating students should still be a guaranteed subject with no item pool. Through data-driven decision-making, this study collected data related to a term test for a compulsory general course for empirical assessments, pre-processed the data and used item response theory to statistically estimate the difficulty, discrimination and lower asymptotic for each item in the two exam papers. Binary goal programing was finally used to analyze and balance the fairness of A/B exams without an item pool. As a result, pairs of associated questions in the two exam papers were optimized in terms of their overall balance in three dimensions (as the goals) through the paired exchanges of items. These exam papers guarantee their consistency (in the tested knowledge points) and also ensure the fairness of the term test (a key psychological factor that motivates continued studies). Such an application is novel as the teacher(s) did not have a pre-set question bank and could formulate the fairest strategy for the A/B exam papers. The model can be employed to address similar teaching practice issues.

AIM: To investigate the published papers of ophthalmology in past ten years and explore the development of ophthalmology. RESULTS: The number of ophthalmology papers increased from 7450 to 9089 during 2007 to 2017. The average rate increased 2.2% annually. USA accounts for one thirds of the total and two thirds of the highly cited papers. In Asia, China, Japan and South Korea were in Top 10 by the number of ophthalmology papers. UK, Germany, Japan and Australia also had great impact in global ophthalmology. The hot spots included endothelial growth factor, optical coherence tomography and open-angle glaucoma. CONCLUSION: USA is in the leading position in global ophthalmology. Part of Asian countries play an important role in the development of ophthalmology, but the impact needs to be improved.

This study evaluated pen and paper examinations development with a view to improving examination validity in open and distance education. The study specifically examined the appropriateness, meaningfulness, and usefulness of pen on paper examinations in the National Open University of Nigeria (NOUN). The study employed the survey research design with a view to reaching a large number of respondents. In all, a total of 122 lecturers formed the respondents used for the study. A self-structured questionnaire was constructed and validated for data collection. Three research questions and two hypotheses were formulated to guide the study. The data were analyzed using the SPSS statistical tool. Descriptive statistics (mean and standard deviation) were used to answer the research questions while inferential statistics (t-test and ANOVA) were used to test the hypotheses at .05 level. The results show that valid procedures of item development and generation are always followed in setting pen and paper examinations in NOUN. Despite this, counsellors need to work on students and staff regularly on the rubrics of examinations to ensure both student and institutional success. The study, therefore, recommended appropriate procedures using Bloom’s taxonomy for test development. Résumé : Cette étude a évalué le développement des examens au stylo et au papier en vue d'améliorer la validité de l'examen dans l'enseignement ouvert et à distance. L'étude a spécifiquement examiné la pertinence, la signification et l'utilité des examens au stylo et au papier de l'Université nationale ouverte du Nigeria (NOUN). L'étude a utilisé la conception de la recherche de l'enquête en vue d'atteindre un grand nombre de répondants. Au total, 122 professeurs ont formé le nombre de répondants utilisés pour l'étude. Un questionnaire auto structuré a été construit et validé pour la collecte de données. Trois questions de recherche et deux hypothèses ont été formulées pour guider l'étude. Les données ont été analysées à l'aide de l'outil statistique SPSS. Des statistiques descriptives (écart moyen et standard) ont été utilisées pour répondre aux questions de recherche, tandis que des statistiques inférentielles (t-test et ANOVA) ont été utilisées pour tester les hypothèses au niveau de 0,05. Les résultats montrent que les procédures valides de développement et de génération d'objets sont toujours suivies dans la mise en place d'examens au stylo et au papier à NOUN. Malgré cela, les conseillers doivent travailler régulièrement sur les rubriques des examens sur les étudiants et le personnel afin d'assurer la réussite des étudiants et de l’institution. L'étude a donc recommandé des procédures appropriées utilisant la taxonomie de Bloom pour le développement des tests.

This article describes a multi-method approach for usability testing. The approach combines paper prototyping and think-aloud with two supplemental methods: advanced scribbling and a handicraft task. The method of advanced scribbling instructs the participants to use different colors for marking important, unnecessary and confusing elements in a paper prototype. In the handicraft task the participants have to tinker a paper prototype of their wish version. Both methods deliver additional information on the needs and expectations of the potential users and provide helpful indicators for clarifying complex or contradictory findings. The multi-method approach and its surplus benefit are illustrated by a pilot study on the redesign of the homepage of a library 2.0. The findings provide positive evidence for the applicability of the advanced scribbling and the handicraft task as well as for the surplus merit of the multi-method approach. The article closes with a discussion and outlook. URN: http://nbn-resolving.de/urn:nbn:de:0114-fqs150379

We define a Landmark Paper Index (LPI), calculate and analyze indices for all papers published in rheological journals (‘η-journals’) between 1990 and 2006. This paper offers some information about the criteria influencing the impact of publications on the (scientific) community. In opposite to the well known Impact Factor (journal sensitive) or the number of citations (article sensitive, publication year insensitive) the LPI helps to identify established and potential breakthrough contributions by considering the number of citations per year after publication, in a way which does not overestimate the few, highly cited, articles when performing averages. We discuss the effect of formal criteria on the LPI.

Introduction: Graduate (PhD) students in medical sciences, who will form future faculties and investigators in Iran’s Universities of Medical Sciences, are not trained on scientific writing during their training. The present study describes the design, implementation, and evaluation of Principles of Writing Biomedical Research Paper course. Methods: The course, prepared based on an extensive search of the literature and books on writing biomedical research papers, was offered as an elective course to PhD students at Shiraz University of Medical Sciences in the second semester of 2011-2012 academic year. The structure and function of various sections of a paper and publication ethics were discussed in lecture and practical sessions over a period of 12 weeks. The course was then evaluated using a self-designed questionnaire. Results: The majority of students gave the highest score (20) to the content and implementation of all sessions of the course. Moreover, most of them believed that the allotted time to the course was not enough, and suggested that it should be increased to 32 hours (equal to two credits). Also, almost all the participants believed that overall the materials lectured were comprehensive, the practical sessions were important in learning the lectured materials, and the course was useful in advancing their abilities and skills to write papers. Conclusion: The evaluation of the present course showed that it was able to increase the participants’ knowledge of the structure of scientific papers, and enhanced their abilities and skills to write papers. The evaluation was used as a basis to modify the course.

We apply the Landmark Paper Index (LPI), calculate and analyze indices for all papers published in rheological journals (‘η-journals’) between 1991 and 2007. We discuss the effect of formal criteria on the LPI.

To evaluate the usefulness of computerized adaptive testing (CAT) in medical school, the General Examination for senior medical students was administered as a paper and pencil test (P&P) and using CAT. The General Examination is a graduate examination, which is also a preliminary examination for the Korean Medical Licensing Examination (KMLE). The correlations between the results of the CAT and P&P and KMLE were analyzed. The correlation between the CAT and P&P was 0.8013 (p=0.000); that between the CAT and P&P was 0.7861 (p=0.000); and that between the CAT and KMLE was 0.6436 (p=0.000). Six out of 12 students with an ability estimate below 0.52 failed the KMLE. The results showed that CAT could replace P&P in medical school. The ability of CAT to predict whether students would pass the KMLE was 0.5 when the criterion of the theta value was set at -0.52 that was chosen arbitrarily for the prediction of pass or failure.

Abstract Background Evidence suggests that prehabilitation interventions, which optimise physical and mental health prior to treatment, can improve outcomes for surgical cancer patients and save costs to the health system through faster recovery and fewer complications. However, robust, theory-based evaluations of these programmes are needed. Using a theory of change (ToC) approach can guide evaluation plans by describing how and why a programme is expected to work. Theories of Change have not been developed for cancer prehabilitation programmes in the literature to date. This paper aims to provide an overview of the methodological steps we used to retrospectively construct a ToC for Prehab2Rehab (P2R), a cancer prehabilitation programme being implemented by the Cardiff and Vale University Health Board. Methods We used an iterative, participatory approach to develop the ToC. Following a literature review and document analysis, we facilitated a workshop with fourteen stakeholders from across the programme using a ‘backwards mapping’ approach. After the workshop, stakeholders had three additional opportunities to refine and validate a final working version of the ToC. Results Our process resulted in the effective and timely development of a ToC. The ToC captures how P2R’s interventions or activities are expected to bring about short, medium and long-term outcomes that, collectively, should result in the overarching desired impacts of the programme, which were improved patient flow and reduced costs to the health system. The process of developing a ToC also enabled us to have a better understanding of the programme and build rapport with key stakeholders. Conclusions The ToC has guided the design of an evaluation that covers the complexity of P2R and will generate lessons for policy and clinical practice on supporting surgical cancer patients in Wales and beyond. We recommend that evaluators apply a ToC development process at the outset of evaluations to bring together stakeholders and enhance the utilisation of the findings. This paper details a pragmatic, efficient and replicable process that evaluators could adopt to develop a ToC. Theory-informed evaluations may provide better evidence to develop and refine cancer prehabilitation interventions and other complex public health interventions.

This paper shows how resilience-based measurements, Industrial Resilience Index (IRI), is able to indicate the performance trend of general manufacturing, measured in Gross Domestic Product (GDP), impacted by shocks represented by the value drops of the Rupiah to the US Dollar. This paper argues that IRI is able to measure not only the resilience of the Metal Product Manufacturing Sector (MPMS) but also the performance dynamic of the general manufacturing industry. This study evaluates the IRI performance by using the cross-correlation method. The cross-correlation process consists of a comparison between IRI and the GDP of the manufacturing industry, as well as a comparison to other indices related to manufacturing sectors, such as the Purchasing Manager Index (PMI), the Production Index of Large and Medium Manufacturing Industry (PII), the Competitiveness Industrial Performance (CIP), and the Global Competitiveness Index (GCI). The positive and high value of the correlations in this study shows IRI’s ability to reflect the sector resilience and the GDP of the general manufacturing industry trend. The result of this study suggests that IRI can be utilized as a dynamic indicator of the general manufacturing industry. Through its data series and trend analysis, decision or policymakers may employ IRI to forecast how resilient MPMS, as well as the general manufacturing industry trend, is when the sector faces shocks in the future. The result of the study shows that cross-correlation coefficient of IRI is 0.74. The coefficient value indicates that IRI is a coincident indicator within the business cycles of the general manufacturing industry. Therefore, as an alternative of resilience-based measurement, the study suggests that IRI is able to demonstrate its significance in predicting the resilience of MPMS and the general manufacturing industry, in anticipating a dynamic shock is in the future.

Abstract Background Co-design with consumers and healthcare professionals is widely used in applied health research. While this approach appears to be ethically the right thing to do, a rigorous evaluation of its process and impact is frequently missing. Evaluation of research co-design is important to identify areas of improvement in the methods and processes, as well as to determine whether research co-design leads to better outcomes. We aimed to build on current literature to develop a framework to assist researchers with the evaluation of co-design processes and impacts. Methods A multifaceted, iterative approach, including three steps, was undertaken to develop a Co-design Evaluation Framework: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The systematic overview of reviews included relevant papers published between 2000 and 2022. OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews were searched for papers that reported co-design evaluation or outcomes in health research. Extracted data was inductively analysed and evaluation themes were identified. Review findings were presented to a stakeholder panel, including consumers, healthcare professionals and researchers, to interpret and critique. A consensus meeting, including a nominal group technique, was applied to agree upon the Co-design Evaluation Framework. Results A total of 51 reviews were included in the systematic overview of reviews. Fifteen evaluation themes were identified and grouped into the following seven clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. If evaluation methods were mentioned, they mainly included qualitative data, informal consumer feedback and researchers’ reflections. The Co-Design Evaluation Framework used a tree metaphor to represent the processes and people in the co-design group (below-ground), underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in the tree. Conclusions The Co-Design Evaluation Framework has been collaboratively developed with various stakeholders to be used prospectively (planning for evaluation), concurrently (making adjustments during the co-design process) and retrospectively (reviewing past co-design efforts to inform future activities).

Abstract Language models and other recent machine learning paradigms blur the distinction between generative and discriminative tasks, in a continuum that is regulated by the degree of pre- and post-supervision that is required from users, as well as the tolerated level of error. In few-shot inference, we need to find a trade-off between the number and cost of the solved examples that have to be supplied, those that have to be inspected (some of them accurate but others needing correction) and those that are wrong but pass undetected. In this paper, we define a new Supply-Inspect Cost Framework, associated graphical representations and comprehensive metrics that consider all these elements. To optimise few-shot inference under specific operating conditions, we introduce novel algorithms that go beyond the concept of rejection rules in both static and dynamic contexts. We illustrate the effectiveness of all these elements for a transformative domain, data wrangling, for which language models can have a huge impact if we are able to properly regulate the reliability-usability trade-off, as we do in this paper.

In the realm of intelligent sports, the integration of triboelectric nanogenerators (TENGs) marks a transformative approach toward energy sustainability and more advanced athletic monitoring. By leveraging the principle of triboelectricity, TENGs ingeniously convert mechanical energy from athletes’ movements into electrical energy, which offers a green and efficient power solution for wearable technology. This paper presents an innovative study on the application of TENG technology in sports science, with the results illustrating the potential utility of TENGs in revolutionizing the way we monitor, analyze, and enhance athletic performance. Through the development of self-powered wearables and equipment, TENGs facilitate real-time data collection on physiological and biomechanical parameters, ultimately enabling personalized training adjustments and injury prevention strategies. Our findings underscore the dual benefit of TENGs in promoting environmental sustainability by reducing the overall reliance on traditional energy sources and growing the capabilities of intelligent sports systems. This research contributes to the burgeoning field of nano-energy sports applications while setting the stage for future explorations into the optimization of TENG integration in athletic performance enhancement. Finally, the paper concludes by discussing remaining challenges in this area and opportunities for further research.

This paper outlines the innovative integration and evaluation of a for-credit Life Design module within the formal curriculum of a university in Ireland, the first time Life Design has been introduced for academic assessment in this context. The module forms part of a larger project at the University of Galway, Designing Futures (DF), which has been funded by the Irish Government (€7.5m, 2020-2025) to support student entrepreneurship, innovation and cross-disciplinary and research-led learning. Furthermore, DF is concerned with the rounded and holistic formation of the student, including supporting them in discerning their personal and professional life goals. The Life Design module facilitates this specific aspect of the DF project, engaging students with a diverse set of Life Design tools as they encounter key life concepts and questions, helping them to figure out how to get more out of the college experience and what they might choose to do upon graduation. This article is focusing on how the Design Your Life module has been designed and refined since its introduction in 2021-2022. The iterative design of the Life Design module has been underpinned by assessment and evaluation. The discussion illustrates how student feedback and learning have been ascertained and assessed. This includes the use of innovative digital storytelling as a narrative mode of assessment, one that we have found is well-suited to the process-oriented, personal and professional development goals of Life Design. While this paper predominantly focuses on undergraduate education in a university setting, the assessment and evaluation strategies, heuristics and digital storytelling outlined here can be adopted and adapted to develop and enhance Life Design innovations in diverse contexts beyond college education.

IntroductionThis paper introduces the concept of Sustainable Digital Rent (SDR), highlighting the shift from traditional economic rent based on tangible assets to rent derived from digital platforms. At the heart of this shift is the “value state,” a dynamic balance between constructive expectations and destructive information. As digital platforms generate increasing amounts of information, expectations are increasingly met and shared more efficiently with all users, leading to a reduction in individual and general motivational, emotional, and cognitive engagement. These platforms, now essential to modern life, facilitate online activities that reduce as well physical engagement and natural interactions, thereby impacting cognitive function and physical health. By extracting rent directly, digital platform operators limit the benefits users could gain to support their mental and physical well-being.MethodsThis paper empirically defines and estimates SDR using the collective estimates of price, cost, and income (PCI) as practiced in North American real estate appraisal, demonstrated through abstract art rent. Our approach provides a new perspective on valuing intangible assets, such as knowledge, by showing the shift from expectation to information, governed by the value state in cognitive evaluations. Emphasizing interdisciplinary relevance, the method underscores the need for an efficient mechanism to redistribute SDR benefits to digital platform users, supporting fair and equitable digital development.Results and discussionThe results show that digital rent is driven primarily by cognitive and informational content, demonstrating the need for redistribution mechanisms to address the growing inequality on digital platforms. The use of abstract art as a case study provides a convenient and illustrative way to explore how intangible assets, like digital rents, can be evaluated and redistributed. SDR offers insights into how digital rents can be captured and redistributed equitably, ensuring that platform users and creators benefit from the knowledge economy’s growth. The findings underscore the relevance of measuring SDR to guide policy recommendations aimed at reducing digital monopolization and promoting sustainable digital development.

Purpose – This paper aims to review the existing literature about consumers’ evaluation of products in virtual reality (VR), provide an accurate overview of this field, systematise the knowledge developed so far, identify the research gaps and propose a future research agenda. Design/methodology/approach – A systematic literature review was performed on Scopus and Web of Science, resulting in a final pool of 31 articles. Findings – Four main themes were identified, and a detailed research agenda is proposed based on the findings and following the theory, context, characteristics, methodology framework. Research limitations/implications – The provision of formal inclusion and exclusion criteria may have resulted in additional potentially relevant articles not indexed in the data set under consideration. Originality/value – The paper highlights how products are perceived in VR, the consumers’ responses, the peculiarities of VR compared to other conditions and VR as a product test environment. To the best of the authors’ knowledge, this paper seems to represent the first systemic review that focusses solely on how consumers assess products in VR. The results lead to a broad proposal of directions for future research that can expand knowledge on VR in marketing. Practical implications concern the use of VR to design product strategies and as a testing and prototyping environment. Objetivo – Este artículo revisa la literatura existente sobre la evaluación de los consumidores de productos en Realidad Virtual, proporciona una visión precisa de este campo, sistematiza el conocimiento desarrollado hasta el momento, identifica las lagunas en la investigación y propone una agenda de investigación futura. Metodología – Se realizó una revisión sistemática de la literatura en Scopus y Web of Science, que dio como resultado un conjunto final de 31 artículos. Resultados – Se identificaron cuatro temas principales y se propone una agenda de investigación detallada basada en las conclusiones y siguiendo el marco Teoría, Contexto, Características, Metodología. Limitaciones de la investigación – La provisión de criterios formales de inclusión y exclusión puede haber dado lugar a artículos adicionales potencialmente relevantes no indexados en el conjunto de datos considerado. Originalidad – El artículo destaca cómo se perciben los productos en la Realidad Virtual, las respuestas de los consumidores, las peculiaridades de la Realidad Virtual en comparación con otras condiciones y la Realidad Virtual como entorno de prueba de productos. Esta parece representar la primera revisión sistémica que se centra exclusivamente en cómo los consumidores evalúan los productos en la Realidad Virtual. Los resultados conducen a una amplia propuesta de direcciones para futuras investigaciones que puedan ampliar los conocimientos sobre la Realidad Virtual en el marketing. Las implicaciones prácticas se refieren al uso de la Realidad Virtual para diseñar estrategias de producto y como entorno de prueba y creación de prototipos. 目的 – 本文回顾了现有的关于消费者对虚拟现实产品评价的文献, 提供了这个领域的准确概述, 系统化了迄今为止的知识, 确定了研究差距, 并提出了一个未来的研究议程。 设计/方法/途径 – 在Scopus和Web of Science上进行了系统的文献综述, 最终形成了31篇文章的资料库。 研究结果 – 确定了四个主要的主题, 并根据研究结果, 按照理论、背景、特征、方法框架提出了详细的研究议程。 研究局限性 – 提供正式的纳入和排除标准可能会导致更多潜在的相关文章没有被收录到所考虑的数据集中。 原创性 – 文章强调了产品在虚拟现实中是如何被感知的, 消费者的反应, 与其他条件相比虚拟现实的特殊性, 以及虚拟现实作为产品测试环境。这似乎代表了第一个只关注消费者如何在虚拟现实中评估产品的系统性评论。研究结果为未来的研究方向提出了一个广泛的建议, 可以扩展营销中的虚拟现实知识。实际意义在于使用虚拟现实技术来设计产品策略, 并作为测试和原型设计环境。

Background: Bruises commonly occur in children and are often due to minor accidental injuries. However, they can also occur in bleeding disorders or inflicted injuries (physical abuse) and is often the most common visible manifestation of child physical abuse. Objective: This paper aims at highlighting the factors that should raise concern about non- accidental injury (physical abuse) in children presenting with bruising and the approach to their evaluation. Method: This paper is based ona manual literature search and review of relevant papers sourced from Pubmed using the search terms “bruising, non-accidental injuries in children, evaluation. Conclusion: It is instructive to carefully and thoroughly evaluate bruise in children utilizing peer review and the necessary ancillary tests. It is also advisable to always consider other possible causes of bruise and bruise-like lesions in forming opinion about suspected bruise as implications of false diagnosis are grave.

Crowdsourcing applications that integrate storytelling and geotagging capabilities offer new avenues for engaging the public in cultural heritage. However, standardised evaluation frameworks are lacking. This paper presents an applied evaluation methodology involving the analysis of relevant web-based tools. Towards this goal, this paper presents the development of crowdsourcing applications using, as a case study, the collection of myths and legends associated with the monumental heritage site of the Palace of the Grand Master of the Knights of Rhodes in Greece. Additionally, the paper presents an evaluation conducted through a criteria-based approach and user-based survey. The study reviews the concepts of crowdsourcing and digital storytelling within digital heritage, along with current concepts of living heritage and folklore, and examines relevant initiatives. The evaluation follows a four-stage methodology: (i) initial web-based tool selection based on the minimum requirements of web compatibility, crowdsourced data display, and map-based storytelling capability; (ii) application development using the selected web-based tools; (iii) a five-criteria assessment, based on the factors of open access, usability/tool support, participatory content/story creation, metrics provision and metadata model usage; and (iv) a crowd-based survey, indicating the most effective option. Findings from 100 respondents reveal limited exposure to participatory storytelling applications but interest in contributing content. Social media and influential figures serve as key channels for promoting crowdsourcing open calls. The results highlight gaps in understanding user expectations and perceptions, suggesting future research for gaining insights into engagement rates.

The concept of Christ’s Priesthood has been the subject of comprehensive research work in recent times. Despite several attempts to investigate the doctrine of Christ’s Priesthood waves a theological enigma. Thus, the study delved into the concept of Christology as denoted in the Scripture. This scholarly analysis focused on the qualities, which are linked to atonement and mediation. Christian theology has various viewpoints, but the primary model connects the sacrifice of Christ to His role as intercessor. The ontological model was a theological framework that stresses Christ’s priesthood on His personality and activities. The sacramental priesthood model denotes a correlation between Christ’s sacrificial act and His intercession in a heavenly realm, with distinct aspects following a predetermined sequence. The functional priesthood model is based on classical Protestant theology which portrays that the sacrifice of Christ and mediation are intrinsically interconnected with His being. The study employed a biblical-theological approach that sought to address the prevailing debates concerning Christ’s Priesthood. Thus, the study injects relevant sources to formulate and build the Christological concept. The research paper opined a holistic understanding of the nature of Christ and His transformative power to redeem humanity through the unction of the Holy Spirit. It is crystal clear that Christ sacrificed Himself to atone for all humankind’s sins. More importantly, the paper contributes to the current exploration of Priestly Christology, thus fostering a deeper understanding of Christ’s Priesthood and its significance to contemporary biblical theological discussion. It also helps to lay bare the salvific value of ecclesiology. This paper is recommended for biblical-theological historical students to advance and improve their studies.

Samah Tawil,1,2 Shafiul Haque,1,3,4 Pascale Salameh1,2,5 1Gilbert and Rose-Marie Chagoury School of Medicine, Lebanese American University, Beirut, Lebanon; 2Institut National de Santé Publique d’Épidémiologie Clinique et de Toxicologie-Liban (INSPECT-LB), Beirut, Lebanon; 3Research and Scientific Studies Unit, College of Nursing and Allied Health Sciences, Jazan University, Jazan, 45142, Saudi Arabia; 4Centre of Medical and Bio-Allied Health Sciences Research, Ajman University, Ajman, 13306, United Arab Emirates; 5Department of Primary Care and Population Health, University of Nicosia Medical School, Nicosia, 2417, CyprusCorrespondence: Samah Tawil, Lebanese American University, Beirut, Lebanon, Tel +9619547254, ext. 3466, Fax +9611867098, Email samah.tawil@lau.edu.lbAbstract: Increases in mental health problems have been observed during COVID-19 pandemic. Therefore, this topic of research became a priority, especially at the academic level. The objectives of this review were to summarize academic contribution to mental health research during the era of COVID-19. A scoping review of studies conducted at different academic institutions and examining alterations in mental health during the pandemic during the last three years was conducted. Fifty-five studies were included. These focused on different mental health changes that occurred in the era of COVID-19 such as changes in work habits or existing psychological conditions, COVID-19-related fear depression, anxiety and stress. Most of the included studies were observational (76.7%). The majority of the publications were published in Scimago Journal Rank Q1 journals (57.1%). The average number of citations per publication was 15.3 ± 30.13 [0– 125]. The number of citations was higher in papers with international collaboration of authors (p = 0.031). Publications with COVID-19 as a main objective were more cited than papers not directly related to this subject (25.9 ± 39.45 vs 4.14 ± 3.2; p = 0.044). Mental health problems are a common response to the COVID-19 pandemic. The contribution of academic universities to different mental health research studies that took place during COVID-19 outbreak have underlined this reality. However, more representative research from other institutional settings will be needed, particularly in vulnerable populations.Keywords: mental, health, research, contribution, evaluation, COVID-19

Museum educators increasingly face demands to evaluate the ‘value’ of non-formal learning (NF-L). This paper offers a unique international and multi-educator perspective on how informants from Portugal, Italy and the United Kingdom navigated these demands. Analysis of interview data highlighted that, although working in three different countries, most of these educators had experienced evaluation as accountability (and disciplinary) focused; employing methodologies inappropriate for evaluating NF-L and rarely team based. Drawing on a composite theoretical framework, these data led to co-constructing the Team Based Evaluation (TBE) model. Two case studies map how TBE was enacted and recommendations concerning organizational change are made. The paper concludes that, whilst set within the museum education space, TBE can be applied across evaluation contexts and micro and macro scales. Locally, TBE can mediate rich evidence and develop team working practices. Nationally and internationally, it can contribute to resetting evaluation from an accountability and disciplinary ‘machine’ to a dialogic, democratic and developmental activity.

This paper reviews how assessment has evolved in the past ten years at Carleton University in Ottawa, Ontario, Canada. From having 1.5 people devoted to assessment to one person in charge of assessment while also being Head of Collections, this paper will examine how it became imperative to determine how assessment could be integrated into the day to day workings of the library. By examining the library’s deployment of the Insync survey, reliance on external and internal university reports, assessment for strategic planning activities, and progression of how data is gathered over time, this paper will reveal how Carleton Library has reviewed the services, collection, and space in a time where assessment is important but the time to devote to it is lacking. Partnering with other departments on campus and within the library is key.

This paper presents results from a small-scale institutional ethnography study of reporting requirements and evaluation practices in two urban community health centers (CHCs), as a sample of community-based nonprofit organizations that focus on social justice and health equity. The study illuminated complex relationships among accountability, reporting and evaluation, and governance. Among the CHCs, reporting and evaluation practices consistent with goals of obtaining social justice and health equity were undermined by an imbalance toward funder-oriented functional accountability. Analysis of accountability and reporting practices as systemic factors shaping knowledge production, decision-making, and action revealed notable consequences for CHCs’ health promotion practice. This paper proposes a wholistic accountability model to encourage equitable power relations in evaluation, enable participatory methods, and better align CHC knowledge production and governance systems with their health promotion goals. This study further adds to the growing literature supporting critical attention to nonprofit accountability in the context of systems change work.

The aim of this paper is to investigate the role of deontic modality as a strategic means of mitigating evaluative meanings within and across texts. Evaluative meanings concern the function of language as used to express the speaker’s or writer’s subjective opinions, and such meanings have been extensively analysed through Appraisal framework. The framework has been used to account for evaluative/attitudinal meanings in texts, as well as dealing with the interaction of voices as one way through which speakers and writers can attribute evaluations to third parties in order to downplay or distance themselves from the evaluations that are expressed. Within the literature on Appraisal, however, the potential for deontic modality to mitigate subjective evaluation in texts has largely been overlooked and, thus, under-analysed. In this paper therefore, we develop a systems network for analysing the role of deontic modality and its interaction with other features as a tool for text analysis. We illustrate the distinctions in the network with examples of contrasting values from hard news stories that covered the 2011 public sector workers’ strike in Botswana and finish up with a short textual analysis to demonstrate how the consideration of deontic modality as a strategy of mitigation can not only enhance our understanding of how evaluative meanings are downplayed or overridden in texts, but also of how the distinctions between text types themselves can be blurred.

The goal of this paper is to determine the degree and direction of the correlation between the self-evaluation of the fifth-grade elementary school students and their success in the classroom. The paper presents the results of the research on students’ self-evaluation and their success in the classroom obtained on a sample of 208 fifth-grade elementary school students from the territory of the city of Banja Luka. An assessment scale was used for the purposes of the research and knowledge tests for the school subjects Serbian Language and Science and Social Studies were prepared. In line with the character of the research, the method of theoretical analysis and synthesis, the survey research method, and the analytical-descriptive method were applied. The results obtained using the Pearson Correlation Coefficient confirmed that students’ self-evaluation is significantly related to their general success at the end of the first term of the fifth grade of elementary school, their success at the end of the first semester in the subjects Serbian Language and Science and Social Studies, and achievements on the knowledge tests for the same school subjects. The obtained results indicate the importance of introducing student self-evaluation in the educational process in the field of achieving better results at school. However, the fact that self-evaluation is not sufficiently represented in our schools is a downside of this research, indicating a need for additional study and research of student self-evaluation, followed by its systematic introduction into educational practice.

The paper analyses the process of textbook selection and materials adaptation in the context of ESP (English for Specific Purposes) courses and in the context of English for Nursing course. Textbooks and materials are essential components of ESP course design, which frequently contribute to the success or failure of the course in completing the course objectives. However, the process of selecting a specific textbook for the course, which will address the course objectives, is not straightforward or simple. In the ESP literature, the views about the need of using textbooks in ESP courses are polarised. The study will examine the benefits of using textbooks in ESP context, which include available sources for the students and facilitate the class preparation for the teachers, as well as the drawbacks which include the impossibility of finding a textbook which corresponds entirely to learners’ needs. The research includes the evaluation of the selected textbook in the context of English for Nursing. The evaluation is based on the literature review on the topic of textbook evaluation in ESP and language teaching and learning context in general. The evaluation described is based on the set of criteria provided by Miekley (2005), which includes the following criteria: content, vocabulary and grammar, exercises and activities, attractiveness of the text, the quality of the teacher’s manual and the context. Two textbooks are examined and compared, in order to select the appropriate one. Apart from the textbook analysis, the paper aims to provide suggestions and recommendations regarding materials adaptation and development of materials which can be used in English for Nursing and Medical English Courses.

Abstract Digital technologies in, on, and around bicycles and cyclists are gaining ground. Collectively called Smart Cycling Technologies (SCTs), it is important to evaluate their impact on subjective cycling experiences. Future evaluations can inform the design of SCTs, which in turn can help to realize the abundant benefits of cycling. Wearable body sensors and advanced driver assistance systems are increasingly studied in other domains, however evaluation methods integrating such sensors and systems in the field of cycling research were under-reviewed and under-conceptualized. This paper therefore presents a systematic literature review and conceptual framework to support the use of body sensors in evaluations of the impact of SCTs on perceptions, emotions, feelings, affect, and more, during outdoor bicycle rides. The literature review (n = 40) showed that there is scarce research on this specific use of body sensors. Moreover, existing research designs are typically not tailored to determine impact of SCTs on cycling experience at large scale. Most studies had small sample sizes and explored limited sensors in chest belts and wristbands for evaluating stress response. The evaluation framework helps to overcome these limitations, by synthesizing crucial factors and methods for future evaluations in four categories: (1) experiences with SCTs, (2) experience measurements, (3) causal analysis, (4) confounding variables. The framework also identifies which types of sensors fit well to which types of experiences and SCTs. The seven directions for future research include, for example, experiences of psychological flow, sensors in e-textiles, and cycling with biofeedback. Future interactions between cyclists and SCTs will likely resemble a collaboration between humans and artificial intelligence. Altogether, this paper helps to understand if future support systems for cyclists truly make cycling safer and more attractive.

This paper approaches fictional telecinematic discourse from a perspective of inter-character negative evaluation in a specific context. The paper adapts a categorisation for interpersonal negative judgements and uses it to analyse instances of negative evaluation where participation in career practices are negotiated between the marital couple of Lucy and Ricky Ricardo in the classic sitcom I Love Lucy. A recurring theme in the show, Lucy’s desire to star in show business and Ricky’s attempts to thwart her ambitions have been discussed as both a representation of domestic containment typical to Post-War gender roles as well as an example of early feminist representation in televised sitcoms. The analysis reveals three intertwined facets of containment: Moral judgements condemning the oppressive behaviour of restricting Lucy’s access to career opportunities, moral judgements condemning the subversion of authority, and moral judgements of fairness condemning the distribution of social resources.

ZB MED – Information Center for Life Sciences is the largest library worldwide in the field of life sciences. From October 2016 to June 2018 Dietrich Nelle was its interim managing director. In this interview, he informs about ZB MED’s evaluation, realignment and cooperations. Furthermore, he speaks about open access, German Medical Science, research data management and research activities at ZB MED. Moreover, Nelle explains the position paper “Academic libraries 2025”, which he co-authored, and gives insight into his motivation to become provisional head of ZB MED. Finally, he drafts a possible way for the instituion’s resumption into the Leibniz Association in January 2022.

A recent report by the Royal Society examined the literature on the effectiveness of non-pharmaceutical interventions (NPIs) aimed at reducing the transmission of SARS-CoV-2 during the COVID-19 pandemic. It emphatically concluded that they were ‘effective’, which we argue may be misleading to policymakers. This paper performs a critical analysis of the Royal Society report and explains what a proper evaluation of NPIs would mean, arguing that theory-based evaluations are required to approach complex issues and nurture democratic debates on societal choices. We argue that, first, testing the relationship between NPI interventions and their effectiveness in reducing SARS-CoV-2 transmission over a bounded period (a single outcome) is irrelevant for policymaking, because several criteria must be balanced to evaluate any intervention, including efficiency, equity, acceptability, long-term impacts, and sustainability. Second, ignoring the disruptive and unintended effects of NPIs leads to wrong conclusions regarding their overall value. Third, we question the mere objective of NPIs, that is, reducing viral transmission. Finally, we question the methodology used by the Royal Society Report regarding the literature selection and quality. To conclude, we argue that the Royal Society report is not only irrelevant and weak from a methodological point of view but also dangerously misleading in terms of policymaking. Four years after the start of the COVID-19 pandemic, it is high time for researchers, journals, and policymakers to stop debating over NPIs’ mere effectiveness against a parameter that requires a PCR test to determine, but engage in public health-based evaluations that weigh all criteria of interest.

Abstract Background Maintaining quality of care and improving the quality of life (QOL) of patients and their families are important issues in palliative care. Therefore, there is a need to continuously evaluate the quality and outcomes of the care provided. In Japan, the Japan hospice and palliative evaluation (J-HOPE) study series has been conducted every three to four years since 2010, and we will conduct the fifth J-HOPE study (J-HOPE5). In the present paper, we describe the protocol of the J-HOPE5 study. The main objectives are: (1) to evaluate the processes, structures and outcomes of care at hospices or palliative care units; (2) to examine bereaved family members’ self-reported psychosocial condition, such as grief and depression as bereavement outcomes; (3) to provide data to ensure and improve the quality of care provided by participating institutions via feedback based on the results from each institution; and (4) to provide clinical and academic information regarding the implications of various issues in palliative care by conducting specific research. Methods We will conduct a cross-sectional, anonymous, self-reported questionnaire survey. In total, 153 institutions have agreed to participate in this study, meaning that approximately 12,240 bereaved family members (n = 80/institution) will be sent a questionnaire. Discussion This is one of the largest cross-sectional bereavement surveys to evaluate the quality of specialized palliative care for patients with cancer, both in Japan and worldwide. The large sample size of this study will enable wide analyses of specific targets and topics.

Abstract Background The primary aim of this scoping review was to synthesise key domains and sub-domains described in existing clinical decision support systems (CDSS) implementation frameworks into a novel taxonomy and demonstrate most-studied and least-studied areas. Secondary objectives were to evaluate the frequency and manner of use of each framework, and catalogue frameworks by implementation stage. Methods A scoping review of Pubmed, Scopus, Web of Science, PsychInfo and Embase was conducted on 12/01/2022, limited to English language, including 2000–2021. Each framework was categorised as addressing one or multiple stages of implementation: design and development, evaluation, acceptance and integration, and adoption and maintenance. Key parts of each framework were grouped into domains and sub-domains. Results Of 3550 titles identified, 58 papers were included. The most-studied implementation stage was acceptance and integration, while the least-studied was design and development. The three main framework uses were: for evaluating adoption, for understanding attitudes toward implementation, and for framework validation. The most frequently used framework was the Consolidated Framework for Implementation Research. Conclusions Many frameworks have been published to overcome barriers to CDSS implementation and offer guidance towards successful adoption. However, for co-developers, choosing relevant frameworks may be a challenge. A taxonomy of domains addressed by CDSS implementation frameworks is provided, as well as a description of their use, and a catalogue of frameworks listed by the implementation stages they address. Future work should ensure best practices for CDSS design are adequately described, and existing frameworks are well-validated. An emphasis on collaboration between clinician and non-clinician affected parties may help advance the field.

This paper presents an evaluation of a cybersecurity risk management and evaluation framework for Blockchain-based Identity Management Systems (BC-IdM) in the Health Internet of Things (HIoT). In this paper, thirteen experts were interviewed using a Delphi method to evaluate the framework, which includes the factors that are used to evaluate any HIoT BC-IdM system and the cybersecurity risk management processes and activities that should be applied. In addition, the Simple Multi-Attribute Rating Technique (SMART) was used in the interviews and questionnaires with the experts to assign weights to the twenty-six identified evaluation factors to rank them based on their importance. The identified factors are divided into four main categories: security and privacy, technical, HIoT-related considerations, and external aspects. This paper shows that the security and privacy factors are the most important among other factors. Using Delphi, an agreement on the details of the framework was sought, including the cybersecurity risk management processes and activities in the main phases: framing, assessment, responding, and mentoring of risks. This article identifies the main and subcategories of the evaluation factors and explains the framework content in detail. It presents recommendations and findings concerning Blockchain-based Identity Management Systems in Health Internet of Things. The framework plays a role in standardizing BC-IdM in HIoT and contributes to the applicability and reliability of such systems by considering security, privacy, technical, HIoT-related, and external considerations.

Purpose – This paper aims to review the challenges of performance appraisal in organizations and argue that these challenges can and must be overcome. Design/methodology/approach – The authors review research on performance appraisal in organizations and on claims that organizations are abandoning performance appraisal. Findings – Structured performance appraisals are still the norm in organizations around the world. There are clear and practical strategies for improving appraisals. These include improving feedback and removing unnecessary complexity, clarifying the goals of appraisal systems, focusing appraisal on behaviors and outcomes under the employee's control and increasing the fairness of appraisal systems. Research limitations/implications – Research is needed on the effects of changing the ways performance appraisals are conducted in organizations. Practical implications – Practical strategies for improving performance appraisal are outlined. Social implications – Better performance appraisals will benefit organizations and their members. Originality/value – This paper refutes the growing claim that organizations are abandoning performance appraisal and illustrates practical strategies for improving performance appraisal.

This paper explores the multifaceted nature of the educational process, focusing on the complex connections between educational actions, the dynamic nature of education, and subsequent evaluations. With a primary focus on primary school settings, the current paper tries to acknowledge the pivotal role of teachers' approaches to education in shaping students' learning experiences. It recognizes the significant influence of teacher discourse and the classroom dynamic on students' academic progress. Through this review of the existing literature, we examine ongoing discussions surrounding how pupils are influenced by various teaching interactions. We tried to highlight the importance of understanding the complexity of the educational journey, recognizing that in a rapidly evolving world, the student remains at the heart of education and evaluation approaches. The paper emphasizes the need to consider the holistic educational experience, encompassing teaching techniques, classroom dynamics, and students' receptiveness to instruction. The paper presents an understanding of how teachers' instructional approaches and classroom dynamics shape students' learning outcomes and performance assessments. By recognizing the central role of the pupil in the educational journey, this study contributes to the ongoing discourse on effective teaching practices and student-centered evaluation approaches.

Co-creation of public services implies new roles and responsibilities that, at least potentially, change the balance of control. In this way, it aligns closely with democratic renewal. This paper draws on the outcomes of a collaborative innovation project Co-creation of Service Innovation in Europe (CoSIE) funded under Horizon 2020. CoSIE built upon the idea that public sector innovations can be best achieved by creating collaborative partnerships between service providers (public sector agencies, third sector organisations, private companies) and citizens who receive services directly or indirectly. CoSIE was implemented through ten real-life innovation pilots in different public services across Europe. Results showed how co-creative methods could promote democratic dimensions, increasing the civic participation of marginalised and often voiceless population groups (residents of depleted urban neighbourhoods, disabled people in remote rural areas, citizens adrift from the world of work, and non-EU migrants). Some CoSIE pilots were more successful than others in extending impact beyond their immediate localities and service contexts. The paper highlights common factors that helped share learning and evolve project innovations into the ‘modus operandi’ of institutions and societies.

Scholarly research faces threats to its sustainability on multiple domains (access, incentives, reproducibility, inclusivity). We argue that “after-the-fact” research papers do not help and actually cause some of these threats because the chronology of the research cycle is lost in a research paper. We propose to give up the academic paper and propose a digitally native “as-you-go” alternative. In this design, modules of research outputs are communicated along the way and are directly linked to each other to form a network of outputs that can facilitate research evaluation. This embeds chronology in the design of scholarly communication and facilitates the recognition of more diverse outputs that go beyond the paper (e.g., code, materials). Moreover, using network analysis to investigate the relations between linked outputs could help align evaluation tools with evaluation questions. We illustrate how such a modular “as-you-go” design of scholarly communication could be structured and how network indicators could be computed to assist in the evaluation process, with specific use cases for funders, universities, and individual researchers.

Road traffic safety should be evaluated throughout the entire life-cycle of road design, operation, maintenance, and expansion construction. However, traditional methods for evaluating road traffic safety based on traffic accidents and conflict technology are limited by their inability to account for the complex environmental factors involved. To address this issue, a new road safety evaluation method has emerged that is based on driving behavior. Because drivers' behaviors may vary depending on the driving environment and their personal characteristics, evaluating road safety from the perspective of driver behavior has become a popular research topic. This paper analyzes current research trends and mainstream journals in the field of road safety evaluation of driving behavior. Additionally, it reviews the three most commonly used driving behavior data collection methods, and compares the advantages and disadvantages of each. The paper proposes the main application scenarios of road safety evaluation methods based on driving behavior, such as road design, evaluation of the effects of road appurtenances, and intelligent highways. Furthermore, the paper summarizes a driving behavior index system based on vehicle data, driver's physiological and psychological data, and driver's subjective questionnaire data. A comprehensive evaluation method based on the fusion of each index system is presented in detail. Finally, the paper points out current research problems and the future development direction of the road safety evaluation method based on driving behavior.

Energy, healthcare, electronics, transportation, ecology, and infrastructure are just a few of the areas that might greatly benefit from the use of new materials and systems. This study delves into the factors that should be taken into account when determining the possible effect of a certain substance or system. This analysis takes into account ecological, monetary, technical, health and safety, regulatory, commercial, and cooperative factors. Stakeholders may make more well-informed decisions and ensure that materials and systems are used to their maximum capacity if they take these into account. This paper used the concept of multi-criteria decision-making (MCDM) to deal with various criteria and factors. The VIKOR method is used as an MCDM method to rank the materials according to various criteria. The VIKOR method is integrated with the single-valued neutrosophic set to overcome uncertain information. This paper used eight criteria and ten materials to select the best one. The results show the cost criteria is the height weight in all criteria.

In a rapidly evolving digital landscape autonomous tools and robots are becoming commonplace. Recognizing the significance of this development, this paper explores the integration of Large Language Models (LLMs) like Generative pre-trained transformer (GPT) into human-robot teaming environments to facilitate variable autonomy through the means of verbal human-robot communication. In this paper, we introduce a novel simulation framework for such a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with simulated robot agents through natural language, each powered by individual GPT cores. By means of OpenAI’s function calling, we bridge the gap between unstructured natural language input and structured robot actions. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a simulated multi-robot environment. Our findings suggest that users may have preconceived expectations on how to converse with robots and seldom try to explore the actual language and cognitive capabilities of their simulated robot collaborators. Still, those users who did explore were able to benefit from a much more natural flow of communication and human-like back-and-forth. We provide a set of lessons learned for future research and technical implementations of similar systems.

Big data analytics has been widely adopted by large companies to achieve measurable benefits including increased profitability, customer demand forecasting, cheaper development of products, and improved stock control. Small and medium sized enterprises (SMEs) are the backbone of the global economy, comprising of 90 % of businesses worldwide. However, only 10 % SMEs have adopted big data analytics despite the competitive advantage they could achieve. Previous research has analysed the barriers to adoption and a strategic framework has been developed to help SMEs adopt big data analytics. The framework was converted into a scoring tool which has been applied to multiple case studies of SMEs in the UK. This paper documents the process of evaluating the framework based on the structured feedback from a focus group composed of experienced practitioners. The results of the evaluation are presented with a discussion on the results, and the paper concludes with recommendations to improve the scoring tool based on the proposed framework. The research demonstrates that this positioning tool is beneficial for SMEs to achieve competitive advantages by increasing the application of business intelligence and big data analytics.

Abstract Background The role of evaluation evidence in guiding health systems strengthening (HSS) investments at the global-level remains contested. A lack of rigorous impact evaluations is viewed by some as an obstacle to scaling resources. However, others suggest that power dynamics and knowledge hierarchies continue to shape perceptions of rigor and acceptability in HSS evaluations. This debate has had major implications on HSS resource allocation in global-level funding decisions. Yet, few studies have examined the relationship between HSS evaluation evidence and prioritization of HSS. In this paper, we explore the perspectives of key global health stakeholders, specifically around the nature of evidence sought regarding HSS and its potential impact on prioritization, the challenges in securing such evidence, and the drivers of intra- and inter-organizational divergences. We conducted a stakeholder analysis, drawing on 25 interviews with senior representatives of major global health organizations, and utilized inductive approaches to data analysis to develop themes. Results Our analysis suggests an intractable challenge at the heart of the relationship between HSS evaluations and prioritization. A lack of evidence was used as a reason for limited investments by some respondents, citing their belief that HSS was an unproven and potentially risky investment which is driven by the philosophy of HSS advocates rather than evidence. The same respondents also noted that the ‘holy grail’ of evaluation evidence that they sought would be rigorous studies that assess the impact of investments on health outcomes and financial accountability, and believed that methodological innovations to deliver this have not occurred. Conversely, others held HSS as a cross-cutting principle across global health investment decisions, and felt that the type of evidence sought by some funders is unachievable and not necessary – an ‘elusive quest’ – given methodological challenges in establishing causality and attribution. In their view, evidence would not change perspectives in favor of HSS investments, and evidence gaps were used as a ‘convenient excuse’. Respondents raised additional concerns regarding the design, dissemination and translation of HSS evaluation evidence. Conclusions Ongoing debates about the need for stronger evidence on HSS are often conducted at cross-purposes. Acknowledging and navigating these differing perspectives on HSS evaluation may help break the gridlock and find a more productive way forward.

Abstract Background On January 31st, 2023, the province of British Columbia (BC), Canada, was granted a federal exemption allowing adults (aged 18 +) to possess up to 2.5 g of select illicit drugs. The exemption will be in place for three years (2023–2026), marking the first formal decriminalization of illicit drug policy reform in Canada. BC’s decriminalization initiative is premised on several goals. This project seeks to evaluate each of these goals and their individual and combined contributions to determine the overall success of this policy. Methods The following protocol paper provides a detailed outline of a five-year (2022-2027) national evaluation of BC’s decriminalization initiative, as well as the specific objectives, methodologies, and planned analyses for eight interrelated sub-studies that comprise the evaluation. These sub-studies fall under the following five topical areas of research: 1) people who use drugs (PWUD), 2) the police and the criminal justice system, 3) the general public, 4) the health services system, and 5) an economic analysis. Additional research activities may also be explored. Results The overall evaluation and specific sub-study designs were informed by intensive stakeholder engagement. The evaluation was developed in collaboration with an international expert committee who came together to undertake a nominal group technique to decide on the final evaluation design and corresponding logic model. The evaluation will also employ an advisory board and individual sub-study working groups comprised of experts and PWUD who will oversee the development and implementation of the overall evaluation as well as each sub-study. Discussion This evaluation will draw on implementation science research practices to evaluate and understand the full impacts of this novel drug policy experiment. Results will be widely disseminated through manuscripts, reports, presentations, and infographics, which will be adapted and tailored for specific audiences. The protocol identifies several anticipated challenges and limitations. This evaluation’s evidence-based findings will be poised to offer pivotal insights that can shape and refine the discourse on drug policy and will serve as a critical resource for understanding the multifaceted impacts of decriminalization.

Abstract Background Addressing the upstream social determinants of health (e.g. built environment, education) can reduce the burden of non-communicable diseases. To do so effectively often requires system-wide collaboration. However, collaborating across multiple sectors, organizations and disciplines within a complex system can be challenging. ActEarly was a public health research consortium that aimed to improve child health by building an interdisciplinary, cross-city partnership to develop and/or evaluate upstream interventions, increase research capacity and improve collaboration between researchers, local authorities and communities. This paper explores ActEarly’s experiences of navigating complexity to identify mechanisms that supported its implementation and proposes recommendations for future intersectoral and interdisciplinary population health research collaborations. Methods We conducted a longitudinal qualitative study of ActEarly, integrating findings from inductive documentary analysis of internal documents (mainly meetings minutes and reports) (n = 114) and interviews (n = 70) with 45 consortium members at three different timepoints (2018, 2021, 2023). Participants worked across different organizations, cities, roles and levels of seniority in the consortium. Findings Clarity, Unity, Flexibility and Feasibility were seen as the key mechanisms required to support ActEarly’s implementation. Clear aims, governance structures and communication were necessary to manage the uncertainty of the complex system. A unified approach, characterized by strong relationships, having a shared vision and communal access to resources supported effective collaboration. Flexibility was required to adjust to different ways of working, respond to wider system events and manage the consortium. Establishing feasible aims that responded to the limitations of the system, the available resources and research infrastructure was required for teams to deliver the work. Conclusions Implementing multi-faceted programmes in a complex system can be challenging. We recommend that future whole-systems consortia seeking to improve population health build Clarity, Unity, Flexibility and Feasibility into their programmes, noting the complex interrelationships between these factors. Iterative reflections from all parties should support delivery amidst the uncertainty that comes with running a population health research collaboration, and strong leadership and governance should play a key role in ensuring that these are built into foundations the programme.