Search

Affect and cognitive load influence many user behaviors. In this paper, we propose Motion as Emotion, a novel method that utilizes fine differences in hand motion to recognise affect and cognitive load in virtual reality (VR). We conducted a study with 22 participants who used common free-hand gesture interactions to carry out tasks of varying difficulty in VR environments. We find that the affect and cognitive load induced by tasks are associated with significant differences in gesture features such as speed, distance and hand tension. Standard support vector classification (SVC) models could accurately predict two levels (low, high) of valence, arousal and cognitive load from these features. Our results demonstrate the potential of Motion as Emotion as an accurate and reliable method of inferring user affect and cognitive load from free-hand gestures, without needing any additional wearable sensors or modifications to a standard VR headset.

The neonatal period is the most vulnerable time for the development of seizures. Seizures in the immature brain lead to detrimental consequences, therefore require early diagnosis. The gold-standard for neonatal seizure detection currently relies on continuous video-EEG monitoring; which involves recording multi-channel electroencephalogram (EEG) alongside real-time video monitoring within a neonatal intensive care unit (NICU). However, video-EEG monitoring technology requires clinical expertise and is often limited to technologically advanced and resourceful settings. Cost-effective new techniques could help the medical fraternity make an accurate diagnosis and advocate treatment without delay. In this work, a novel explainable deep learning model to automate the neonatal seizure detection process with a reduced EEG montage is proposed, which employs convolutional nets, graph attention layers, and fully connected layers. Beyond its ability to detect seizures in real-time with a reduced montage, this model offers the unique advantage of real-time interpretability. By evaluating the performance on the Zenodo dataset with 10-fold cross-validation, the presented model achieves an absolute improvement of 8.31% and 42.86% in area under curve (AUC) and recall, respectively., Comment: Paper is accepted to IEEE International Conference on Systems, Man, and Cybernetics (SMC) 2024. Final Version

The extreme popularity over the years for railway transportation urges the necessity to maintain efficient railway management systems around the globe. Even though, at present, there exist a large collection of Computer Aided Designed Railway Technical Maps (RTMs) but available only in the portable document format (PDF). Using Deep Learning and Optical Character Recognition techniques, this research work proposes a generic system to digitize the relevant map component data from a given input image and create a formatted text file per image. Out of YOLOv3, SSD and Faster-RCNN object detection models used, Faster-RCNN yields the highest mean Average Precision (mAP) and the highest F1 score values 0.68 and 0.76 respectively. Further it is proven from the results obtained that, one can improve the results with OCR when the text containing image is being sent through a sophisticated pre-processing pipeline to remove distortions., Comment: 9 pages, 8 figures

Internet of Things (IoT) has seen a prolific rise in recent times and provides the ability to solve several key challenges faced by our societies and environment. Data produced by IoT provides a significant opportunity to infer context that is key for IoT applications to make decisions/actuations. Context Management Platform (CMP) is a middleware to facilitate the exchange and management of such context information among IoT applications. In this paper, we propose a novel approach to monitoring context freshness as a key metric, to improving the CMP's caching performance to support the real-time context needs of IoT applications. Our proposed hybrid algorithm uses Analytic Hierarchy Process (AHP) and Sliding Window technique to ensure the most relevant (as needed by the IoT applications) context information is cached. By continuously monitoring and prioritizing context attributes, the strategy adapts to IoT environment changes, keeping cached context fresh and reliable. Through experimental evaluation and using mock data obtained from a real-world mobile IoT scenario in section~\ref{use case}, we demonstrate that the proposed algorithm can substantially enhance context cache performance, by monitoring the context attributes in real time.

Retrieval-Augmented Generation (RAG) enhances Large Language Model (LLM) output by providing prior knowledge as context to input. This is beneficial for knowledge-intensive and expert reliant tasks, including legal question-answering, which require evidence to validate generated text outputs. We highlight that Case-Based Reasoning (CBR) presents key opportunities to structure retrieval as part of the RAG process in an LLM. We introduce CBR-RAG, where CBR cycle's initial retrieval stage, its indexing vocabulary, and similarity knowledge containers are used to enhance LLM queries with contextually relevant cases. This integration augments the original LLM query, providing a richer prompt. We present an evaluation of CBR-RAG, and examine different representations (i.e. general and domain-specific embeddings) and methods of comparison (i.e. inter, intra and hybrid similarity) on the task of legal question-answering. Our results indicate that the context provided by CBR's case reuse enforces similarity between relevant components of the questions and the evidence base leading to significant improvements in the quality of generated answers., Comment: Submitted to ICCBR'24

Emergency Medical Services (EMS) responders often operate under time-sensitive conditions, facing cognitive overload and inherent risks, requiring essential skills in critical thinking and rapid decision-making. This paper presents CognitiveEMS, an end-to-end wearable cognitive assistant system that can act as a collaborative virtual partner engaging in the real-time acquisition and analysis of multimodal data from an emergency scene and interacting with EMS responders through Augmented Reality (AR) smart glasses. CognitiveEMS processes the continuous streams of data in real-time and leverages edge computing to provide assistance in EMS protocol selection and intervention recognition. We address key technical challenges in real-time cognitive assistance by introducing three novel components: (i) a Speech Recognition model that is fine-tuned for real-world medical emergency conversations using simulated EMS audio recordings, augmented with synthetic data generated by large language models (LLMs); (ii) an EMS Protocol Prediction model that combines state-of-the-art (SOTA) tiny language models with EMS domain knowledge using graph-based attention mechanisms; (iii) an EMS Action Recognition module which leverages multimodal audio and video data and protocol predictions to infer the intervention/treatment actions taken by the responders at the incident scene. Our results show that for speech recognition we achieve superior performance compared to SOTA (WER of 0.290 vs. 0.618) on conversational data. Our protocol prediction component also significantly outperforms SOTA (top-3 accuracy of 0.800 vs. 0.200) and the action recognition achieves an accuracy of 0.727, while maintaining an end-to-end latency of 3.78s for protocol prediction on the edge and 0.31s on the server., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Real-time recognition and prediction of surgical activities are fundamental to advancing safety and autonomy in robot-assisted surgery. This paper presents a multimodal transformer architecture for real-time recognition and prediction of surgical gestures and trajectories based on short segments of kinematic and video data. We conduct an ablation study to evaluate the impact of fusing different input modalities and their representations on gesture recognition and prediction performance. We perform an end-to-end assessment of the proposed architecture using the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) dataset. Our model outperforms the state-of-the-art (SOTA) with 89.5\% accuracy for gesture prediction through effective fusion of kinematic features with spatial and contextual video features. It achieves the real-time performance of 1.1-1.3ms for processing a 1-second input window by relying on a computationally efficient model., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

While maps provide upfront content, this might not always be the most effective way for users to remember information. With the proliferation of interactive displays for tourists and visitors in public spaces, we can create a more playful user experience with maps than just exploring them. Adding interactions with the map could also help users retain more information as they use them. In this paper, we investigated whether completing a jigsaw puzzle of a map supports users in retaining more information about a specific map. The results of a between-subject study with a sample of n=28 indicate that additional interaction helped improve mean scores of textual and spatial recall but not visual recall. However, the results are not statistically significant, and the topic is subject to further investigation. Our findings contribute to discussions on using interactive touchscreen displays in similar learning scenarios involving memory retention., Comment: HCI SI 2023: Human-Computer Interaction Slovenia 2023, January 26, 2024, Maribor, Slovenia, 2 tables, 4 figures, 10 pages

The rapid digitization of real-world data offers an unprecedented opportunity for optimizing healthcare delivery and accelerating biomedical discovery. In practice, however, such data is most abundantly available in unstructured forms, such as clinical notes in electronic medical records (EMRs), and it is generally plagued by confounders. In this paper, we present TRIALSCOPE, a unifying framework for distilling real-world evidence from population-level observational data. TRIALSCOPE leverages biomedical language models to structure clinical text at scale, employs advanced probabilistic modeling for denoising and imputation, and incorporates state-of-the-art causal inference techniques to combat common confounders. Using clinical trial specification as generic representation, TRIALSCOPE provides a turn-key solution to generate and reason with clinical hypotheses using observational data. In extensive experiments and analyses on a large-scale real-world dataset with over one million cancer patients from a large US healthcare network, we show that TRIALSCOPE can produce high-quality structuring of real-world data and generates comparable results to marquee cancer trials. In addition to facilitating in-silicon clinical trial design and optimization, TRIALSCOPE may be used to empower synthetic controls, pragmatic trials, post-market surveillance, as well as support fine-grained patient-like-me reasoning in precision diagnosis and treatment., Comment: 6 Figures, 22 Pages, 3 Tables

Approximate Bayesian Computation (ABC) has gained popularity as a method for conducting inference and forecasting in complex models, most notably those which are intractable in some sense. In this paper we use ABC to produce probabilistic forecasts in state space models (SSMs). Whilst ABC-based forecasting in correctly-specified SSMs has been studied, the misspecified case has not been investigated, and it is that case which we emphasize. We invoke recent principles of 'focused' Bayesian prediction, whereby Bayesian updates are driven by a scoring rule that rewards predictive accuracy; the aim being to produce predictives that perform well in that rule, despite misspecification. Two methods are investigated for producing the focused predictions. In a simulation setting, 'coherent' predictions are in evidence for both methods: the predictive constructed via the use of a particular scoring rule predicts best according to that rule. Importantly, both focused methods typically produce more accurate forecasts than an exact, but misspecified, predictive. An empirical application to a truly intractable SSM completes the paper.

In this paper we introduce a new method based on a library of chemical reactions for constructing a system of ordinary differential equations from stochastic simulations arising from an agent-based model. The advantage of this approach is that this library respects any coupling between systems components, whereas the SINDy algorithm (introduced by Brunton, Proctor and Kutz) treats the individual components as decoupled from one another. Another advantage of our approach is that we can use a non-negative least squares algorithm to find the non-negative rate constants in a very robust, stable and simple manner. We illustrate our ideas on an agent-based model of tumour growth on a 2D lattice., Comment: 16 pages

Clinical trial matching is a key process in health delivery and discovery. In practice, it is plagued by overwhelming unstructured data and unscalable manual processing. In this paper, we conduct a systematic study on scaling clinical trial matching using large language models (LLMs), with oncology as the focus area. Our study is grounded in a clinical trial matching system currently in test deployment at a large U.S. health network. Initial findings are promising: out of box, cutting-edge LLMs, such as GPT-4, can already structure elaborate eligibility criteria of clinical trials and extract complex matching logic (e.g., nested AND/OR/NOT). While still far from perfect, LLMs substantially outperform prior strong baselines and may serve as a preliminary solution to help triage patient-trial candidates with humans in the loop. Our study also reveals a few significant growth areas for applying LLMs to end-to-end clinical trial matching, such as context limitation and accuracy, especially in structuring patient information from longitudinal medical records., Comment: 24 pages, 5 figures, accepted at Machine Learning for Healthcare (MLHC) 2023

This paper presents a novel approach to generate a use case diagram by analyzing the given user story using NLP and ML. Use case diagrams play a major role in the designing phase of the SDLC. This proves the fact that automating the use case diagram designing process would save a lot of time and effort. Numerous manual and semi-automated tools have been developed previously. This paper also discusses the need for use case diagrams and problems faced during designing that. This paper is an attempt to solve those issues by generating the use case diagram in a fully automatic manner., Comment: 5 pages, 2 figures, 4 tables

Technology integration in educational settings has led to the development of novel sensor-based tools that enable students to measure and interact with their environment. Although reports from using such tools can be positive, evaluations are often conducted under controlled conditions and short timeframes. There is a need for longitudinal data collected in realistic classroom settings. However, sustained and authentic classroom use requires technology platforms to be seen by teachers as both easy to use and of value. We describe our development of a sensor-based platform to support science teaching that followed a 14-month user-centered design process. We share insights from this design and development approach, and report findings from a 6-month large-scale evaluation involving 35 schools and 1245 students. We share lessons learnt, including that technology integration is not an educational goal per se and that technology should be a transparent tool to enable students to achieve their learning goals.