Search

Brain differences linked to autism spectrum disorder (ASD) can manifest before observable symptoms. Studying these early neural precursors in larger and more diverse cohorts is crucial for advancing our understanding of developmental pathways and potentially facilitating earlier identification. EEG is an ideal tool for investigating early neural differences in ASD, given its scalability and high tolerability in infant populations. In this context, we integrated EEG into an existing multi-site MRI study of infants with a higher familial likelihood of developing ASD. This paper describes the comprehensive protocol established to collect longitudinal, high-density EEG data from infants across five sites as part of the Infant Brain Imaging Study (IBIS) Network and reports interim feasibility and data quality results. We evaluated feasibility by measuring the percentage of infants from whom we successfully collected each EEG paradigm. The quality of task-free data was assessed based on the duration of EEG recordings remaining after artifact removal. Preliminary analyses revealed low data loss, with average in-session loss rates at 4.16 % and quality control loss rates at 11.66 %. Overall, the task-free data retention rate, accounting for both in-session issues and quality control, was 84.16 %, with high consistency across sites. The insights gained from this preliminary analysis highlight key sources of data attrition and provide practical considerations to guide similar research endeavors.

STUDY DESIGN: Development of a clinical practice guideline following the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) process. OBJECTIVE: The objectives of this study were to develop guidelines that outline the utility of intraoperative neuromonitoring (IONM) to detect intraoperative spinal cord injury (ISCI) among patients undergoing spine surgery, to define a subset of patients undergoing spine surgery at higher risk for ISCI and to develop protocols to prevent, diagnose, and manage ISCI. METHODS: All systematic reviews were performed according to PRISMA standards and registered on PROSPERO. A multidisciplinary, international Guidelines Development Group (GDG) reviewed and discussed the evidence using GRADE protocols. Consensus was defined by 80% agreement among GDG members. A systematic review and diagnostic test accuracy (DTA) meta-analysis was performed to synthesize pooled evidence on the diagnostic accuracy of IONM to detect ISCI among patients undergoing spinal surgery. The IONM modalities evaluated included somatosensory evoked potentials (SSEPs), motor evoked potentials (MEPs), electromyography (EMG), and multimodal neuromonitoring. Utilizing this knowledge and their clinical experience, the multidisciplinary GDG created recommendations for the use of IONM to identify ISCI in patients undergoing spine surgery. The evidence related to existing care pathways to manage ISCI was summarized and based on this a novel AO Spine-PRAXIS care pathway was created. RESULTS: Our recommendations are as follows: (1) We recommend that intraoperative neurophysiological monitoring be employed for high risk patients undergoing spine surgery, and (2) We suggest that patients at high risk for ISCI during spine surgery be proactively identified, that after identification of such patients, multi-disciplinary team discussions be undertaken to manage patients, and that an intraoperative protocol including the use of IONM be implemented. A care pathway for the prevention, diagnosis, and management of ISCI has been developed by the GDG. CONCLUSION: We anticipate that these guidelines will promote the use of IONM to detect and manage ISCI, and promote the use of preoperative and intraoperative checklists by surgeons and other team members for high risk patients undergoing spine surgery. We welcome teams to implement and evaluate the care pathway created by our GDG.

Machine learning (ML) applications in medical artificial intelligence (AI) systems have shifted from traditional and statistical methods to increasing application of deep learning models. This survey navigates the current landscape of multimodal ML, focusing on its profound impact on medical image analysis and clinical decision support systems. Emphasizing challenges and innovations in addressing multimodal representation, fusion, translation, alignment, and co-learning, the paper explores the transformative potential of multimodal models for clinical predictions. It also highlights the need for principled assessments and practical implementation of such models, bringing attention to the dynamics between decision support systems and healthcare providers and personnel. Despite advancements, challenges such as data biases and the scarcity of big data in many biomedical domains persist. We conclude with a discussion on principled innovation and collaborative efforts to further the mission of seamless integration of multimodal ML models into biomedical practice.

Proper neurological function relies on the cellular and molecular microenvironment of the brain, with perturbations of this environment leading to neurological disorders. However, studying the microenvironments of neurological tissue has proven difficult because of its inherent complexity. Both the cell type and metabolomic underpinnings of the cell have crucial functional roles, thus making multimodal characterization methods key to acquiring a holistic view of the brains microenvironment. This study investigates methods for combining matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) and immunofluorescence (IF) microscopy to enable the concurrent investigation of cell types and lipid profiles on the same sample. In brief, 1,5-diaminonaphthalene (DAN), α-cyano-4-hydroxy-cinnamic acid (CHCA), and 2,5-dihydroxybenzoic acid (DHB) were tested in addition to instrument-specific parameters for compatibility with IF. Alternatively, the effects of IF protocols on MALDI MSI were also tested, showing significant signal loss with all tested permutations. Ultimately, the use of CHCA for MALDI MSI resulted in the best IF images, while the use of DAN gave the lowest quality IF images. Overall, increasing the laser power and number of shots per laser burst resulted in the most tissue ablation. However, optimized parameter settings allowed for minimal tissue ablation while maintaining sufficient MALDI MSI signal.

Major Depressive Disorder (MDD) is an affective disorder that can lead to persistent sadness and a decline in the quality of life, increasing the risk of suicide. Utilizing multimodal data such as electroencephalograms and patient interview audios can facilitate the timely detection of MDD. However, existing depression detection methods either consider only a single modality or do not fully account for the differences and similarities between modalities in multimodal approaches, potentially overlooking the latent information inherent in various modal data. To address these challenges, we propose EMO-GCN, a multimodal depression detection method based on an adaptive multi-graph neural network. By employing graph-based methods to model data from various modalities and extracting features from them, the potential correlations between modalities are uncovered. The model's performance on the MODMA dataset is outstanding, achieving an accuracy (ACC) of 96.30%. Ablation studies further confirm the effectiveness of the model's individual components.The experimental results of EMO-GCN demonstrate the application prospects of graph-based multimodal analysis in the field of mental health, offering new perspectives for future research. [ABSTRACT FROM AUTHOR]

In this work, the development of a flexible Double‐Gate (DG) organic thin film transistor (DG‐OTFT), and its employment is reported for the realization of multimodal tactile sensors. Due to the self‐encapsulation of the stacked DG architecture, highly stable organic transistors are obtained that show almost negligible degradation after 6 months. Moreover, such configuration is also very useful for the development of sensing devices. In the case, one of the two gates is used to bias and set the working point of the devices, whereas the second one is connected to a polyvinylidene fluoride(PVDF)‐capacitor, a pyro/piezoelectric material. It is demonstrated that the charge displacement induced by the PVDF capacitor due to an applied external pressure or due to a temperature variation led to a reproducible variation of the device's output current. Using this approach high‐performing multimodal tactile sensors are obtained with sensitivity to up to 241 nA N−1 and 442 nA °C−1 respectively. [ABSTRACT FROM AUTHOR]

Cyber-physical systems (CPS), along with information technology devices such as portable devices and robots, materials including knowledge contents, and Internet resources, can be integrated into curricula to guide learners in observing and experiencing STEM (science, technology, engineering, and mathematics) knowledge and skills. This integration provides opportunities for learners to cultivate their abilities to design creative works. The objective of this study was to investigate the impact of a robot-assisted teaching and learning system in STEM education on learners' academic knowledge, skills, and attitudes. To achieve this objective, an experimental design was employed, with 30 college students assigned to learn and create a Drawdio electronic sound synthesizer using robot teachers and tablet computers. Pre- and post-tests were administered to measure students' academic achievement in STEM, and a survey was conducted to assess their attitudes toward STEM. The findings of the study indicate that the combination of robot teachers and tablet computers served as a supplementary learning resource within the learning environment. It accelerated knowledge acquisition, expanded hands-on experiences, and improved engagement and motivation among the learners. These results suggest that robot-assisted teaching and learning systems have the potential to enhance learning outcomes and attitudes among students. However, it is important to note that human teachers still outperformed robot teachers in certain aspects, particularly in higher levels of knowledge acquisition according to Bloom's taxonomy. Therefore, this study suggests the adoption of a collaborative model involving both robot and human teachers to effectively promote STEM education in the classroom. [ABSTRACT FROM AUTHOR]

This paper introduces a neural network‐based model designed for classifying emotional states by leveraging multimodal physiological signals. The model utilizes data from the AMIGOS and SEED‐V databases. The AMIGOS database integrates inputs from electroencephalogram (EEG), electrocardiogram (ECG), and galvanic skin response (GSR) to analyze emotional responses, while the SEED‐V database continuously updates EEG signals. We implemented a sequential neural network architecture featuring two hidden layers, which underwent substantial hyperparameter tuning to achieve optimal performance. Our model's effectiveness was tested through binary classification tasks focusing on arousal and valence, as well as a more complex four‐class classification that delineates emotional quadrants for the emotional tags: happy, sad, neutral, and disgust. In these varied scenarios, the model consistently demonstrated accuracy levels ranging from 79% to 86% in the AMIGOS database and up to 97% in SEED‐V. A notable aspect of our approach is the model's ability to accurately recognize emotions without the need for extensive signal preprocessing, a common challenge in multimodal emotion analysis. This feature enhances the practical applicability of our model in real‐world scenarios where rapid and efficient emotion recognition is essential. [ABSTRACT FROM AUTHOR]

Background: Correctly distinguishing between benign and malignant pulmonary nodules can avoid unnecessary invasive procedures. This study aimed to construct a deep learning radiomics clinical nomogram (DLRCN) for predicting malignancy of pulmonary nodules. Methods: One thousand and ninety‐eight patients with 6–30 mm pulmonary nodules who received histopathologic diagnosis at 3 centers were included and divided into a primary cohort (PC), an internal test cohort (I‐T), and two external test cohorts (E‐T1, E‐T2). The DLRCN was built by integrating adipose tissue radiomics features, intranodular and perinodular deep learning features, and clinical characteristics for diagnosing malignancy of pulmonary nodules. The least absolute shrinkage and selection operator (LASSO) was used for feature selection. The performance of DLRCN was assessed with respect to its calibration curve, area under the curve (AUC), and decision curve analysis (DCA). Furthermore, we compared it with three radiologists. The net reclassification improvement (NRI), integrated discrimination improvement (IDI), and subgroup analysis were also taken into account. Results: The incorporation of adipose tissue radiomics features led to significant NRI and IDI (NRI = 1.028, p < 0.05, IDI = 0.137, p < 0.05). In the I‐T, E‐T1, and E‐T2, the AUCs of DLRCN were 0.946 (95% CI: 0.936, 0.955), 0.948 (95% CI: 0.933, 0.963) and 0.962 (95% CI: 0.945, 0.979), The calibration curve revealed good predictive accuracy between the actual probability and predicted probability (p > 0.05). DCA showed that the DLRCN was clinically useful. Under equal specificity, the sensitivity of DLRCN increased by 8.6% compared to radiologist assessments. The subgroup analysis conducted on adipose tissue radiomics features further demonstrated their supplementary value in determining the malignancy of pulmonary nodules. Conclusion: The DLRCN demonstrated good performance in predicting the malignancy of pulmonary nodules, which was comparable to radiologist assessments. The adipose tissue radiomics features have notably enhanced the performance of DLRCN. [ABSTRACT FROM AUTHOR]

Recently, vision transformer (ViT) based multimodal learning methods have been proposed to improve the robustness of face anti-spoofing (FAS) systems. However, there are still no works to explore the fundamental natures (e.g., modality-aware inputs, suitable multimodal pre-training, and efficient finetuning) in vanilla ViT for multimodal FAS. In this paper, we investigate three key factors (i.e., inputs, pre-training, and finetuning) in ViT for multimodal FAS with RGB, Infrared (IR), and Depth. First, in terms of the ViT inputs, we find that leveraging local feature descriptors (such as histograms of oriented gradients) benefits the ViT on IR modality but not RGB or Depth modalities. Second, in consideration of the task (FAS vs. generic object classification) and modality (multimodal vs. unimodal) gaps, ImageNet pre-trained models might be sub-optimal for the multimodal FAS task. Finally, in observation of the inefficiency on direct finetuning the whole or partial ViT, we design an adaptive multimodal adapter (AMA), which can efficiently aggregate local multimodal features while freezing majority of ViT parameters. To bridge these gaps, we propose the modality-asymmetric masked autoencoder (M 2 A 2 E) for multimodal FAS self-supervised pre-training without costly annotated labels. Compared with the previous modality-symmetric autoencoder, the proposed M 2 A 2 E is able to learn more intrinsic task-aware representation and compatible with modality-agnostic (e.g., unimodal, bimodal, and trimodal) downstream settings. Extensive experiments with both unimodal (RGB, Depth, IR) and multimodal (RGB+Depth, RGB+IR, Depth+IR, RGB+Depth+IR) settings conducted on multimodal FAS benchmarks demonstrate the superior performance of the proposed methods. One highlight is that the proposed method is robust under various missing-modality cases where previous multimodal FAS models suffer serious performance drops. We hope these findings and solutions can facilitate the future research for ViT-based multimodal FAS. [ABSTRACT FROM AUTHOR]

Immersive virtual reality (VR) is anticipated to peak in development this decade bringing new opportunities for 3D multimodal designing across all levels of education. The need for students to gain capabilities with multimodal texts—texts that combine two or more modes, such as spoken, written, and visual—is emphasised at all levels of education from P-12 in the Australian Curriculum. Likewise, the use of technology-supported pedagogies is increasing worldwide, rendering multimodal texts ubiquitous across all knowledge domains. This original, qualitative classroom research investigated students' 3D designing of multimodal texts using an immersive VR head-mounted display. Upper primary students (ages 10–12 years, n = 48) transferred their knowledge of ancient Rome through 2D drawing, writing, speaking, and 3D multimodal designing with VR. The application of multimodal analysis to video data, screen recordings, and think-aloud protocols, and the thematic coding of student and teacher interviews yielded four key findings: (i) VR gaming supported 3D multimodal designing through haptic and embodied experience, (ii) VR improved performance through creative redesigning, (iii) VR-supported knowledge application, consolidation, and transfer, and (iv) pedagogical strengths of VR were situated and transformed practice. This research is timely and significant given the increasing accessibility and affordability of VR and the need to connect research and pedagogical practice to support students' advanced knowledge and capabilities with multimodal learning across the curriculum. [ABSTRACT FROM AUTHOR]

Multimodal named entity recognition (MNER) is an emerging foundational task in natural language processing. However, existing methods have two main limitations: 1) previous methods have focused on the visual representation of the entire image or target objects. However, they overlook the fine-grained semantic correspondence between entities and visual target objects, or ignore the visual cues of the overall scene and background details in the image. 2) Existing methods have not effectively overcome the semantic gap between different modalities due to the heterogeneity between text and images. To address these issues, we propose a novel multimodal heterogeneous graph entity-level fusion method for MNER (HGMVG) to achieve cross-modal feature interaction from coarse to fine between text and images under the guidance of visual information at different granularities, which can improve the accuracy of named entity recognition. Specifically, to resolve the first issue, we cascade cross-modal semantic interaction information between text and vision at different visual granularities to obtain a comprehensive and effective multimodal representation. For the second issue, we describe the precise semantic correspondences between entity-level words and visual target objects via multimodal heterogeneous graphs, and utilize heterogeneous graph attention networks to achieve cross-modal fine-grained semantic interactions. We conduct extensive experiments on two publicly available Twitter datasets, and the experimental results demonstrate that HGMVG outperforms the current state-of-the-art models in the MNER task. [ABSTRACT FROM AUTHOR]

Video Multimodal Entity Linking (VMEL) is a task to link entities mentioned in videos to entities in multimodal knowledge bases. However, current entity linking methods primarily focus on text and image modalities, neglecting the significance of video modality. To address this challenge, we propose a novel framework called the multi-perspective enhanced Subgraph Contrastive Network (SCMEL) and construct a VMEL dataset named SceneMEL, based on tourism domain. We first integrate textual, auditory and visual modal contexts of videos to generate a comprehensive high-recall candidate entity set. Furthermore, a semantic-enhanced video description subgraph generation module is utilized to convert videos into a multimodal feature graph structure and perform subgraph sampling on the domain-specific knowledge graph. Lastly, we conduct contrastive learning on local perspectives (text, audio, visual) within the video subgraphs and the knowledge graph subgraphs, as well as global perspectives, to capture fine-grained semantic information about videos and entities. A series of experimental results on SceneMel demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

As consumer electronics evolve towards greater intelligence, their automation and complexity also increase, making it difficult for users to diagnose faults when they occur. To address the problem where users, relying solely on their own knowledge, struggle to diagnose faults in consumer electronics promptly and accurately, we propose a multimodal knowledge graph-based text generation method. Our method begins by using deep learning models like the Residual Network (ResNet) and Bidirectional Encoder Representations from Transformers (BERT) to extract features from user-provided fault information, which can include images, text, audio, and even olfactory data. These multimodal features are then combined to form a comprehensive representation. The fused features are fed into a graph convolutional network (GCN) for fault inference, identifying potential fault nodes in the electronics. These fault nodes are subsequently fed into a pre-constructed knowledge graph to determine the final diagnosis. Finally, this information is processed through the Bias-term Fine-tuning (BitFit) enhanced Chinese Pre-trained Transformer (CPT) model, which generates the final fault diagnosis text for the user. The experimental results show that our proposed method achieves a 4.4% improvement over baseline methods, reaching a fault diagnosis accuracy of 98.4%. Our approach effectively leverages multimodal fault information, addressing the challenges users face in diagnosing faults through the integration of graph convolutional network and knowledge graph technologies. [ABSTRACT FROM AUTHOR]

Duchenne muscular dystrophy (DMD) is a progressive X‐linked neuromuscular disorder caused by the absence of functional dystrophin protein. In addition to muscle, dystrophin is expressed in the brain in both neurons and glial cells. Previous studies have shown altered white matter microstructure in patients with DMD using diffusion tensor imaging (DTI). However, DTI measures the diffusion properties of water, a ubiquitous molecule, making it difficult to unravel the underlying pathology. Diffusion‐weighted spectroscopy (DWS) is a complementary technique which measures diffusion properties of cell‐specific intracellular metabolites. Here we performed both DWS and DTI measurements to disentangle intra‐ and extracellular contributions to white matter changes in patients with DMD. Scans were conducted in patients with DMD (15.5 ± 4.6 y/o) and age‐ and sex‐matched healthy controls (16.3 ± 3.3 y/o). DWS measurements were obtained in a volume of interest (VOI) positioned in the left parietal white matter. Apparent diffusion coefficients (ADCs) were calculated for total N‐acetylaspartate (tNAA), choline compounds (tCho), and total creatine (tCr). The tNAA/tCr and tCho/tCr ratios were calculated from the non‐diffusion‐weighted spectrum. Mean diffusivity (MD), radial diffusivity (RD), axial diffusivity (AD), and fractional anisotropy of water within the VOI were extracted from DTI measurements. DWS and DTI data from patients with DMD (respectively n = 20 and n = 18) and n = 10 healthy controls were included. No differences in metabolite ADC or in concentration ratios were found between patients with DMD and controls. In contrast, water diffusion (MD, t = −2.727, p = 0.011; RD, t = −2.720, p = 0.011; AD, t = −2.715, p = 0.012) within the VOI was significantly higher in patients compared with healthy controls. Taken together, our study illustrates the potential of combining DTI and DWS to gain a better understanding of microstructural changes and their association with disease mechanisms in a clinical setting. [ABSTRACT FROM AUTHOR]

One‐for‐all phototheranostics that allows the simultaneous implementations of multiple optical imaging and therapeutic modalities by utilizing a single component, is growing into a sparkling frontier in cancer treatment. Of particular interest is phototheranostic agent with emission in the second near‐infrared (NIR‐II) window. Nevertheless, the practical uses of those conventional NIR‐II agents are severely impeded by their unsatisfactory features including insufficient stability, low synthetic yield, to be extended absorption/ emission wavelengths, and inefficient phototheranostic outcomes. Developing exceptional phototheranostic agents is thus highly desirable yet remains formidably challenging. Herein, we synthesized two novel N‐heteroacenes‐based NIR‐II luminogens, namely 2TT‐PPT and 4TT‐PBPT, by respectively employing pyrene‐fused phenaziothiadiazoles and pyrene‐fused bisphenaziothiadiazoles as acceptor skeletons. There is strength in numbers by increasing the fusing rings in N‐heteroacenes moieties and numbers of appended donors. Compared to less ring‐fused 2TT‐PPT, the giant molecule 4TT‐PBPT shows improved photophysical characteristics, such as enhanced light absorbance, red‐shifted wavelengths, higher brightness, favorable reactive oxygen species (ROS) generation, and elevated photothermal conversion efficiency, which render 4TT‐PBPT nanoparticles excellent fluorescence‐photoacoustic‐photothermal trimodal imaging guided photodynamic‐photothermal synergistic therapy for orthotopic bladder cancer. [ABSTRACT FROM AUTHOR]

The floor constitutes one of the largest areas within a building with which users interact most frequently in daily activities. Employing floor sensors is vital for smart‐building digital twins, wherein triboelectric nanogenerators demonstrate wide application potential due to their good performance and self‐powering characteristics. However, their sensing stability, reliability, and multimodality require further enhancement to meet the rapidly evolving demands. Thus, this work introduces a multimodal intelligent flooring system, implementing a 4 × 4 floor array for multimodal information detection (including position, pressure, material, user identity, and activity) and human–machine interactions. The floor unit incorporates a hybrid structure of triboelectric pressure sensors and a top‐surface material sensor, facilitating linear and enhanced sensitivity across a wide pressure range (0–800 N), alongside the material recognition capability. The floor array is implemented by an advanced output‐ratio method with minimalist output channels, which is insensitive to environmental factors such as humidity and temperature. In addition to multimodal sensing, energy harvesting is co‐designed with the pressure sensors for scavenging waste energy to power smart‐building sensor nodes. This developed flooring system enables multimodal sensing, energy harvesting, and smart‐sport interactions in smart buildings, greatly expanding the floor sensing scenarios and applications. [ABSTRACT FROM AUTHOR]

Gait Recognition, as a long-distance biometric technique for identity recognition, has attracted widespread attention in recent years. Previous academia typically employs minor convolutional networks to extract single-modal features from the silhouette or the joint skeleton. Nevertheless, silhouette-based methods are susceptible to clothing variations, while skeleton-based methods encounter the issue of missing physique information. Therefore, we propose a novel multimodal triple-branch network dubbed GaitRA to comprehensively acquire gait features from two aspects of silhouette and skeleton simultaneously. GaitRA consists of three branches: a 3D-CNN branch to extract the primary features of silhouette sequences, a 2D-CNN branch to obtain the secondary features of silhouette sequences, and a branch based on Spatio-Temporal Graph Convolution (ST-GCN) to gain joint skeleton features. More importantly, we innovatively introduce the RepLK-ACTION Module, which combines the RepLK Block based on the Swin Transformer and the ACTION Module based on a mixed attention mechanism from the Action Recognition. RepLK-ACTION Module establishes larger Effective Receptive Fields (ERFs) through the large-kernel re-parameterized convolution to attain more discriminative multimodal gait information, thereby enhancing the model performance under complex walking conditions. Experiments have demonstrated that GaitRA significantly improves performance on both CASIA-B and Gait3D datasets. Especially, the proposed method achieves excellent results of 73.1% (Rank-1), 86.4% (Rank-5), 64.3% (mAP), and 36.2% (mINP) on the Gait3D dataset. [ABSTRACT FROM AUTHOR]

Currently, knowledge representation learning has been effectively utilized in knowledge graph completion tasks to predict missing entities or relationships within knowledge graphs, achieving considerable success. However, the structural features of triples after data construction are the only considerations for most traditional completion methods based on representation learning, the results in the rich textual, image, and other multimodal information in the original data not being effectively utilized. To address this problem, it is proposed that the energy function of triples should be defined as the sum of sub-energy functions, which are characterized by the textual features of relationships and the multimodal features of entities, to fully exploit the semantic information in the triple. In addition, a multimodal negative sampling strategy has been devised for diverse energy functions, aiming to align the disparate features of entities, enhance the quality of knowledge representation, and boost the performance of knowledge graph completion. Link prediction experiments were conducted on publicly available datasets DB15K and FB15K-237. The results show that the proposed model exhibits remarkable capabilities in knowledge graph completion. [ABSTRACT FROM AUTHOR]

The rupture of vulnerable plaque (VP) are significant pathogenic factors leading to cardiovascular and cerebrovascular diseases. This study aims to construct a vulnerable plaque prediction model (VPPM) by combining multimodal vascular ultrasound parameters and clinical risk factors, and to validate it. A total of 196 atherosclerotic patients who underwent carotid endarterectomy (CEA) from January 2017 to December 2023 were collected and divided into a modeling group (n = 137) and a validation group (n = 59). Clinical information including: hypertension, diabetes, smoking history, and body mass index (BMI) was included in the analysis. All patients underwent carotid ultrasound and contrast-enhanced ultrasound (CEUS) examination after admission, with main ultrasound parameters including thickness, echogenicity types, stenosis degree, and CEUS neovascularization grading of plaques. Independent risk factors for VP in CEA patients were screened through binary Logistic regression analysis, and a prediction model was established along with a nomogram. The calibration curve, receiver-operating characteristic curve (ROC), and decision curve analysis (DCA) were employed to assess the calibration, diagnostic efficacy, and clinical utility of the VPPM model. There were no significant statistical differences in multimodal vascular ultrasound parameters and clinical risk factors between the modeling and validation groups (P > 0.05). Binary Logistic regression analysis identified plaque thickness, echo type, CEUS neovascularization grading, BMI, and smoking history as 5 variables entering the prediction model. The VPPM model showed good diagnostic efficacy, with an area under the ROC curve of 0.959 (95% CI 0.915–0.999). Using the nomogram with a VPPM risk assessment score of 135.42 as the diagnostic cutoff value in the modeling group, the sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and Youden index were 88.1%, 94.1%, 14.98, 0.126, and 82.2%, respectively. In the DCA curve, the VPPM model curve was significantly better than two extreme lines, indicating good clinical utility. The VPPM model constructed by integrating multimodal ultrasound parameters and clinical key risk factors has high diagnostic efficacy and is expected to be an auxiliary tool for clinical diagnosis of vulnerable plaques. [ABSTRACT FROM AUTHOR]

Multimodal technology is poised to revolutionize clinical practice by integrating artificial intelligence with traditional diagnostic modalities. This evolution traces its roots from Hippocrates' humoral theory to the use of sophisticated AI-driven platforms that synthesize data across multiple sensory channels. The interplay between historical medical practices and modern technology challenges conventional patient–clinician interactions and redefines diagnostic accuracy. Highlighting applications from neurology to radiology, the potential of multimodal technology emerges, suggesting a future where AI not only supports but enhances human sensory inputs in medical diagnostics. This shift invites the medical community to navigate the ethical, practical, and technological changes reshaping the landscape of clinical medicine. [ABSTRACT FROM AUTHOR]

Users often select different modalities in multimodal systems based on context and function. This study aimed to investigate the impact of context on users' modality selection in a multimodal system. A modality selection task was presented through the developed multimodal system, where the task's difficulty was controlled based on the number of syllables and touches. The experiments were examined in four usage contexts (baseline, watching, reading, and driving), and the selected modality, physical and mental effort, and satisfaction were measured. A regression model for predicting modality selection was established using binomial logistic regression. The results showed that voice usage increased in the order of baseline, watching, reading, and driving contexts. It was revealed that these changes were attributable to differences in physical and mental interaction efforts of modalities according to contexts. Our finding provides valuable insights into users' modality selection in different contexts, which could inform the design of more efficient multimodal systems. [ABSTRACT FROM AUTHOR]

In public spaces, threats to societal security are a major concern, and emerging technologies offer potential countermeasures. The proposed intelligent person identification system monitors and identifies individuals in public spaces using gait, face, and iris recognition. The system employs a multimodal approach for secure identification and utilises deep convolutional neural networks (DCNNs) that have been pretrained to predict individuals. For increased accuracy, the proposed system is implemented on a cloud server and integrated with citizen identification systems such as Aadhar/SSN. The performance of the system is determined by the rate of accuracy achieved when identifying individuals in a public space. The proposed multimodal secure identification system achieves a 94% accuracy rate, which is higher than that of existing public space person identification systems. Integration with citizen identification systems improves precision and provides immediate life-saving assistance to those in need. Utilising secure deep learning techniques for precise person identification, the proposed system offers a promising solution to security threats in public spaces. This research is necessary to investigate the efficacy and potential applications of the proposed system, including accident identification, theft identification, and intruder identification in public spaces. [ABSTRACT FROM AUTHOR]

Latin Cuengh is a low-resource dialect that is prevalent in select ethnic minority regions in China. This language presents unique challenges for intelligent research and preservation efforts, primarily due to its oral tradition and the limited availability of textual resources. Prior research has sought to bolster intelligent processing capabilities with regard to Latin Cuengh through data augmentation techniques leveraging scarce textual data, with modest success. In this study, we introduce an innovative multimodal seed data augmentation model designed to significantly enhance the intelligent recognition and comprehension of this dialect. After supplementing the pre-trained model with extensive speech data, we fine-tune its performance with a modest corpus of multilingual textual seed data, employing both Latin Cuengh and Chinese texts as bilingual seed data to enrich its multilingual properties. We then refine its parameters through a variety of downstream tasks. The proposed model achieves a commendable performance across both multi-classification and binary classification tasks, with its average accuracy and F1 measure increasing by more than 3%. Moreover, the model's training efficiency is substantially ameliorated through strategic seed data augmentation. Our research provides insights into the informatization of low-resource languages and contributes to their dissemination and preservation. [ABSTRACT FROM AUTHOR]

In recent years, due to the wide application of multi-sensor vision systems, multimodal image acquisition technology has continued to develop. Monomodal images cannot meet the needs of image analysis, which requires image fusion and stitching to process images for better image analysis. Image registration is an important prerequisite of image fusion and stitching. Most of the existing multimodal image registration methods are only suitable for two modalities, and cannot uniformly register multimodal image data. Therefore, this paper proposes a multimodal remote sensing image registration method based on adaptive multi-scale PIIFD (AM-PIIFD). This method extracts KAZE features in the scale space constructed by nonlinear diffusion filtering. It can effectively preserve the edge feature information while filtering out the noise. Then, the proposed AM-PIIFD feature descriptor is used to describe the multi-scale features. Finally, according to the consistency of the feature main orientation, most of the mismatches are removed, and the image alignment transformation is realized. The qualitative and quantitative comparisons with the other three advanced methods show that our method can achieve good performance in multimodal remote sensing image registration. [ABSTRACT FROM AUTHOR]

This manuscript examines multimodal storytelling as community inquiry for an urban high school class of 30 first‐ and second‐generation bi/multilingual immigrant students, most of whom maintained transnational connections. We share how these students, in an A.P. Research class, engaged in community‐based inquiry and utilized various multimodal artifacts from a university cultural heritage collection, including a field trip to the cultural heritage archives, to analyze signs and symbols. We asked: How did students engage with semiotic resources and approaches to support inquiry about their school and neighborhood communities? Findings showed that students drew upon their linguistic and cultural experiences to analyze signs and symbols, yet in their digital storytelling presentations some student groups chose signs and symbols that reified stereotypical views of their neighborhood despite their insider status in the local community. We offer recommendations for using cultural heritage artifacts to advance multimodal meaning‐making in teaching and research. [ABSTRACT FROM AUTHOR]

Depression is a prevalent mental condition, and it is essential to diagnose and treat patients as soon as possible to maximize their chances of rehabilitation and recovery. An intelligent detection model based on multimodal fusion technology is proposed based on the findings of this study to address the difficulties associated with depression detection. Text data and electroencephalogram (EEG) data are used in the model as representatives of subjective and objective nature, respectively. These data are processed by the BERT–TextCNN model and the CNN–LSTM model, which are responsible for processing them. While the CNN–LSTM model is able to handle time-series data in an effective manner, the BERT–TextCNN model is able to adequately capture the semantic information that is included in text data. This enables the model to consider the various features that are associated with the various types of data. In this research, a weighted fusion technique is utilized to combine the information contained within the two modal datasets. This strategy involves assigning a weight to the outcomes of each modal data processing in accordance with the degree of contribution that each modal data will make to produce the ultimate depression detection results. In regard to the task of depression identification, the suggested model demonstrates great validity and robustness, as demonstrated by the results of the experimental validation that we carried out on a dataset that we manufactured ourselves. A viable and intelligent solution for the early identification of depression is provided by the proposed model. This solution will likely be widely utilized in clinical practice and will provide new ideas and approaches for the growth of the field of precision medicine. [ABSTRACT FROM AUTHOR]