Search

Inspired by the success of deep learning in predicting static protein structures, researchers are now actively exploring other deep learning algorithms aimed at predicting the conformational changes of proteins. Currently, a major challenge in the development of such models lies in the limited training data characterizing different conformational transitions. To address this issue, molecular dynamics simulations is combined with enhanced sampling methods to create a large-scale database. To this end, the study simulates the conformational changes of 2635 proteins featuring two known stable states, and collects the structural information along each transition pathway. Utilizing this database, a general deep learning model capable of predicting the transition pathway for a given protein is developed. The model exhibits general robustness across proteins with varying sequence lengths (ranging from 44 to 704 amino acids) and accommodates different types of conformational changes. Great agreement is shown between predictions and experimental data in several systems and successfully apply this model to identify a novel allosteric regulation in an important biological system, the human β-cardiac myosin. These results demonstrate the effectiveness of the model in revealing the nature of protein conformational changes., (© 2024 The Author(s). Advanced Science published by Wiley‐VCH GmbH.)

Background: The volume measurement of intracerebral hemorrhage (ICH) and intraventricular hemorrhage (IVH) provides critical information for precise treatment of patients with spontaneous ICH but remains a big challenge, especially for IVH segmentation. However, the previously proposed ICH and IVH segmentation tools lack external validation and segmentation quality assessment., Purpose: This study aimed to develop a robust deep learning model for the segmentation of ICH and IVH with external validation, and to provide quality assessment for IVH segmentation., Methods: In this study, a Residual Encoding Unet (REUnet) for the segmentation of ICH and IVH was developed using a dataset composed of 977 CT images (all contained ICH, and 338 contained IVH; a five-fold cross-validation procedure was adopted for training and internal validation), and externally tested using an independent dataset consisting of 375 CT images (all contained ICH, and 105 contained IVH). The performance of REUnet was compared with six other advanced deep learning models. Subsequently, three approaches, including Prototype Segmentation (ProtoSeg), Test Time Dropout (TTD), and Test Time Augmentation (TTA), were employed to derive segmentation quality scores in the absence of ground truth to provide a way to assess the segmentation quality in real practice., Results: For ICH segmentation, the median (lower-quantile-upper quantile) of Dice scores obtained from REUnet were 0.932 (0.898-0.953) for internal validation and 0.888 (0.859-0.916) for external test, both of which were better than those of other models while comparable to that of nnUnet3D in external test. For IVH segmentation, the Dice scores obtained from REUnet were 0.826 (0.757-0.868) for internal validation and 0.777 (0.693-0.827) for external tests, which were better than those of all other models. The concordance correlation coefficients between the volumes estimated from the REUnet-generated segmentations and those from the manual segmentations for both ICH and IVH ranged from 0.944 to 0.987. For IVH segmentation quality assessment, the segmentation quality score derived from ProtoSeg was correlated with the Dice Score (Spearman r = 0.752 for the external test) and performed better than those from TTD (Spearman r = 0.718) and TTA (Spearman r = 0.260) in the external test. By setting a threshold to the segmentation quality score, we were able to identify low-quality IVH segmentation results by ProtoSeg., Conclusions: The proposed REUnet offers a promising tool for accurate and automated segmentation of ICH and IVH, and for effective IVH segmentation quality assessment, and thus exhibits the potential to facilitate therapeutic decision-making for patients with spontaneous ICH in clinical practice., (© 2024 American Association of Physicists in Medicine.)

Objectives: We aimed to investigate the value of deep learning (DL) models based on multimodal ultrasonographic (US) images to quantify RA activity., Methods: Static greyscale (SGS), dynamic greyscale (DGS), static power Doppler (SPD) and dynamic power Doppler (DPD) US images were collected and evaluated by two expert radiologists according to the EULAR-OMERACT Synovitis Scoring system. Four DL models were developed based on the ResNet-type structure, evaluated on two separate test cohorts, and finally compared with the performance of 12 radiologists with different levels of experience., Results: In total, 1244 images were used for the model training, and 152 and 354 for testing (cohort 1 and 2, respectively). The best-performing models for the scores of 0/1/2/3 were the DPD, SGS, DGS and SPD models, respectively (Area Under the receiver operating characteristic Curve [AUC] = 0.87/0.95/0.74/0.95; no significant differences). All the DL models provided results comparable to the experienced radiologists on a per-image basis (intraclass correlation coefficient: 0.239-0.756, P < 0.05). The SPD model performed better than the SGS one on test cohort 1 (score of 0/2/3: AUC = 0.82/0.67/0.95 vs 0.66/0.66/0.75, respectively) and test cohort 2 (score of 0: AUC = 0.89 vs 0.81). The dynamic DL models performed better than the static ones in most of the scoring processes and were more accurate than the most of senior radiologists, especially the DPD model., Conclusion: DL models based on multimodal US images allow a quantitative and objective assessment of RA activity. Dynamic DL models in particular have potential value in assisting radiologists to improve the accuracy of RA US-based grading., (© The Author(s) 2023. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

Transport proteins (TPs) are vital to the growth and life of all living things, especially in fields of microbial pathogenesis and drug resistance of tumor cells. Accurately identifying potential TPs remains an important challenge for the advancement of functional genomics. This study aimed to develop a tool for predicting TPs using the deep learning approach. Here, we proposed DeepTP, a convolutional neural network model that uses parallel subnetworks to extract features from protein sequences and uses fully connected layers for TP classification. To train and evaluate the performance of the developed model, datasets were collected from the UniProtKB/Swiss-Prot database. The test results revealed that the proposed model could successfully identify TPs with the AUCROC, accuracy, F-value, and Matthews correlation coefficient of 0.9719, 0.9513, 0.8982, and 0.8679, respectively. By further comparison, DeepTP achieved better performance than other commonly used methods. Analysis of the gradients of prediction score concerning input suggested that DeepTP makes predictions by recognizing the functional domains of TPs. We anticipate that DeepTP will serve as a useful tool for predicting TPs in large-scale genome projects, which will facilitate the discovery of novel TPs., Competing Interests: Declaration of competing interest None Declared., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Objective: To investigate the utility of an automatic deep learning (DL) method for segmentation of T2 maps in patients with idiopathic inflammatory myopathy (IIM) against healthy controls, and also the association of quantitative T2 values in patients with laboratory and pulmonary findings., Methods: Structural MRI and T2 mapping of bilateral thigh muscles from patients with IIM and healthy volunteers were segmented using dedicated software based on a pre-trained convolutional neural network. Incremental and federated learning were implemented for continuous adaptation and improvement. Muscle T2 values derived from DL segmentation were compared between patients and healthy controls, and T2 values of patients were further analyzed with serum muscle enzymes, and interstitial lung disease (ILD) which was diagnosed and graded based on chest HRCT., Results: Overall, 64 patients (27 patients with dermatomyositis, 29 with polymyositis, and 8 with antisynthetase syndrome (ASS)) and 10 healthy controls were included. By using DL-based muscle segmentation, T2 values generated from T2 maps accurately differentiated patients from those of controls (p < 0.001) with a cutoff value of 36.4 ms (sensitivity 96.9%, and specificity 100%). In patients with IIM, muscle T2 values positively correlated with all the serum muscle enzymes (all p < 0.05). ILD score of patients with ASS was markedly higher than that of those without ASS (p = 0.011), while dissociation between the severity of muscular involvement and ILD was observed (p = 0.080)., Conclusion: Automatic DL could be used to segment thigh muscles and help quantitatively assess muscular inflammation of IIM through T2 mapping., Key Points: • Muscle T2 mapping automatically segmented by deep learning can differentiate IIM from healthy controls. • T2 value, an indicator of active muscle inflammation, positively correlates with serum muscle enzymes. • T2 mapping can detect muscle disease in patients with normal muscle enzyme levels., (© 2022. The Author(s).)

The widespread dissemination of facial forgery technology has brought many ethical issues and aroused widespread concern in society. Most research today treats deepfake detection as a fine grained classification task, which however makes it difficult to enable the feature extractor to express the features related to the real and fake attributes. This paper proposes a depth map guided triplet network, which mainly consists of a depth prediction network and a triplet feature extraction network. The depth map predicted by the depth prediction network can effectively reflect the differences between real and fake faces in discontinuity, inconsistent illumination, and blurring, thus in favor of deepfake detection. Regardless of the facial appearance changes induced by deepfake, we argue that real and fake faces should correspond to their respective latent feature spaces. Particularly, the pair of real faces (original-target) remain close in the latent feature space, while the two pairs of real-fake faces (original-fake, target-fake) instead keep faraway. Following this paradigm, we suggest a triplet loss supervision network to extract the sufficiently discriminative deep features, which minimizes the distance of the original-target pair and maximize the distance of the original-fake (also target-fake) pair. The extensive results on public FaceForensics++ and Celeb-DF datasets validate the superiority of our method over competitors., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Zhongyuan Wang reports financial support was provided by National Natural Science Foundation of China., (Copyright © 2022 Elsevier Ltd. All rights reserved.)

Background: Suicide attempt is one of the most severe consequences for patients with mood disorders. This study aimed to perform deep learning and machine learning while using text mining to identify patients with suicide attempts and to compare their effectiveness., Methods: A total of 13,100 patients with mood disorders were selected. Two traditional text mining methods, logistic regression and Support vector machine (SVM), and one deep learning model (Convolutional neural network, CNN) were adopted to perform overall analysis and gender-specific subgroup analysis of patients to identify suicide attempts. The classification effectiveness of these models was evaluated by accuracy, F1-value, precision, recall, and the area under Receiver operator characteristic curve (ROC)., Results: CNN's results were greater than the other two for all indicators except recall which was slightly smaller than SVM in male subgroup analysis. The accuracy values of the CNN were 98.4 %, 98.2 %, and 98.5 % in the overall analysis and the subgroup analysis for males and females, respectively. The results of McNemar's test showed that CNN and SVM models' predictions were statistically different from the logistic regression model's predictions in the overall analysis and the subgroup analysis for females (P < 0.050)., Limitations: A fixed number of features were selected based on document frequency to train models; this was a single-site study., Conclusions: CNN model was a better way to detect suicide attempts in patients with mood disorders prior to hospital admission, saving time and resources in recognizing high-risk patients and preventing suicide., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Background: Metabolomics is a primary omics topic, which occupies an important position in both clinical applications and basic researches for metabolic signatures and biomarkers. Unfortunately, the relevant studies are challenged by the batch effect caused by many external factors. In last decade, the technique of deep learning has become a dominant tool in data science, such that one may train a diagnosis network from a known batch and then generalize it to a new batch. However, the batch effect inevitably hinders such efforts, as the two batches under consideration can be highly mismatched., Results: We propose an end-to-end deep learning framework, for joint batch effect removal and then classification upon metabolomics data. We firstly validate the proposed deep learning framework on a public CyTOF dataset as a simulated experiment. We also visually compare the t-SNE distribution and demonstrate that our method effectively removes the batch effects in latent space. Then, for a private MALDI MS dataset, we have achieved the highest diagnostic accuracy, with about 5.1 ~ 7.9% increase on average over state-of-the-art methods., Conclusions: Both experiments conclude that our method performs significantly better in classification than conventional methods benefitting from the effective removal of batch effect., (© 2022. The Author(s).)

The amount of available biological data has exploded since the emergence of high-throughput technologies, which is not only revolting the way we recognize molecules and diseases but also bringing novel analytical challenges to bioinformatics analysis. In recent years, deep learning has become a dominant technique in data science. However, classification accuracy is plagued with domain discrepancy. Notably, in the presence of multiple batches, domain discrepancy typically happens between individual batches. Most pairwise adaptation approaches may be suboptimal as they fail to eliminate external factors across multiple batches and take the classification task into account simultaneously. We propose a joint deep learning framework for integrating batch effect removal, classification, and downstream pathway activities upon biological data. To this end, we validate it on two MALDI MS-based metabolomics datasets. We have achieved the highest diagnostic accuracy (ACC), with a notable ∼10% improvement over other methods. Overall, these results indicate that our approach removes batch effect more effectively than state-of-the-art methods and yields more accurate classification as well as biomarkers for smart diagnosis.

Proton therapy requires accurate dose calculation for treatment planning to ensure the conformal doses are precisely delivered to the targets. The conversion of CT numbers to material properties is a significant source of uncertainty for dose calculation. The aim of this study is to develop a physics-informed deep learning (PIDL) framework to derive accurate mass density and relative stopping power maps from dual-energy computed tomography (DECT) images. The PIDL framework allows deep learning (DL) models to be trained with a physics loss function, which includes a physics model to constrain DL models. Five DL models were implemented including a fully connected neural network (FCNN), dual-FCNN (DFCNN), and three variants of residual networks (ResNet): ResNet-v1 (RN-v1), ResNet-v2 (RN-v2), and dual-ResNet-v2 (DRN-v2). An artificial neural network (ANN) and the five DL models trained with and without physics loss were explored to evaluate the PIDL framework. Two empirical DECT models were implemented to compare with the PIDL method. DL training data were from CIRS electron density phantom 062M (Computerized Imaging Reference Systems, Inc., Norfolk, VA). The performance of DL models was tested by CIRS adult male, adult female, and 5-year-old child anthropomorphic phantoms. For density map inference, the physics-informed RN-v2 was 3.3%, 2.9% and 1.9% more accurate than ANN for the adult male, adult female, and child phantoms. The physics-informed DRN-v2 was 0.7%, 0.6%, and 0.8% more accurate than DRN-v2 without physics training for the three phantoms, respectfully. The results indicated that physics-informed training could reduce uncertainty when ANN/DL models without physics training were insufficient to capture data structures or derived significant errors. DL models could also achieve better image noise control compared to the empirical DECT parametric mapping methods. The proposed PIDL framework can potentially improve proton range uncertainty by offering accurate material properties conversion from DECT., (© 2022 Institute of Physics and Engineering in Medicine.)

Oligomeric organization of G protein-coupled receptors is proposed to regulate receptor signaling and function, yet rapid and precise identification of the oligomeric status especially for native receptors on a cell membrane remains an outstanding challenge. By using blinking carbon dots (CDs), we now develop a deep learning (DL)-based blinking fingerprint recognition method, named deep-blinking fingerprint recognition (BFR), which allows automatic classification of CD-labeled receptor organizations on a cell membrane. This DL model integrates convolutional layers, long-short-term memory, and fully connected layers to extract time-dependent blinking features of CDs and is trained to a high accuracy (∼95%) for identifying receptor organizations. Using deep blinking fingerprint recognition, we found that CXCR4 mainly exists as 87.3% monomers, 12.4% dimers, and <1% higher-order oligomers on a HeLa cell membrane. We further demonstrate that the heterogeneous organizations can be regulated by various stimuli at different degrees. The receptor-binding ligands, agonist SDF-1α and antagonist AMD3100, can induce the dimerization of CXCR4 to 33.1 and 20.3%, respectively. In addition, cytochalasin D, which inhibits actin polymerization, similarly prompts significant dimerization of CXCR4 to 30.9%. The multi-pathway organization regulation will provide an insight for understanding the oligomerization mechanism of CXCR4 as well as for elucidating their physiological functions.

Purpose: Our purposes were (1) to explore the methodologic quality of the studies on the deep learning in knee imaging with CLAIM criterion and (2) to offer our vision for the development of CLAIM to assure high-quality reports about the application of AI to medical imaging in knee joint., Materials and Methods: A Checklist for Artificial Intelligence in Medical Imaging systematic review was conducted from January 1, 2015, to June 1, 2020, using PubMed, EMBASE, and Web of Science databases. A total of 36 articles discussing deep learning applications in knee joint imaging were identified, divided by imaging modality, and characterized by imaging task, data source, algorithm type, and outcome metrics., Results: A total of 36 studies were identified and divided into: X-ray (44.44%) and MRI (55.56%). The mean CLAIM score of the 36 studies was 27.94 (standard deviation, 4.26), which was 66.53% of the ideal score of 42.00. The CLAIM items achieved an average good inter-rater agreement (ICC 0.815, 95% CI 0.660-0.902). In total, 32 studies performed internal cross-validation on the data set, while only 4 studies conducted external validation of the data set., Conclusions: The overall scientific quality of deep learning in knee imaging is insufficient; however, deep learning remains a promising technology for diagnostic or predictive purpose. Improvements in study design, validation, and open science need to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application, pre-trained scoring procedure, and modification of CLAIM in response to clinical needs are necessary in the future., Key Points: • Limited deep learning studies were established in knee imaging with mean score of 27.94, which was 66.53% of the ideal score of 42.00, commonly due to invalidated results, retrospective study design, and absence of a clear definition of the CLAIM items in detail. • A previous trained data extraction instrument allowed reaching moderate inter-rater agreement in the application of the CLAIM, while CLAIM still needs improvement in scoring items and result reporting to become a wide adaptive tool in reviews of deep learning studies., (© 2021. European Society of Radiology.)

Purpose: The incidence of thyroid cancer has significantly increased in the last few decades. However, diagnosis of the thyroid nodules is labor and time intensive for radiologists and strongly depends on the personal experience of the radiologists. In this pursuit, the present study envisaged to develop a deep learning-based computer-aided diagnosis (CAD) method that enabled the automatic detection and classification of suspicious thyroid nodules in order to reduce the unnecessary fine-needle aspiration biopsy., Methods: The CAD method consisted of two main parts: detecting the location of thyroid nodules using a multiscale detection network and classifying the detected thyroid nodules by an attention-based classification network., Results: The performance of the proposed method was evaluated and compared with that of other state-of-the-art deep learning methods and experienced radiologists. The proposed detection method outperformed three other detection architectures (average precision, 82.1% vs. 78.3%, 77.2%, and 74.8%). Moreover, the classification method showed a superior performance compared with four other state-of-the-art classification networks (accuracy, 94.8% vs. 91.2%, 85.0%, 80.8%, and 72.1%) and that by experienced radiologists (mean value of area under the curve, 0.941 vs. 0.833)., Conclusions: Our study verified the high efficiency of the proposed detection method. The findings can help improve the diagnostic performance of radiologists. However, the developed CAD system requires more training and evaluation in a large-population study., (© 2021 American Association of Physicists in Medicine.)

Cervical cancer, one of the most common fatal cancers among women, can be prevented by regular screening to detect any precancerous lesions at early stages and treat them. Pap smear test is a widely performed screening technique for early detection of cervical cancer, whereas this manual screening method suffers from high false-positive results because of human errors. To improve the manual screening practice, machine learning (ML) and deep learning (DL) based computer-aided diagnostic (CAD) systems have been investigated widely to classify cervical Pap cells. Most of the existing studies require pre-segmented images to obtain good classification results. In contrast, accurate cervical cell segmentation is challenging because of cell clustering. Some studies rely on handcrafted features, which cannot guarantee the classification stage's optimality. Moreover, DL provides poor performance for a multiclass classification task when there is an uneven distribution of data, which is prevalent in the cervical cell dataset. This investigation has addressed those limitations by proposing DeepCervix, a hybrid deep feature fusion (HDFF) technique based on DL, to classify the cervical cells accurately. Our proposed method uses various DL models to capture more potential information to enhance classification performance. Our proposed HDFF method is tested on the publicly available SIPaKMeD dataset and compared the performance with base DL models and the late fusion (LF) method. For the SIPaKMeD dataset, we have obtained the state-of-the-art classification accuracy of 99.85%, 99.38%, and 99.14% for 2-class, 3-class, and 5-class classification. This method is also tested on the Herlev dataset and achieves an accuracy of 98.32% for 2-class and 90.32% for 7-class classification. The source code of the DeepCervix model is available at: https://github.com/Mamunur-20/DeepCervix., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

Background: Early diagnosis of Peritoneal metastasis (PM) is clinically significant regarding optimal treatment selection and avoidance of unnecessary surgical procedures. Cytopathology plays an important role in early screening of PM. We aimed to develop a deep learning (DL) system to achieve intelligent cytopathology interpretation, especially in ascites cytopathology., Methods: The original ascites cytopathology image dataset consists of 139 patients' original hematoxylin-eosin (HE) and Papanicolaou (PAP) Staining images. DL system was developed using transfer learning (TL) to achieve cell detection and classification. Pre-trained alexnet, vgg16, goolenet, resnet18 and resnet50 models were studied. Cell detection dataset consists of 176 cropped images with 6573 annotated cell bounding boxes. Cell classification data set consists of 487 cropped images with 18,558 and 6089 annotated malignant and benign cells in total, respectively., Results: We established a novel ascites cytopathology image dataset and achieved automatically cell detection and classification. DetectionNet based on Faster R-CNN using pre-trained resnet18 achieved cell detection with 87.22% of cells' Intersection of Union (IoU) bigger than the threshold of 0.5. The mean average precision (mAP) was 0.8316. The ClassificationNet based on resnet50 achieved the greatest performance in cell classification with AUC = 0.8851, Precision = 96.80%, FNR = 4.73%. The DL system integrating the separately trained DetectionNet and Classificationnet showed great performance in the cytopathology image interpretation., Conclusions: We demonstrate that the integration of DL can improve the efficiency of healthcare. The DL system we developed using TL techniques achieved accurate cytopathology interpretation, and had great potential to be integrated into clinician workflow.

The coronavirus disease (COVID-19) is rapidly spreading all over the world, and has infected more than 1,436,000 people in more than 200 countries and territories as of April 9, 2020. Detecting COVID-19 at early stage is essential to deliver proper healthcare to the patients and also to protect the uninfected population. To this end, we develop a dual-sampling attention network to automatically diagnose COVID-19 from the community acquired pneumonia (CAP) in chest computed tomography (CT). In particular, we propose a novel online attention module with a 3D convolutional network (CNN) to focus on the infection regions in lungs when making decisions of diagnoses. Note that there exists imbalanced distribution of the sizes of the infection regions between COVID-19 and CAP, partially due to fast progress of COVID-19 after symptom onset. Therefore, we develop a dual-sampling strategy to mitigate the imbalanced learning. Our method is evaluated (to our best knowledge) upon the largest multi-center CT data for COVID-19 from 8 hospitals. In the training-validation stage, we collect 2186 CT scans from 1588 patients for a 5-fold cross-validation. In the testing stage, we employ another independent large-scale testing dataset including 2796 CT scans from 2057 patients. Results show that our algorithm can identify the COVID-19 images with the area under the receiver operating characteristic curve (AUC) value of 0.944, accuracy of 87.5%, sensitivity of 86.9%, specificity of 90.1%, and F1-score of 82.0%. With this performance, the proposed algorithm could potentially aid radiologists with COVID-19 diagnosis from CAP, especially in the early stage of the COVID-19 outbreak.

Background: The novel coronavirus disease 2019 (COVID-19) constitutes a public health emergency globally. The number of infected people and deaths are proliferating every day, which is putting tremendous pressure on our social and healthcare system. Rapid detection of COVID-19 cases is a significant step to fight against this virus as well as release pressure off the healthcare system., Objective: One of the critical factors behind the rapid spread of COVID-19 pandemic is a lengthy clinical testing time. The imaging tool, such as Chest X-ray (CXR), can speed up the identification process. Therefore, our objective is to develop an automated CAD system for the detection of COVID-19 samples from healthy and pneumonia cases using CXR images., Methods: Due to the scarcity of the COVID-19 benchmark dataset, we have employed deep transfer learning techniques, where we examined 15 different pre-trained CNN models to find the most suitable one for this task., Results: A total of 860 images (260 COVID-19 cases, 300 healthy and 300 pneumonia cases) have been employed to investigate the performance of the proposed algorithm, where 70% images of each class are accepted for training, 15% is used for validation, and rest is for testing. It is observed that the VGG19 obtains the highest classification accuracy of 89.3% with an average precision, recall, and F1 score of 0.90, 0.89, 0.90, respectively., Conclusion: This study demonstrates the effectiveness of deep transfer learning techniques for the identification of COVID-19 cases using CXR images.

Magnetic resonance fingerprinting (MRF) is a quantitative imaging technique that can simultaneously measure multiple important tissue properties of human body. Although MRF has demonstrated improved scan efficiency as compared to conventional techniques, further acceleration is still desired for translation into routine clinical practice. The purpose of this paper is to accelerate MRF acquisition by developing a new tissue quantification method for MRF that allows accurate quantification with fewer sampling data. Most of the existing approaches use the MRF signal evolution at each individual pixel to estimate tissue properties, without considering the spatial association among neighboring pixels. In this paper, we propose a spatially constrained quantification method that uses the signals at multiple neighboring pixels to better estimate tissue properties at the central pixel. Specifically, we design a unique two-step deep learning model that learns the mapping from the observed signals to the desired properties for tissue quantification, i.e.: 1) with a feature extraction module for reducing the dimension of signals by extracting a low-dimensional feature vector from the high-dimensional signal evolution and 2) a spatially constrained quantification module for exploiting the spatial information from the extracted feature maps to generate the final tissue property map. A corresponding two-step training strategy is developed for network training. The proposed method is tested on highly undersampled MRF data acquired from human brains. Experimental results demonstrate that our method can achieve accurate quantification for T1 and T2 relaxation times by using only 1/4 time points of the original sequence (i.e., four times of acceleration for MRF acquisition).

High-grade gliomas are the most aggressive malignant brain tumors. Accurate pre-operative prognosis for this cohort can lead to better treatment planning. Conventional survival prediction based on clinical information is subjective and could be inaccurate. Recent radiomics studies have shown better prognosis by using carefully-engineered image features from magnetic resonance images (MRI). However, feature engineering is usually time consuming, laborious and subjective. Most importantly, the engineered features cannot effectively encode other predictive but implicit information provided by multi-modal neuroimages. We propose a two-stage learning-based method to predict the overall survival (OS) time of high-grade gliomas patient. At the first stage, we adopt deep learning, a recently dominant technique of artificial intelligence, to automatically extract implicit and high-level features from multi-modal, multi-channel preoperative MRI such that the features are competent of predicting survival time. Specifically, we utilize not only contrast-enhanced T1 MRI, but also diffusion tensor imaging (DTI) and resting-state functional MRI (rs-fMRI), for computing multiple metric maps (including various diffusivity metric maps derived from DTI, and also the frequency-specific brain fluctuation amplitude maps and local functional connectivity anisotropy-related metric maps derived from rs-fMRI) from 68 high-grade glioma patients with different survival time. We propose a multi-channel architecture of 3D convolutional neural networks (CNNs) for deep learning upon those metric maps, from which high-level predictive features are extracted for each individual patch of these maps. At the second stage, those deeply learned features along with the pivotal limited demographic and tumor-related features (such as age, tumor size and histological type) are fed into a support vector machine (SVM) to generate the final prediction result (i.e., long or short overall survival time). The experimental results demonstrate that this multi-model, multi-channel deep survival prediction framework achieves an accuracy of 90.66%, outperforming all the competing methods. This study indicates highly demanded effectiveness on prognosis of deep learning technique in neuro-oncological applications for better individualized treatment planning towards precision medicine.

Medical imaging plays a critical role in various clinical applications. However, due to multiple considerations such as cost and radiation dose, the acquisition of certain image modalities may be limited. Thus, medical image synthesis can be of great benefit by estimating a desired imaging modality without incurring an actual scan. In this paper, we propose a generative adversarial approach to address this challenging problem. Specifically, we train a fully convolutional network (FCN) to generate a target image given a source image. To better model a nonlinear mapping from source to target and to produce more realistic target images, we propose to use the adversarial learning strategy to better model the FCN. Moreover, the FCN is designed to incorporate an image-gradient-difference-based loss function to avoid generating blurry target images. Long-term residual unit is also explored to help the training of the network. We further apply Auto-Context Model to implement a context-aware deep convolutional adversarial network. Experimental results show that our method is accurate and robust for synthesizing target images from the corresponding source images. In particular, we evaluate our method on three datasets, to address the tasks of generating CT from MRI and generating 7T MRI from 3T MRI images. Our method outperforms the state-of-the-art methods under comparison in all datasets and tasks.

Significance: Analysis of modern large-scale, multicenter or diseased data requires deformable registration algorithms that can cope with data of diverse nature., Objective: We propose a novel deformable registration method, which is based on a cue-aware deep regression network, to deal with multiple databases with minimal parameter tuning., Methods: Our method learns and predicts the deformation field between a reference image and a subject image. Specifically, given a set of training images, our method learns the displacement vector associated with a pair of reference-subject patches. To achieve this, we first introduce a key-point truncated-balanced sampling strategy to facilitate accurate learning from the image database of limited size. Then, we design a cue-aware deep regression network, where we propose to employ the contextual cue, i.e., the scale-adaptive local similarity, to more apparently guide the learning process. The deep regression network is aware of the contextual cue for accurate prediction of local deformation., Results and Conclusion: Our experiments show that the proposed method can tackle various registration tasks on different databases, giving consistent good performance without the need of manual parameter tuning, which could be applicable to various clinical applications.

Train vibrations are the primary concern in environmental engineering and civil engineering. It is significantly imperative to find new methods for reducing and isolating vibrations. The locally resonant metamaterials (LRMs) propose a novel method and concept for reducing train vibration. However, the accurate and quick design structures of LRMs based on vibration characteristics are still an issue. Thus, this study presents a novel inverse design model of three-component locally resonant metamaterial barriers (LRMBs) for vibration reduction based on deep learning. The bandgap characteristics and vibration modes of the LRMB are investigated by using the improved plane wave expansion (IPWE) and finite element method (FEM). Besides, the gradient-combined LRMBs are proposed based on time–frequency features of measured vibration caused by trains and the novel inverse design model, and a two-dimensional finite element model coupling with infinite element boundaries is established to study the reduction efficiency of the gradient-combined LRMBs. And the performances of different LRMBs are fully analyzed in time and frequency domains. The results show that the novel inverse design model can be successfully used to design the LRMB based on vibration features. Moreover, the gradient-combined LRMBs have better isolation performance. [ABSTRACT FROM AUTHOR]

The sensor that simultaneously perceives bending strain and magnetic field has the potential to detect the finger bending state and hand position of the human and robot. Based on unique magneto-mechanical coupling effect of magnetostrictive materials, the proposed a bi-perceptive flexible sensor, consisting of the Co–Fe film and magnetic sensing plane coils, can realize dual information perception of strain/magnetic field through the change of magnetization state. The sensor structure and interface circuit of the sensing system are designed to provide high sensitivity and fast response, based on the input–output characteristics of the simulation model. An asynchronous multi-task deep learning method is proposed, which takes the output of the position task as the partial input of the bending state task to analyze the output information of the sensor quickly and accurately. The sensing system, integrating with the proposed model, can better predict the bending state and approach distance of human or robot hand. [ABSTRACT FROM AUTHOR]

Utilizing artificial intelligence methods for blood flow pressure estimation can significantly enhance the computational speed of blood flow pressure. However, current related research can only calculate the blood flow pressure parameters of vessels with different geometric shapes under fixed boundary conditions, thus fail to achieve transient flow field calculation and consider the hemodynamic differences formed by patients' varying physiological and pathological conditions. In view of this, this study proposes a method for relative pressure estimation based on four-dimensional flow magnetic resonance imaging (4D flow MRI) of patient blood flow and deep learning. 4D flow MRI was used to obtain the patient's blood flow velocity gradient data, and feature engineering processing is performed on the sampled data. Then, a novel neural network was proposed to acquire the characteristic relationship between velocity gradient and pressure gradient in the vicinity of the point to be measured and within adjacent sampling time periods, thereby achieving the calculation of the relative pressure in the vicinity of the point to be measured. Statistical analysis was performed to evaluate the efficacy of the method, comparing it with computational fluid dynamics methods and catheter pressure measurement techniques. The accuracy of the proposed method exceeded 96%, while computational efficiency was improved by several tens of times, and no manual setting of physiological parameters was required. Furthermore, the results were compared with clinical catheter-measured pressure results, r2 = 0.9053, indicating a significant consistency between the two methods. Compared to previous research, the method proposed in this study can take the blood flow velocity conditions of different patients at different times as input features via 4D flow MRI, thus enabling the calculation of pressure in transient flow fields, which significantly improved computational efficiency and reduced costs while maintaining a high level of calculation accuracy. This provides new direction for future research on machine learning prediction of blood flow pressure. [ABSTRACT FROM AUTHOR]

Simple Summary: Cotton is a crucial economic crop, but it is often threatened by various pests and diseases during its growth, significantly impacting its yield and quality. Earlier image classification methods often suffer from low accuracy and struggle to perform effectively in complex real-world environments. This paper proposes a novel image classification network named SpemNet, specifically designed for cotton pest and disease recognition. By introducing the Efficient Multi-Scale Attention (EMA) module and the Stacking Patch Embedding (SPE) module, the network enhances the ability to learn local features and integrate multi-scale information, thereby significantly improving the accuracy and efficiency of cotton pest and disease recognition. Extensive experiments conducted on the publicly available CottonInsect and IP102 datasets, as well as a self-collected cotton leaf disease dataset, demonstrate that SpemNet exhibits significant advantages in key metrics such as precision, recall, and F1 score, confirming its effectiveness and superiority in the task of cotton pest and disease recognition. We propose a cotton pest and disease recognition method, SpemNet, based on efficient multi-scale attention and stacking patch embedding. By introducing the SPE module and the EMA module, we successfully solve the problems of local feature learning difficulty and insufficient multi-scale feature integration in the traditional Vision Transformer model, which significantly improve the performance and efficiency of the model. In our experiments, we comprehensively validate the SpemNet model on the CottonInsect dataset, and the results show that SpemNet performs well in the cotton pest recognition task, with significant effectiveness and superiority. The SpemNet model excels in key metrics such as precision and F1 score, demonstrating significant potential and superiority in the cotton pest and disease recognition task. This study provides an efficient and reliable solution in the field of cotton pest and disease identification, which is of great theoretical and applied significance. [ABSTRACT FROM AUTHOR]

In the non-deterministic, large-scale navigation environment under the Mars exploration mission, there is a large space for action and many environmental states. Traditional reinforcement learning algorithms that can only obtain rewards at target points and obstacles will encounter the problems of reward sparsity and dimension explosion, making the training speed too slow or even impossible. This work proposes a deep layered learning algorithm based on the goal-driven layered deep Q-network (GDH-DQN), which is more suitable for mobile robots to explore, navigate, and avoid obstacles without a map. The algorithm model is designed in two layers. The lower layer provides behavioral strategies to achieve short-term goals, and the upper layer provides selection strategies for multiple short-term goals. Use known position nodes as short-term goals to guide the mobile robot forward and achieve long-term obstacle avoidance goals. Hierarchical execution not only simplifies tasks but also effectively solves the problems of reward sparsity and dimensionality explosion. In addition, each layer of the algorithm integrates a Hindsight Experience Replay mechanism to improve performance, make full use of the goal-driven function of the node, and effectively avoid the possibility of misleading the agent by complex processes and reward function design blind spots. The agent adjusts the number of model layers according to the number of short-term goals, further improving the efficiency and adaptability of the algorithm. Experimental results show that, compared with the hierarchical DQN method, the navigation success rate of the GDH-DQN algorithm is significantly improved, and it is more suitable for unknown scenarios such as Mars exploration. [ABSTRACT FROM AUTHOR]

Wearable devices, such as data gloves and electronic skins, can perceive human instructions, behaviors and even emotions by tracking a hand's motion, with the help of knowledge learning. The shape or position single-mode sensor in such devices often lacks comprehensive information to perceive interactive gestures. Meanwhile, the limited computing power of wearable applications restricts the multimode fusion of different sensing data and the deployment of deep learning networks. We propose a perceptive fusion electronic skin (PFES) with a bioinspired hierarchical structure that utilizes the magnetization state of a magnetostrictive alloy film to be sensitive to external strain or magnetic field. Installed at the joints of a hand, the PFES realizes perception of curvature (joint shape) and magnetism (joint position) information by mapping corresponding signals to the two-directional continuous distribution such that the two edges represent the contributions of curvature radius and magnetic field, respectively. By autonomously selecting knowledge closer to the user's hand movement characteristics, the reinforced knowledge distillation method is developed to learn and compress a teacher model for rapid deployment on wearable devices. The PFES integrating the autonomous learning algorithm can fuse curvature-magnetism dual information, ultimately achieving human machine interaction with gesture recognition and haptic feedback for cross-space perception and manipulation. [ABSTRACT FROM AUTHOR]

Adolescent idiopathic scoliosis (AIS) necessitates accurate spinal curvature assessment for effective clinical management. Traditional two-dimensional (2D) Cobb angle measurements have been the standard, but the emergence of three-dimensional (3D) automatic measurement techniques, such as those using weight-bearing 3D imaging (WR3D), presents an opportunity to enhance the accuracy and comprehensiveness of AIS evaluation. This study aimed to compare traditional 2D Cobb angle measurements with 3D automatic measurements utilizing the WR3D imaging technique in patients with AIS. A cohort of 53 AIS patients was recruited, encompassing 88 spinal curves, for comparative analysis. The patient sample consisted of 53 individuals diagnosed with AIS. Cobb angles were calculated using the conventional 2D method and three different 3D methods: the Analytical Method (AM), the Plane Intersecting Method (PIM), and the Plane Projection Method (PPM). The 2D cobb angle was manually measured by 3 experienced clinicians with 2D frontal whole-spine radiographs. For 3D cobb angle measurements, the spine and femoral heads were segmented from the WR3D images using a 3D-UNet deep-learning model, and the automatic calculations of the angles were performed with the 3D slicer software. AM and PIM estimates were found to be significantly larger than 2D measurements. Conversely, PPM results showed no statistical difference compared to the 2D method. These findings were consistent in a subgroup analysis based on 2D Cobb angles. Each 3D measurement method provides a unique assessment of spinal curvature, with PPM offering values closely resembling 2D measurements, while AM and PIM yield larger estimations. The utilization of WR3D technology alongside deep learning segmentation ensures accuracy and efficiency in comparative analyses. However, additional studies, particularly involving patients with severe curves, are required to validate and expand on these results. This study emphasizes the importance of selecting an appropriate measurement method considering the imaging modality and clinical context when assessing AIS, and it also underlines the need for continuous refinement of these techniques for optimal use in clinical decision-making and patient management. [ABSTRACT FROM AUTHOR]

Timely detection of self-replicating malware in the high market share Windows operating system can effectively prevent personal or corporate financial losses. The form and characteristics of malware are constantly evolving, leading to a concept drift issue that gradually decreases the effectiveness of traditional detection methods. Therefore, we propose WinMDet, a Windows malware detection method based on malicious tendency image and ResNet algorithm. First, to tackle the complexity and difficulty in accurately characterizing malware features, WinMDet retains detailed malware features and encodes them into malicious tendency images to better describe malware across different periods. Secondly, WinMDet utilizes previously generated malicious tendency images to train the initial detection model. Then, to alleviate the issue of malware concept drift, WinMDet employs Local Maximum Mean Discrepancy (LMMD) as the criterion for model transfer, enhancing the initial detection model's ability to distinguish between malware and benign software. We conducted a comprehensive evaluation of WinMDet using common metrics such as accuracy, precision and recall. The results indicate that WinMDet performs remarkably well in terms of accuracy, exceeding 82%. Additionally, significant improvements were observed in precision and recall, surpassing 82.42% and 82.06%, respectively. After employing our LMMD-based transfer method, the initial detection model improved the detection accuracy of malware in 2021 and 2022 by approximately 4.22% to 8.06%. The false negative rate decreased by at most 4.34%, and the false positive rate decreased by at most 4.61%. [ABSTRACT FROM AUTHOR]

With the popularization of intelligent devices and the development of internet technology, the Internet of Things technology is receiving increasing attention. However, how to effectively schedule and manage resources in the Internet of Things environment has become an increasingly important issue. In order to solve this problem, a cloud scheduling simulation system for IoT resources in a digital intelligent environment has emerged. This system utilizes cloud computing and IoT technology to simulate the behavior of intelligent devices and deep learning technology, achieving efficient resource scheduling. In addition, experimental data shows that the accuracy of the IoT resource cloud scheduling simulation system based on deep learning algorithm combined with digital intelligent environment is over 70%; the average task processing time is within 0.3 seconds; the average task response time is within 0.6 seconds. From this, it can be seen that combining deep learning algorithms with the system can quickly respond and solve problems, improving the reliability and stability of the system. The application range of this system is very wide, which can be used to optimize resource scheduling in many fields such as production, and can also be applied in fields such as smart home, smart medical, and intelligent transportation. With the continuous development of technology, this system would also be further improved and expanded to better serve people's lives and work. [ABSTRACT FROM AUTHOR]

In the data-rich everything-connected world, rapid and green data processing is essential, especially for some delay-sensitive and computation-intensive tasks. Motivated by these requirements, an energy and delay co-aware intelligent fog computation offloading and resource allocation scheme is conceived in this paper. Specifically, we formulate a weighted sum minimization problem of the task completion time and energy consumption at the local fog to achieve efficient task computation. A deep learning-based joint offloading decision and resource allocation (DL-JODRA) algorithm is developed to address this problem by jointly optimizing the offloading action, local CPU, bandwidth and external CPU occupation ratios. The optimal offloading decision based comprehensive optimization consideration of network resources further improves the network efficiency. Subsequently, to improve the efficiency and solution in a large-scale network scenario, a deep reinforcement learning-based joint offloading decision and resource allocation (DRL-JODRA) algorithm is constructed to obtain the optimal offloading and resource allocation policy. This approach is more suitable and stable for the solution of continuous-discrete action space by integrating the design of probabilistic discrete operation and inverting the gradient update. Finally, the extensive simulation results demonstrate that the proposed DL-JODRA and DRL-JODRA algorithms can rapidly achieve optimal offloading decision and resource allocation and gain a significant reduction in network costs (i.e., delay and energy) compared with other benchmark methods. [ABSTRACT FROM AUTHOR]

The objective of multimodal sentiment analysis is to extract and integrate feature information from text, image, and audio data accurately, in order to identify the emotional state of the speaker. While multimodal fusion schemes have made some progress in this research field, previous studies still lack adequate approaches for handling inter-modal information consistency and the fusion of different categorical features within a single modality. This study aims to effectively extract sentiment coherence information among video, audio, and text and consequently proposes a multimodal sentiment analysis method named joint chain interactive attention (VAE-JCIA, Video Audio Essay–Joint Chain Interactive Attention). In this approach, a 3D CNN is employed for extracting facial features from video, a Conformer is employed for extracting audio features, and a Funnel-Transformer is employed for extracting text features. Furthermore, the joint attention mechanism is utilized to identify key regions where sentiment information remains consistent across video, audio, and text. This process acquires reinforcing features that encapsulate information regarding consistency among the other two modalities. Inter-modal feature interactions are addressed through chained interactive attention, and multimodal feature fusion is employed to efficiently perform emotion classification. The method is experimentally validated on the CMU-MOSEI dataset and the IEMOCAP dataset. The experimental results demonstrate that the proposed method significantly enhances the performance of the multimodal sentiment analysis model. [ABSTRACT FROM AUTHOR]

Satellite remote sensing plays a significant role in the detection of smoke from forest fires. However, existing methods for detecting smoke from forest fires based on remote sensing images rely solely on the information provided by the images, overlooking the positional information and brightness temperature of the fire spots in forest fires. This oversight significantly increases the probability of misjudging smoke plumes. This paper proposes a smoke detection model, Forest Smoke-Fire Net (FSF Net), which integrates wildfire smoke images with the dynamic brightness temperature information of the region. The MODIS_Smoke_FPT dataset was constructed using a Moderate Resolution Imaging Spectroradiometer (MODIS), the meteorological information at the site of the fire, and elevation data to determine the location of smoke and the brightness temperature threshold for wildfires. Deep learning and machine learning models were trained separately using the image data and fire spot area data provided by the dataset. The performance of the deep learning model was evaluated using metric M A P , while the regression performance of machine learning was assessed with Root Mean Square Error ( R M S E ) and Mean Absolute Error ( M A E ). The selected machine learning and deep learning models were organically integrated. The results show that the Mask_RCNN_ResNet50_FPN and XGR models performed best among the deep learning and machine learning models, respectively. Combining the two models achieved good smoke detection results ( P r e c i s i o n smoke = 89.12 % ). Compared with wildfire smoke detection models that solely use image recognition, the model proposed in this paper demonstrates stronger applicability in improving the precision of smoke detection, thereby providing beneficial support for the timely detection of forest fires and applications of remote sensing. [ABSTRACT FROM AUTHOR]

Neuroimaging technology is considered a non-invasive method research the structure and function of the brain which have been widely used in neuroscience, psychiatry, psychology, and other fields. The development of Deep Learning Neural Network (DLNN), based on the deep learning algorithms of neural imaging techniques in brain disease diagnosis plays a more and more important role. In this paper, a deep neural network imaging technology based on Stack Auto-Encoder (SAE) feature extraction is constructed, and then Support Vector Machine (SVM) was used to solve binary classification problems (Alzheimer's disease [AD] and Mild Cognitive Impairment [MCI]). Four sets of experimental data were employed to perform the training and testing stages of DLNN. The number of neurons in each of the DLNNs was determined using the grid search technique. Overall, the results of DLNNs performance indicated that the SAE feature extraction was superior over (Accuracy Rate [AR] = 74.9% with structure of 93-171-49-22-93) shallow layer features extraction (AR = 70.8% with structure of 93-22-93) and primary features extraction (AR = 69.2%). [ABSTRACT FROM AUTHOR]

Through-wall radar human body pose recognition technology has broad applications in both military and civilian sectors. Identifying the current pose of targets behind walls and predicting subsequent pose changes are significant challenges. Conventional methods typically utilize radar information along with machine learning algorithms such as SVM and random forests to aid in recognition. However, these approaches have limitations, particularly in complex scenarios. In response to this challenge, this paper proposes a cross-modal supervised through-wall radar human body pose recognition method. By integrating information from both cameras and radar, a cross-modal dataset was constructed, and a corresponding deep learning network architecture was designed. During training, the network effectively learned the pose features of targets obscured by walls, enabling accurate pose recognition (e.g., standing, crouching) in scenarios with unknown wall obstructions. The experimental results demonstrated the superiority of the proposed method over traditional approaches, offering an effective and innovative solution for practical through-wall radar applications. The contribution of this study lies in the integration of deep learning with cross-modal supervision, providing new perspectives for enhancing the robustness and accuracy of target pose recognition. [ABSTRACT FROM AUTHOR]

As a vital anthropometric characteristic, human height information not only helps to understand overall developmental status and genetic risk factors, but is also important for forensic DNA phenotyping. We utilized linear regression analysis to test the association between each CpG probe and the height phenotype. Next, we designed a methylation sequencing panel targeting 959 CpGs and subsequent height inference models were constructed for the Chinese population. A total of 11,730 height-associated sites were identified. By employing KPCA and deep neural networks, a prediction model was developed, of which the cross-validation RMSE, MAE and R2 were 5.62 cm, 4.45 cm and 0.64, respectively. Genetic factors could explain 39.4% of the methylation level variance of sites used in the height inference models. Collectively, we demonstrated an association between height and DNA methylation status through an EWAS analysis. Targeted methylation sequencing of only 959 CpGs combined with deep learning techniques could provide a model to estimate human height with higher accuracy than SNP-based prediction models. [ABSTRACT FROM AUTHOR]