Search

Objectives: Anti-HER2 targeted therapy significantly reduces risk of relapse in HER2 + breast cancer. New measures are needed for a precise risk stratification to guide (de-)escalation of anti-HER2 strategy., Methods: A total of 726 HER2 + cases who received no/single/dual anti-HER2 targeted therapies were split into three respective cohorts. A deep learning model (DeepTEPP) based on preoperative breast magnetic resonance (MR) was developed. Patients were scored and categorized into low-, moderate-, and high-risk groups. Recurrence-free survival (RFS) was compared in patients with different risk groups according to the anti-HER2 treatment they received, to validate the value of DeepTEPP in predicting treatment efficacy and guiding anti-HER2 strategy., Results: DeepTEPP was capable of risk stratification and guiding anti-HER2 treatment strategy: DeepTEPP-Low patients (60.5%) did not derive significant RFS benefit from trastuzumab (p = 0.144), proposing an anti-HER2 de-escalation. DeepTEPP-Moderate patients (19.8%) significantly benefited from trastuzumab (p = 0.048), but did not obtain additional improvements from pertuzumab (p = 0.125). DeepTEPP-High patients (19.7%) significantly benefited from dual HER2 blockade (p = 0.045), suggesting an anti-HER2 escalation., Conclusions: DeepTEPP represents a pioneering MR-based deep learning model that enables the non-invasive prediction of adjuvant anti-HER2 effectiveness, thereby providing valuable guidance for anti-HER2 (de-)escalation strategies. DeepTEPP provides an important reference for choosing the appropriate individualized treatment in HER2 + breast cancer patients, warranting prospective validation., Clinical Relevance Statement: We built an MR-based deep learning model DeepTEPP, which enables the non-invasive prediction of adjuvant anti-HER2 effectiveness, thus guiding anti-HER2 (de-)escalation strategies in early HER2-positive breast cancer patients., Key Points: • DeepTEPP is able to predict anti-HER2 effectiveness and to guide treatment (de-)escalation. • DeepTEPP demonstrated an impressive prognostic efficacy for recurrence-free survival and overall survival. • To our knowledge, this is one of the very few, also the largest study to test the efficacy of a deep learning model extracted from breast MR images on HER2-positive breast cancer survival and anti-HER2 therapy effectiveness prediction., (© 2024. The Author(s).)

Accurate and rapid segmentation of the hippocampus can help doctors perform intractable temporal lobe epilepsy (TLE) preoperative evaluations to identify good surgical candidates. This study aims to establish a radiomics system for the automatic diagnosis of hippocampal sclerosis with the help of machine learning. A total of 240 cases were analysed to develop a diagnostic model. First, an automatic hippocampal segmentation process was established that exploits a priori knowledge of the relatively fixed location of the hippocampus in brain partitions, as well as a deep-learning segmentation network based on an Attention U-net. Then, we extracted 527 radiomics features from each side of the segmented hippocampus. The iterative sparse representation based on feature selection and a support vector machine classifier were finally used to establish the diagnostic model of hippocampal sclerosis. The diagnostic model consists of two consecutive steps: distinguish hippocampal sclerosis (HS) from normal control (NC) and detect whether the HS is located on the left or right side. When the automatic diagnosis model identified HS and NC, the sensitivity and specificity reached 0.941 and 0.917 in the 10-fold cross-validation set and 0.920 and 0.909 in the independent testing set. When the diagnostic model detected HS lateralization, the sensitivity and specificity reached 0.923 and 0.920 in cross-validation and 0.909 and 0.929 in independent testing. Our results show that the developed radiomics model can help detect TLE patients with hippocampal sclerosis and has the potential to simplify preoperative evaluations and select surgical candidates.

Background: Molecular diagnosis is crucial for biomarker-assisted glioma resection and management. However, some limitations of current molecular diagnostic techniques prevent their widespread use intraoperatively. With the unique advantages of ultrasound, this study developed a rapid intraoperative molecular diagnostic method based on ultrasound radio-frequency signals., Methods: We built a brain tumor ultrasound bank with 169 cases enrolled since July 2020, of which 43483 RF signal patches from 67 cases with a pathological diagnosis of glioma were a retrospective cohort for model training and validation. IDH1 and TERT promoter (TERTp) mutations and 1p/19q co-deletion were detected by next-generation sequencing. We designed a spatial-temporal integration model (STIM) to diagnose the three molecular biomarkers, thus establishing a rapid intraoperative molecular diagnostic system for glioma, and further analysed its consistency with the fifth edition of the WHO Classification of Tumors of the Central Nervous System (WHO CNS5). We tested STIM in 16-case prospective cohorts, which contained a total of 10384 RF signal patches. Two other RF-based classical models were used for comparison. Further, we included 20 cases additional prospective data for robustness test (ClinicalTrials.govNCT05656053)., Findings: In the retrospective cohort, STIM achieved a mean accuracy and AUC of 0.9190 and 0.9650 (95% CI, 0.94-0.99) respectively for the three molecular biomarkers, with a total time of 3 s and a 96% match to WHO CNS5. In the prospective cohort, the diagnostic accuracy of STIM is 0.85 ± 0.04 (mean ± SD) for IDH1, 0.84 ± 0.05 for TERTp, and 0.88 ± 0.04 for 1p/19q. The AUC is 0.89 ± 0.02 (95% CI, 0.84-0.94) for IDH1, 0.80 ± 0.04 (95% CI, 0.71-0.89) for TERTp, and 0.85 ± 0.06 (95% CI, 0.73-0.98) for 1p/19q. Compared to the second best available method based on RF signal, the diagnostic accuracy of STIM is improved by 16.70% and the AUC is improved by 19.23% on average., Interpretation: STIM is a rapid, cost-effective, and easy-to-manipulate AI method to perform real-time intraoperative molecular diagnosis. In the future, it may help neurosurgeons designate personalized surgical plans and predict survival outcomes., Funding: A full list of funding bodies that contributed to this study can be found in the Acknowledgements section., Competing Interests: Declaration of interests The authors declare that they have no competing interests., (Copyright © 2023 The Author(s). Published by Elsevier B.V. All rights reserved.)

Background: For patients with early-stage breast cancers, neoadjuvant treatment is recommended for non-luminal tumors instead of luminal tumors. Preoperative distinguish between luminal and non-luminal cancers at early stages will facilitate treatment decisions making. However, the molecular immunohistochemical subtypes based on biopsy specimens are not always consistent with final results based on surgical specimens due to the high intra-tumoral heterogeneity. Given that, we aimed to develop and validate a deep learning radiopathomics (DLRP) model to preoperatively distinguish between luminal and non-luminal breast cancers at early stages based on preoperative ultrasound (US) images, and hematoxylin and eosin (H&E)-stained biopsy slides., Methods: This multicentre study included three cohorts from a prospective study conducted by our team and registered on the Chinese Clinical Trial Registry (ChiCTR1900027497). Between January 2019 and August 2021, 1809 US images and 603 H&E-stained whole slide images (WSIs) from 603 patients with early-stage breast cancers were obtained. A Resnet18 model pre-trained on ImageNet and a multi-instance learning based attention model were used to extract the features of US and WSIs, respectively. An US-guided Co-Attention module (UCA) was designed for feature fusion of US and WSIs. The DLRP model was constructed based on these three feature sets including deep learning US feature, deep learning WSIs feature and UCA-fused feature from a training cohort (1467 US images and 489 WSIs from 489 patients). The DLRP model's diagnostic performance was validated in an internal validation cohort (342 US images and 114 WSIs from 114 patients) and an external test cohort (270 US images and 90 WSIs from 90 patients). We also compared diagnostic efficacy of the DLRP model with that of deep learning radiomics model and deep learning pathomics model in the external test cohort., Findings: The DLRP yielded high performance with area under the curve (AUC) values of 0.929 (95% CI 0.865-0.968) in the internal validation cohort, and 0.900 (95% CI 0.819-0.953) in the external test cohort. The DLRP also outperformed deep learning radiomics model based on US images only (AUC 0.815 [0.719-0.889], p = 0.027) and deep learning pathomics model based on WSIs only (AUC 0.802 [0.704-0.878], p = 0.013) in the external test cohort., Interpretation: The DLRP can effectively distinguish between luminal and non-luminal breast cancers at early stages before surgery based on pretherapeutic US images and biopsy H&E-stained WSIs, providing a tool to facilitate treatment decision making in early-stage breast cancers., Funding: Natural Science Foundation of Guangdong Province (No. 2023A1515011564), and National Natural Science Foundation of China (No. 91959127; No. 81971631)., Competing Interests: Declaration of interests All authors declare no competing interests., (Copyright © 2023 The Author(s). Published by Elsevier B.V. All rights reserved.)

Raman spectroscopy is a spectroscopic technique typically used to determine vibrational modes of molecules and is commonly used in chemistry to provide a structural fingerprint by which molecules can be identified. With the help of deep learning, Raman spectroscopy can be analyzed more efficiently and thus provide more accurate molecular information. However, no general neural network is designed for one-dimensional Raman spectral data so far. Furthermore, different combinations of hyperparameters of neural networks lead to results with significant differences, so the optimization of hyperparameters is a crucial issue in deep learning modeling. In this work, we propose a deep learning model designed for Raman spectral data and a hyperparameter optimization method to achieve its best performance, i.e., a method based on the simulated annealing algorithm to optimize the hyperparameters of the model. The proposed model and optimization method have been fully validated in a glioma Raman spectroscopy dataset. Compared with other published methods including linear regression, support vector regression, long short-term memory, VGG and ResNet, the mean squared error is reduced by 0.1557 while the coefficient determination is increased by 0.1195 on average., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Purpose: To explore a new approach mainly based on deep learning residual network (ResNet) to detect infarct cores on non-contrast CT images and improve the accuracy of acute ischemic stroke diagnosis., Methods: We continuously enrolled magnetic resonance diffusion weighted image (MR-DWI) confirmed first-episode ischemic stroke patients (onset time: less than 9 h) as well as some normal individuals in this study. They all underwent CT plain scan and MR-DWI scan with same scanning range, layer thickness (4 mm) and interlayer spacing (4 mm) (The time interval between two examinations: less than 4 h). Setting MR-DWI as gold standard of infarct core and using deep learning ResNet combined with a maximum a posteriori probability (MAP) model and a post-processing method to detect the infarct core on non-contrast CT images. After that, we use decision curve analysis (DCA) establishing models to analyze the value of this new method in clinical practice., Results: 116 ischemic stroke patients and 26 normal people were enrolled. 58 patients were allocated into training dataset and 58 were divided into testing dataset along with 26 normal samples. The identification accuracy of our ResNet based approach in detecting the infarct core on non-contrast CT is 75.9%. The DCA shows that this deep learning method is capable of improving the net benefit of ischemic stroke patients., Conclusions: Our deep learning residual network assisted with optimization methods is able to detect early infarct core on non-contrast CT images and has the potential to help physicians improve diagnostic accuracy in acute ischemic stroke patients., Competing Interests: Declaration of Competing Interest The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Radiomics has potential advantages in the noninvasive histopathological and molecular diagnosis of gliomas. We aimed to develop a novel image signature (IS)-based radiomics model to achieve multilayered preoperative diagnosis and prognostic stratification of gliomas. Herein, we established three separate case cohorts, consisting of 655 glioma patients, and carried out a retrospective study. Image and clinical data of three cohorts were used for training (N = 188), cross-validation (N = 411), and independent testing (N = 56) of the IS model. All tumors were segmented from magnetic resonance (MR) images by the 3D U-net, followed by extraction of high-throughput network features, which were referred to as IS. IS was then used to perform noninvasive histopathological diagnosis and molecular subtyping. Moreover, a new IS-based clustering method was applied for prognostic stratification in IDH-wild-type lower-grade glioma (IDHwt LGG) and triple-negative glioblastoma (1p19q retain/IDH wild-type/TERTp-wild-type GBM). The average accuracies of histological diagnosis and molecular subtyping were 89.8 and 86.1% in the cross-validation cohort, while these numbers reached 83.9 and 80.4% in the independent testing cohort. IS-based clustering method was demonstrated to successfully divide IDHwt LGG into two subgroups with distinct median overall survival time (48.63 vs 38.27 months respectively, P = 0.023), and two subgroups in triple-negative GBM with different median OS time (36.8 vs 18.2 months respectively, P = 0.013). Our findings demonstrate that our novel IS-based radiomics model is an effective tool to achieve noninvasive histo-molecular pathological diagnosis and prognostic stratification of gliomas. This IS model shows potential for future routine use in clinical practice.

Accurate identification of axillary lymph node (ALN) involvement in patients with early-stage breast cancer is important for determining appropriate axillary treatment options and therefore avoiding unnecessary axillary surgery and complications. Here, we report deep learning radiomics (DLR) of conventional ultrasound and shear wave elastography of breast cancer for predicting ALN status preoperatively in patients with early-stage breast cancer. Clinical parameter combined DLR yields the best diagnostic performance in predicting ALN status between disease-free axilla and any axillary metastasis with areas under the receiver operating characteristic curve (AUC) of 0.902 (95% confidence interval [CI]: 0.843, 0.961) in the test cohort. This clinical parameter combined DLR can also discriminate between low and heavy metastatic burden of axillary disease with AUC of 0.905 (95% CI: 0.814, 0.996) in the test cohort. Our study offers a noninvasive imaging biomarker to predict the metastatic extent of ALN for patients with early-stage breast cancer.

Microvascularity is highly correlated with the grading and subtyping of gliomas, making this one of its most important histological features. Accurate quantitative analysis of microvessels is helpful for the development of a targeted therapy for antiangiogenesis. The deep-learning algorithm is by far the most effective segmentation and detection model and enables location and recognition of complex microvascular networks in large images obtained from hematoxylin and eosin (H&E) stained specimens. We proposed an automated deep-learning-based method to detect and quantify the microvascularity in glioma and applied it to comprehensive clinical analyses. A total of 350 glioma patients were enrolled in our study, for which digitalized imaging of H&E stained slides were reviewed, molecular diagnosis was performed and follow-up was investigated. The microvascular features were compared according to their histologic types, molecular types, and patients' prognosis. The results show that the proposed method can quantify microvascular characteristics automatically and effectively. Significant increases of microvascular density and microvascular area were observed in glioblastomas (95% p < 0.001 in density, 170% p < 0.001 in area) in comparison with other histologic types; increases were also observed in cases with TERT-mut only (68% p < 0.001 in density, 54% p < 0.001 in area) compared with other molecular types. Survival analysis showed that microvascular features can be used to cluster cases into two groups with different survival periods (hazard ratio [HR] 2.843, log-rank <0.001), which indicates the quantified microvascular features may potentially be alternative signatures for revealing patients' prognosis. This deep-learning-based method may be a useful tool in routine clinical practice for precise diagnosis and antiangiogenic treatment.

Background: Improving imaging quality is a fundamental problem in ultrasound contrast agent imaging (UCAI) research. Plane wave imaging (PWI) has been deemed as a potential method for UCAI due to its' high frame rate and low mechanical index. High frame rate can improve the temporal resolution of UCAI. Meanwhile, low mechanical index is essential to UCAI since microbubbles can be easily broken under high mechanical index conditions. However, the clinical practice of ultrasound contrast agent plane wave imaging (UCPWI) is still limited by poor imaging quality for lack of transmit focus. The purpose of this study was to propose and validate a new post-processing method that combined with deep learning to improve the imaging quality of UCPWI. The proposed method consists of three stages: (1) first, a deep learning approach based on U-net was trained to differentiate the microbubble and tissue radio frequency (RF) signals; (2) then, to eliminate the remaining tissue RF signals, the bubble approximated wavelet transform (BAWT) combined with maximum eigenvalue threshold was employed. BAWT can enhance the UCA area brightness, and eigenvalue threshold can be set to eliminate the interference areas due to the large difference of maximum eigenvalue between UCA and tissue areas; (3) finally, the accurate microbubble imaging were obtained through eigenspace-based minimum variance (ESBMV)., Results: The proposed method was validated by both phantom and in vivo rabbit experiment results. Compared with UCPWI based on delay and sum (DAS), the imaging contrast-to-tissue ratio (CTR) and contrast-to-noise ratio (CNR) was improved by 21.3 dB and 10.4 dB in the phantom experiment, and the corresponding improvements were 22.3 dB and 42.8 dB in the rabbit experiment., Conclusions: Our method illustrates superior imaging performance and high reproducibility, and thus is promising in improving the contrast image quality and the clinical value of UCPWI.

Deep learning-based radiomics (DLR) was developed to extract deep information from multiple modalities of magnetic resonance (MR) images. The performance of DLR for predicting the mutation status of isocitrate dehydrogenase 1 (IDH1) was validated in a dataset of 151 patients with low-grade glioma. A modified convolutional neural network (CNN) structure with 6 convolutional layers and a fully connected layer with 4096 neurons was used to segment tumors. Instead of calculating image features from segmented images, as typically performed for normal radiomics approaches, image features were obtained by normalizing the information of the last convolutional layers of the CNN. Fisher vector was used to encode the CNN features from image slices of different sizes. High-throughput features with dimensionality greater than 1.6*10 4 were obtained from the CNN. Paired t-tests and F-scores were used to select CNN features that were able to discriminate IDH1. With the same dataset, the area under the operating characteristic curve (AUC) of the normal radiomics method was 86% for IDH1 estimation, whereas for DLR the AUC was 92%. The AUC of IDH1 estimation was further improved to 95% using DLR based on multiple-modality MR images. DLR could be a powerful way to extract deep information from medical images.

Background: Brain tumor segmentation is a challenging problem in medical image processing and analysis. It is a very time-consuming and error-prone task. In order to reduce the burden on physicians and improve the segmentation accuracy, the computer-aided detection (CAD) systems need to be developed. Due to the powerful feature learning ability of the deep learning technology, many deep learning-based methods have been applied to the brain tumor segmentation CAD systems and achieved satisfactory accuracy. However, deep learning neural networks have high computational complexity, and the brain tumor segmentation process consumes significant time. Therefore, in order to achieve the high segmentation accuracy of brain tumors and obtain the segmentation results efficiently, it is very demanding to speed up the segmentation process of brain tumors. Results: Compared with traditional computing platforms, the proposed FPGA accelerator has greatly improved the speed and the power consumption. Based on the BraTS19 and BraTS20 dataset, our FPGA-based brain tumor segmentation accelerator is 5.21 and 44.47 times faster than the TITAN V GPU and the Xeon CPU. In addition, by comparing energy efficiency, our design can achieve 11.22 and 82.33 times energy efficiency than GPU and CPU, respectively. Conclusion: We quantize and retrain the neural network for brain tumor segmentation and merge batch normalization layers to reduce the parameter size and computational complexity. The FPGA-based brain tumor segmentation accelerator is designed to map the quantized neural network model. The accelerator can increase the segmentation speed and reduce the power consumption on the basis of ensuring high accuracy which provides a new direction for the automatic segmentation and remote diagnosis of brain tumors. [ABSTRACT FROM AUTHOR]

Objectives: Brain arteriovenous malformation (AVM) is one of the most common causes of intracranial hemorrhage in young adults, and its expeditious diagnosis on digital subtraction angiography (DSA) is essential for clinical decision-making. This paper firstly proposed a deep learning network to extract vascular time-domain features from DSA videos. Then, the temporal features were combined with spatial radiomics features to build an AVM-assisted diagnosis model. Materials and method: Anteroposterior position (AP) DSA videos from 305 patients, 153 normal and 152 with AVM, were analyzed. A deep learning network based on Faster-RCNN was proposed to track important vascular features in DSA. Then the appearance order of important vascular structures was quantified as the temporal features. The structure distribution and morphological features of vessels were quantified as 1,750 radiomics features. Temporal features and radiomics features were fused in a classifier based on sparse representation and support vector machine. An AVM diagnosis and grading system that combined the temporal and spatial radiomics features of DSA was finally proposed. Accuracy (ACC), sensitivity (SENS), specificity (SPEC), and area under the receiver operating characteristic curve (AUC) were calculated to evaluate the performance of the radiomics model. Results: For cerebrovascular structure detection, the average precision (AP) was 0.922, 0.991, 0.769, 0.899, and 0.929 for internal carotid artery, Willis circle, vessels, large veins, and venous sinuses, respectively. The mean average precision (mAP) of five time phases was 0.902. For AVM diagnosis, the models based on temporal features, radiomics features, and combined features achieved AUC of 0.916, 0.918, and 0.942, respectively. In the AVM grading task, the proposed combined model also achieved AUC of 0.871 in the independent testing set. Conclusion: DSA videos provide rich temporal and spatial distribution characteristics of cerebral blood vessels. Clinicians often interpret these features based on subjective experience. This paper proposes a scheme based on deep learning and traditional machine learning, which effectively integrates the complex spatiotemporal features in DSA, and verifies the value of this scheme in the diagnosis of AVM. [ABSTRACT FROM AUTHOR]

Background: Lung ultrasound (LUS) can be an important imaging tool for the diagnosis and assessment of lung involvement. Ultrasound sonograms have been confirmed to illustrate damage to a person's lungs, which means that the correct classification and scoring of a patient's sonogram can be used to assess lung involvement.Methods: The purpose of this study was to establish a lung involvement assessment model based on deep learning. A novel multimodal channel and receptive field attention network combined with ResNeXt (MCRFNet) was proposed to classify sonograms, and the network can automatically fuse shallow features and determine the importance of different channels and respective fields. Finally, sonogram classes were transformed into scores to evaluate lung involvement from the initial diagnosis to rehabilitation.Results and Conclusion: Using multicenter and multimodal ultrasound data from 104 patients, the diagnostic model achieved 94.39% accuracy, 82.28% precision, 76.27% sensitivity, and 96.44% specificity. The lung involvement severity and the trend of COVID-19 pneumonia were evaluated quantitatively. [ABSTRACT FROM AUTHOR]

Purpose: Due to the low contrast, blurry boundaries, and large amount of shadows in breast ultrasound (BUS) images, automatic tumor segmentation remains a challenging task. Deep learning provides a solution to this problem, since it can effectively extract representative features from lesions and the background in BUS images. Methods: A novel automatic tumor segmentation method is proposed by combining a dilated fully convolutional network (DFCN) with a phase‐based active contour (PBAC) model. The DFCN is an improved fully convolutional neural network with dilated convolution in deeper layers, fewer parameters, and batch normalization techniques; and has a large receptive field that can separate tumors from background. The predictions made by the DFCN are relatively rough due to blurry boundaries and variations in tumor sizes; thus, the PBAC model, which adds both region‐based and phase‐based energy functions, is applied to further improve segmentation results. The DFCN model is trained and tested in dataset 1 which contains 570 BUS images from 89 patients. In dataset 2, a 10‐fold support vector machine (SVM) classifier is employed to verify the diagnostic ability using 460 features extracted from the segmentation results of the proposed method. Results: Advantages of the present method were compared with three state‐of‐the‐art networks; the FCN‐8s, U‐net, and dilated residual network (DRN). Experimental results from 170 BUS images show that the proposed method had a Dice Similarity coefficient of 88.97 ± 10.01%, a Hausdorff distance (HD) of 35.54 ± 29.70 pixels, and a mean absolute deviation (MAD) of 7.67 ± 6.67 pixels, which showed the best segmentation performance. In dataset 2, the area under curve (AUC) of the 10‐fold SVM classifier was 0.795 which is similar to the classification using the manual segmentation results. Conclusions: The proposed automatic method may be sufficiently accurate, robust, and efficient for medical ultrasound applications. [ABSTRACT FROM AUTHOR]

[Display omitted] • A triple-branch feature fusion network (TriFNet) is proposed for Raman spectroscopy-based pH detection. • TriFNet is able to fully exploit the spatial-channel features in the spectra to achieve accurate pH prediction. • The features learned by the model are biologically interpretable and the model has robustness to different SERS chips. Due to the acidic tumor microenvironment caused by metabolic changes in tumor cells, the accurate pH detection of extracellular fluid is helpful for doctors in precise tumor resection. The combination of Raman spectroscopy and deep learning provides a solution for pH detection. However, most existing studies use one-dimensional convolutional neural networks (1D-CNNs) for spectral analysis, which limits the performance due to insufficient feature extraction. In this work, we propose a 2D triple-branch feature fusion network (TriFNet) for accurate pH determination using surface-enhanced Raman spectra (SERS). Specifically, we design a triple-branch network structure by converting Raman spectra into three types of images to extensively extract complex patterns in spectra. In addition, an attention fusion module, which leverages the complementarity among features in both space and channel, is designed to obtain the valuable information, achieving further accurate pH determination. On our Raman spectral dataset containing 14,137 samples, we achieved mean absolute error (MAE) of 0.059, standard deviation of the absolute error (SD) of 0.07, root mean squared error (RMSE) of 0.092, and coefficient of determination (R2) of 0.991 on the test set. Compared with other published methods, the four metrics showed an average improvement of 47%, 39%, 43%, and 6%, respectively. In addition, visualization validates the diagnostic capability of our model to correlate with biomolecular signatures. Meanwhile, our model has robustness to different SERS chips. These results prove the potential of our method to develop an effective technology based on Raman spectroscopy for accurate pH determination to guide surgery. [ABSTRACT FROM AUTHOR]

Ultrasonography is one of the main imaging methods for monitoring and diagnosing atherosclerosis due to its non-invasiveness and low-cost. Automatic differentiation of carotid plaque fibrous cap integrity by using multi-modal ultrasound videos has significant diagnostic and prognostic value for cardiovascular and cerebrovascular disease patients. However, the task faces several challenges, including high variation in plaque location and shape, the absence of analysis mechanism focusing on fibrous cap, the lack of effective mechanism to capture the relevance among multi-modal data for feature fusion and selection, etc. To overcome these challenges, we propose a new target boundary and perfusion feature guided video analysis network (BP-Net) based on conventional B-mode ultrasound and contrast-enhanced ultrasound videos for assessing the integrity of fibrous cap. Based on our previously proposed plaque auto-tracking network, in our BP-Net, we further introduce the plaque edge attention module and reverse mechanism to focus the dual video analysis on the fiber cap of plaques. Moreover, to fully explore the rich information on the fibrous cap and inside/outside of the plaque, we propose a feature fusion module for B-mode and contrast video to filter out the most valuable features for fibrous cap integrity assessment. Finally, multi-head convolution attention is proposed and embedded into transformer-based network, which captures semantic features and global context information to obtain accurate evaluation of fibrous caps integrity. The experimental results demonstrate that the proposed method has high accuracy and generalizability with an accuracy of 92.35% and an AUC of 0.935, which outperforms than the state-of-the-art deep learning based methods. A series of comprehensive ablation studies suggest the effectiveness of each proposed component and show great potential in clinical application. [Display omitted] • The first boundary and perfusion feature guided dual-modality ultrasound video analysis network for fibrous cap integrity assessment. • A plaque edge attention module is proposed to focus the dual video analysis on the fibrous cap of plaques. • The proposed multi-modal fusion block can explore the inside/outside of the plaque and rich inter/intra modal interactions. • The multi-head convolution attention can utilize the robustness of CNN and sophisticated global modeling of Transformer. [ABSTRACT FROM AUTHOR]

• The first automatic tracking and segmentation network for ultrasound video of multiple carotid plaques, and the effectiveness is confirmed by comprehensive experiments. • A new multi-object detector dual attention U-Net is proposed to segment carotid plaques and blood vessels simultaneously. • The newly proposed single-object tracker (MSTUnet) can utilize the previous tracking results efficiently and achieve stable tracking via a ranking channel attention module (RCAM), which is presented to select valuable information from previous tracking results. • The proposed method can detect target automatically and adjust the number of targets without manual initialization. Carotid plaque tracking and segmentation in ultrasound videos is the premise for subsequent plaque property evaluation and treatment plan development. However, the task is quite challenging, as it needs to address the problems of poor image quality, plaque shape variations among frames, the existence of multiple plaques, etc. To overcome these challenges, we propose a new automatic multi-plaque tracking and segmentation (AMPTS) framework. AMPTS consists of three modules. The first module is a multi-object detector, in which a Dual Attention U-Net is proposed to detect multiple plaques and vessels simultaneously. The second module is a set of single-object trackers that can utilize the previous tracking results efficiently and achieve stable tracking of the current target by using channel attention and a ranking strategy. To make the first module and the second module work together, a parallel tracking module based on a simplified 'tracking-by-detection' mechanism is proposed to solve the challenge of tracking object variation. Extensive experiments are conducted to compare the proposed method with several state-of-the-art deep learning based methods. The experimental results demonstrate that the proposed method has high accuracy and generalizability with a Dice similarity coefficient of 0.83 which is 0.16, 0.06 and 0.27 greater than MAST (Lai et al., 2020), Track R-CNN (Voigtlaender et al., 2019) and VSD (Yang et al., 2019) respectively and has made significant improvements on seven other indicators. In the additional Testing set 2, our method achieved a Dice similarity coefficient of 0.80, an accuracy of 0.79, a precision of 0.91, a Recall 0.70, a F1 score of 0.79, an AP@0.5 of 0.92, an AP@0.7 of 0.74, and an expected average overlap of 0.79. Numerous ablation studies suggest the effectiveness of each proposed component and the great potential for multiple carotid plaques tracking and segmentation in clinical practice. [Display omitted] [ABSTRACT FROM AUTHOR]

• Quantitative evaluation of liver fibrosis based on ultrasound radio frequency signal. • Bidirectional long short-term memory network designed for the data mining. • Deep learning with radio frequency signal provides visual distribution of fibrosis. Chronic liver disease is an important cause of liver failure and death worldwide, and liver fibrosis is a common pathological process of most chronic liver diseases. There still lacks a useful tool for evaluating liver fibrosis progression precisely and non-invasively. The purpose of this study was to explore the use of ultrasound radio frequency (RF) signals combined with deep learning approach to evaluate the degree of liver fibrosis quantitatively. In this study, by extracting the output of deep learning models as a prediction value, a quantitative liver fibrosis prediction method was achieved based on the bidirectional long short-term memory (Bi-LSTM) network to analyze radio frequency (RF) signals. The dataset consisted of 160 sets of ultrasound RF signals of rat livers, including five fibrosis stages 0−4, upon pathological diagnosis. In total, 150 sets of RF signals were used to train four deep learning classification models, the output of which contained quantitative information. In each training stage of the four models, a large number of signal segments were extracted from the 150 sets and divided randomly into training and validation sets in a ratio of 80:20. Ten sets of RF data using the gold standard of quantitative fibrosis parameter (q-FP) of liver tissues were left for independent testing. To validate the proposed method, correlation analysis was carried out between q-FP and the quantitative prediction results based on the independent test data. The accuracy of the four deep learning networks using the training and validation data was above 0.83 and 0.80, and the corresponding areas under the receiver operating characteristic curves were higher than 0.95 and 0.93, respectively. For the quantitative analysis in the independent test set, the determination coefficient, R2, of the linear regression analysis between the quantitative prediction results and q-FP was above 0.93. liver fibrosis is a common pathological process of most chronic liver diseases. This study indicates that a prediction system based on ultrasound RF signals and a deep learning approach is promising for realizing quantitative and visualized diagnosis of liver fibrosis, which would be of great value in monitoring liver fibrosis non-invasively. The figure shows the results of quantitative and visual diagnosis of hepatic fibrosis by our deep learning network model. The pre-trained deep learning networks were applied to extract high-throughput features of the 10 sets of RF data in the independent test set and analyze them quantitatively, the overall fibrosis degrees were accurately predicted using a voting strategy, The prediction values of the RF segments are displayed on their corresponding ultrasound images in color, and their mean values are also given. The proposed method gives not only the overall decision of the liver fibrosis degree, but also the possibility of every fibrosis degree that the analyzed object may have in a pie chart. we could clearly see the distribution of predicted liver fibrosis in the areas of interest. The darker the color, the lower the fibrosis grade, and the lighter the color, the higher the liver fibrosis grade. Image, graphical abstract [ABSTRACT FROM AUTHOR]