Search

Intelligent medicine is eager to automatically generate radiology reports to ease the tedious work of radiologists. Previous researches mainly focused on the text generation with encoder-decoder structure, while CNN networks for visual features ignored the long-range dependencies correlated with textual information. Besides, few studies exploit cross-modal mappings to promote radiology report generation. To alleviate the above problems, we propose a novel end-to-end radiology report generation model dubbed Self-Supervised dual-Stream Network (S3-Net). Specifically, a Dual-Stream Visual Feature Extractor (DSVFE) composed of ResNet and SwinTransformer is proposed to capture more abundant and effective visual features, where the former focuses on local response and the latter explores long-range dependencies. Then, we introduced the Fusion Alignment Module (FAM) to fuse the dual-stream visual features and facilitate alignment between visual features and text features. Furthermore, the Self-Supervised Learning with Mask(SSLM) is introduced to further enhance the visual feature representation ability. Experimental results on two mainstream radiology reporting datasets (IU X-ray and MIMIC-CXR) show that our proposed approach outperforms previous models in terms of language generation metrics.

With the development of modern industry, fault diagnosis has become an important research field. Currently, many methods for fault diagnosis have been proposed. As a method designed to overcome the high computational complexity of the convolutional neural network (CNN), PCANet is also used in the field of fault diagnosis. The core algorithm of the PCANet is the principal component analysis (PCA) algorithm. However, PCA is a global dimensionality reduction method, which cannot effectively analyze the local spatial geometry structure of data and may even weaken or destroy the local structure information. Furthermore, algorithms based on L2-norm are very sensitive to noise and outliers. To address the problems, a generalized neighborhood preserving embedding (GNPE) method based on Lp-norm is first proposed, and it is used to learn the convolutional filters of CNN. In this way, a novel CNN with local structure is proposed, called the GNPE Network (GNPENet). We conduct fault diagnosis experiments on five datasets of three types and comprehensively evaluate the proposed GNPENet method. The experimental results show that, compared with some state-of-the-art models, GNPENet has better feature extraction and generalization capabilities.