Search

Learning self-supervised image representations has been broadly studied to boost various visual understanding tasks. Existing methods typically learn a single level of image semantics like pairwise semantic similarity or image clustering patterns. However, these methods can hardly capture multiple levels of semantic information that naturally exists in an image dataset, e.g., the semantic hierarchy of "Persian cat to cat to mammal" encoded in an image database for species. It is thus unknown whether an arbitrary image self-supervised learning (SSL) approach can benefit from learning such hierarchical semantics. To answer this question, we propose a general framework for Hierarchical Image Representation Learning (HIRL). This framework aims to learn multiple semantic representations for each image, and these representations are structured to encode image semantics from fine-grained to coarse-grained. Based on a probabilistic factorization, HIRL learns the most fine-grained semantics by an off-the-shelf image SSL approach and learns multiple coarse-grained semantics by a novel semantic path discrimination scheme. We adopt six representative image SSL methods as baselines and study how they perform under HIRL. By rigorous fair comparison, performance gain is observed on all the six methods for diverse downstream tasks, which, for the first time, verifies the general effectiveness of learning hierarchical image semantics. All source code and model weights are available at https://github.com/hirl-team/HIRL, Comment: Research project paper. arXiv v1: all source code and model weights released

Introduction: Background field removal (BFR) is a critical step required for successful quantitative susceptibility mapping (QSM). However, eliminating the background field in brains containing significant susceptibility sources, such as intracranial hemorrhages, is challenging due to the relatively large scale of the field induced by these pathological susceptibility sources. Method: This study proposes a new deep learning-based method, BFRnet, to remove background field in healthy and hemorrhagic subjects. The network is built with the dual-frequency octave convolutions on the U-net architecture, trained with synthetic field maps containing significant susceptibility sources. The BFRnet method is compared with three conventional BFR methods and one previous deep learning method using simulated and in vivo brains from 4 healthy and 2 hemorrhagic subjects. Robustness against acquisition field-of-view (FOV) orientation and brain masking are also investigated. Results: For both simulation and in vivo experiments, BFRnet led to the best visually appealing results in the local field and QSM results with the minimum contrast loss and the most accurate hemorrhage susceptibility measurements among all five methods. In addition, BFRnet produced the most consistent local field and susceptibility maps between different sizes of brain masks, while conventional methods depend drastically on precise brain extraction and further brain edge erosions. It is also observed that BFRnet performed the best among all BFR methods for acquisition FOVs oblique to the main magnetic field. Conclusion: The proposed BFRnet improved the accuracy of local field reconstruction in the hemorrhagic subjects compared with conventional BFR algorithms. The BFRnet method was effective for acquisitions of titled orientations and retained whole brains without edge erosion as often required by traditional BFR methods., Comment: 23 pages, 8 figures, 2 tables

Hierarchical semantic structures naturally exist in an image dataset, in which several semantically relevant image clusters can be further integrated into a larger cluster with coarser-grained semantics. Capturing such structures with image representations can greatly benefit the semantic understanding on various downstream tasks. Existing contrastive representation learning methods lack such an important model capability. In addition, the negative pairs used in these methods are not guaranteed to be semantically distinct, which could further hamper the structural correctness of learned image representations. To tackle these limitations, we propose a novel contrastive learning framework called Hierarchical Contrastive Selective Coding (HCSC). In this framework, a set of hierarchical prototypes are constructed and also dynamically updated to represent the hierarchical semantic structures underlying the data in the latent space. To make image representations better fit such semantic structures, we employ and further improve conventional instance-wise and prototypical contrastive learning via an elaborate pair selection scheme. This scheme seeks to select more diverse positive pairs with similar semantics and more precise negative pairs with truly distinct semantics. On extensive downstream tasks, we verify the superior performance of HCSC over state-of-the-art contrastive methods, and the effectiveness of major model components is proved by plentiful analytical studies. We build a comprehensive model zoo in Sec. D. Our source code and model weights are available at https://github.com/gyfastas/HCSC, Comment: Accepted by CVPR 2022. arXiv v3: 800 epoch multi-crop model released; arXiv v2: more model weights released; arXiv v1: code & model weights released

Introduction: Quantitative Susceptibility Mapping (QSM) is generally acquired with full brain coverage, even though many QSM brain-iron studies focus on the deep grey matter (DGM) region only. Reducing the spatial coverage to the DGM vicinity can substantially shorten the scan time or enhance the spatial resolution without increasing scan time; however, this may lead to significant DGM susceptibility underestimation. Method: A recently proposed deep learning-based QSM method, namely xQSM, is investigated to assess the accuracy of dipole inversion on reduced brain coverages. Pre-processed magnetic field maps are extended symmetrically from the centre of globus pallidus in the coronal plane to simulate QSM acquisitions of difference spatial coverages.Results: The proposed xQSM network led to the lowest DGM contrast lost with the smallest susceptibility variation range across all spatial coverages. For the digital brain phantom simulation, xQSM improved the DGM susceptibility underestimation more than 20% in small spatial coverages. For the in vivo acquisition, less than 5% DGM susceptibility error was achieved in 48 mm axial slabs using the xQSM network, while a minimum of 112 mm coverage was required for conventional methods. It is also shown that the background field removal process performed worse in reduced brain coverages, which further deteriorated the subsequent dipole inversion. Conclusion: The recently proposed deep learning-based xQSM method significantly improves the accuracy of DGM QSM from small spatial coverages as compared with conventional QSM algorithms, which can shorten DGM QSM acquisition time substantially., Comment: 25 pages, 8 figures, 1 supplementary figure and 1 supplementary table

Quantitative susceptibility mapping (QSM) is a valuable magnetic resonance imaging (MRI) contrast mechanism that has demonstrated broad clinical applications. However, the image reconstruction of QSM is challenging due to its ill-posed dipole inversion process. In this study, a new deep learning method for QSM reconstruction, namely xQSM, was designed by introducing modified state-of-the-art octave convolutional layers into the U-net backbone. The xQSM method was compared with recentlyproposed U-net-based and conventional regularizationbased methods, using peak signal to noise ratio (PSNR), structural similarity (SSIM), and region-of-interest measurements. The results from a numerical phantom, a simulated human brain, four in vivo healthy human subjects, a multiple sclerosis patient, a glioblastoma patient, as well as a healthy mouse brain showed that the xQSM led to suppressed artifacts than the conventional methods, and enhanced susceptibility contrast, particularly in the ironrich deep grey matter region, than the original U-net, consistently. The xQSM method also substantially shortened the reconstruction time from minutes using conventional iterative methods to only a few seconds., Comment: 37 pages, 10 figures, 3 table

Neural networks in the real domain have been studied for a long time and achieved promising results in many vision tasks for recent years. However, the extensions of the neural network models in other number fields and their potential applications are not fully-investigated yet. Focusing on color images, which can be naturally represented as quaternion matrices, we propose a quaternion convolutional neural network (QCNN) model to obtain more representative features. In particular, we redesign the basic modules like convolution layer and fully-connected layer in the quaternion domain, which can be used to establish fully-quaternion convolutional neural networks. Moreover, these modules are compatible with almost all deep learning techniques and can be plugged into traditional CNNs easily. We test our QCNN models in both color image classification and denoising tasks. Experimental results show that they outperform the real-valued CNNs with same structures.