Search

Objectives: The microscopic evaluation of slides has been gradually moving towards all digital in recent years, leading to the possibility for computer-aided diagnosis. It is worthwhile to know the similarities between deep learning models and pathologists before we put them into practical scenarios. The simple criteria of colorectal adenoma diagnosis make it to be a perfect testbed for this study., Design: The deep learning model was trained by 177 accurately labelled training slides (156 with adenoma). The detailed labelling was performed on a self-developed annotation system based on iPad. We built the model based on DeepLab v2 with ResNet-34. The model performance was tested on 194 test slides and compared with five pathologists. Furthermore, the generalisation ability of the learning model was tested by extra 168 slides (111 with adenoma) collected from two other hospitals., Results: The deep learning model achieved an area under the curve of 0.92 and obtained a slide-level accuracy of over 90% on slides from two other hospitals. The performance was on par with the performance of experienced pathologists, exceeding the average pathologist. By investigating the feature maps and cases misdiagnosed by the model, we found the concordance of thinking process in diagnosis between the deep learning model and pathologists., Conclusions: The deep learning model for colorectal adenoma diagnosis is quite similar to pathologists. It is on-par with pathologists' performance, makes similar mistakes and learns rational reasoning logics. Meanwhile, it obtains high accuracy on slides collected from different hospitals with significant staining configuration variations., Competing Interests: Competing interests: SW is the cofounder, chief technology officer, and equity holder of Thorough Images. CL, ZS, CK are algorithm researchers of Thorough Images. All remaining authors have declared no conflicts of interest., (© Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Scene flow estimation is one of the most crucial components of many scene understanding tasks, which represents the complete 3D motion of objects in dynamic scene. Most of existing scene flow estimation approaches are usually based on joint learning framework, which cast the problem of scene flow estimation to dense prediction of optical flow and stereo matching. However, these approaches need to reconstruct 3D motion from optical flow and disparity using 2D stereo image pairs, which leads the estimation process to be indirect. Recently, FlowNet3D attempts to learn scene flow from 3D point clouds, which adopts element-wise max-pooling to aggregate features from different points. Nevertheless, max-pooling operation only can obtain the strongest activation on features across a local or global region, which may increase the loss of some useful detailed and contextual. Specifically, for dense estimation task, the ability to transfer information gradually from coarse to fine layers is important. To address this problem, we investigate a new deep architecture, 3D point convolutional network, to learn scene flow from 3D point clouds. This specific architecture uses multi-layer perceptron (MLP) to approximate the weight function and applies a density scale to re-weight the learned weight functions for each convolutional filter, which can make the network permutation-invariant and translation invariant on 3D space which is beneficial for feature aggregation. Extensive experimental results are conducted on FlyingThings3D and KITTI scene flow datasets and show the effectiveness of our proposed approach. Our algorithm can achieve state-of-the-art performance on FlyingThings3D and KITTI datasets. [ABSTRACT FROM AUTHOR]

Recently, optical flow estimation benefits greatly from deep learning based techniques. Most approaches use encoder-decoder architecture (U-Net) or spatial pyramid network (SPN) to learn optical flow. Both U-Net and SPN can extract multi-scale features and can predict optical flow directly. However, existing networks ignore to exploit the global information among channel features and inter-spatial relationship of features. In this paper, we propose a dual self-attention pyramid network, which adaptively integrates local features with their global dependencies and focuses on important features and suppresses unimportant features. Specifically, we introduce two types of attention modules into SPN, which emphasizes meaningful features along channel and spatial axes. The channel attention can adaptively re-weight channel-wise features by considering interdependencies among channels. Moreover, the spatial attention can utilize global contextual information to emphasize or suppress features in different spatial locations. In addition, two attention modules are embedded into each pyramidal level, which can refine features at different scale. We evaluate our method on MPI-Sintel and KITTI. The experimental results show that using the dual self-attention module can improve the representation power of network and further increase the accuracy of optical flow estimation. [ABSTRACT FROM AUTHOR]

Supervised depth prediction and optical flow estimation have achieved promising performance due to the advanced deep network architectures. Since the ground truths are difficult to be collected, many recent works try to learn the depth and flow in an unsupervised manner. However, existing methods only use features from convolutional layers or a simple aggregation of multi-level features to predict the depth and flow maps, which is insufficient to exploit context information. In this paper, we attempt to exploit object contextual information and investigate the effect of the object context for joint learning of depth and optical flow. Specifically, we present a novel combination of object context and the framework of joint learning depth and optical flow. Our proposed network can exploit and integrate the object context for both tasks by aggregating the context according to pair-wise similarities. Furthermore, we adopt the existing spatial pyramid network (SPN) to estimate the depth and flow in a coarse-to-fine strategy effectively. Given temporally adjacent stereo pairs, our network can be trained end-to-end in an unsupervised manner and can predict the depth and flow maps simultaneously. We conduct experiments on two publicly available datasets, KITTI2012 and KITTI2015. Our proposed approach yields comparable performance on both depth and flow tasks, compared to the recent deep learning-based approaches. Experimental results demonstrate that exploiting object contextual information is useful and beneficial for depth and optical flow estimation. [ABSTRACT FROM AUTHOR]

Image segmentation based on superpixel is used in urban and land cover change detection for fast locating region of interest. However, the segmentation algorithms often degrade due to speckle noise in synthetic aperture radar images. In this paper, a feature learning method using a stacked contractive autoencoder (sCAE) is presented to extract the temporal change feature from superpixel with noise suppression. First, an affiliated temporal change image, which obtains temporal difference in the pixel level, are built by three different metrics. Second, the simple linear iterative clustering algorithm is used to generate superpixels, which tightly adhere to the change image boundaries for the purpose of acquiring homogeneous change samples. Third, a sCAE network is trained with the superpixel samples as input to learn the change features in semantic. Then, the encoded features by this sCAE model are binary classified to create the change result map. Finally, the proposed method is compared with methods based on principal components analysis and Markov random fields. Experiment results show that our deep learning model can separate nonlinear noise efficiently from change features and obtain better performance in change detection for synthetic aperture radar images than conventional change detection algorithms. [ABSTRACT FROM AUTHOR]

Since remote sensing images are difficult to obtain and need to go through a complicated administrative procedure for use in China, it cannot meet the requirement of huge training samples for Waterside Change Detection based on deep learning. Recently, data augmentation has become an effective method to address the issue of an absence of training samples. Therefore, an improved Generative Adversarial Network (GAN), i.e., BTD-sGAN (Text-based Deeply-supervised GAN), is proposed to generate training samples for remote sensing images of Anhui Province, China. The principal structure of our model is based on Deeply-supervised GAN(D-sGAN), and D-sGAN is improved from the point of the diversity of the generated samples. First, the network takes Perlin Noise, image segmentation graph, and encoded text vector as input, in which the size of image segmentation graph is adjusted to 128 × 128 to facilitate fusion with the text vector. Then, to improve the diversity of the generated images, the text vector is used to modify the semantic loss of the downsampled text. Finally, to balance the time and quality of image generation, only a two-layer Unet++ structure is used to generate the image. Herein, "Inception Score", "Human Rank", and "Inference Time" are used to evaluate the performance of BTD-sGAN, StackGAN++, and GAN-INT-CLS. At the same time, to verify the diversity of the remote sensing images generated by BTD-sGAN, this paper compares the results when the generated images are sent to the remote sensing interpretation network and when the generated images are not added; the results show that the generated image can improve the precision of soil-moving detection by 5%, which proves the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

• To the best of our knowledge, this is the first survey paper to cover both optical flow and scene flow estimation comprehensively. • In contrast to existing surveys, this survey includes knowledge-driven, data-driven and hybrid-driven methods. • Comprehensive comparisons of existing methods on several datasets are provided with insightful observations and sufficient analyses. • Some of the open issues and potential research directions are discussed. Motion analysis is one of the most fundamental and challenging problems in the field of computer vision, which can be widely applied in many areas, such as autonomous driving, action recognition, scene understanding, and robotics. In general, the displacement field between subsequent frames can be divided into two types: optical flow and scene flow. The optical flow represents the pixel motion of adjacent frames. In contrast, the scene flow is a 3D motion field of the dynamic scene between two frames. Traditional approaches for the estimation of optical flow and scene flow usually leverage the variational technique, which can be solved as an energy minimization process. In recent years, deep learning has emerged as a powerful technique for learning feature representations directly from data. It has led to remarkable progress in the field of optical flow and scene flow estimation. In this paper, we provide a comprehensive survey of optical flow and scene flow estimation. First, we briefly review the pioneering approaches that use variational technique and then we delve in detail into the deep learning-based approaches. Furthermore, we present insightful observations on evaluation issues, specifically benchmark datasets, evaluation metrics, and state-of-the-art performance. Finally, we give the promising directions for future research. To the best of our knowledge, we are the first to review both optical flow and scene flow estimation, and the first to cover both traditional and deep learning-based approaches. [ABSTRACT FROM AUTHOR]

Learning optical flow based on convolutional neural networks has made great progress in recent years. These approaches usually design an encoder-decoder network that can be trained end-to-end. In encoder part, high-level feature information is extracted through a series of strided convolution, which is similar to most image classification networks. In contrast to classification task, spatial feature maps are then enlarged to full scale of input by conducting successive deconvolution layer in decoder part. However, optical flow estimation is a pixel-level task, and blurry flow fields are usually generated, which is caused by unrefined features and low-resolution. To address this problem, we propose a novel network, which combines attention mechanism and dilated convolutional neural network. In this network, the channel-wise features are adaptively weighted by building interdependencies among channels, which can weaken the weights of useless features and can enhance the directivity of feature extraction. Meanwhile, spatial precision is achieved by employing dilated convolution which improves the receptive field without large computational source and keeps the spatial resolution of feature map unchanged. Our network is trained on FlyingChairs and FlyingThings3D datasets in a supervised manner. Extensive experiments are conducted on MPI-Sintel and KITTI datasets to verify the effectiveness of the proposed method. The experimental results show that attention mechanism and dilated convolution are beneficial for optical flow estimation. Moreover, our method achieves better accuracy and visual improvements comparing to most of recent approaches. [ABSTRACT FROM AUTHOR]