Search

Active studies have been conducted on video saliency prediction, which predicts human visual attention toward videos. Most deep learning-based video saliency prediction models implicitly learn features that contribute to video saliency prediction, greatly improving accuracy. This study proposes a model called optical flow-based feature warping ViNet (OFF-ViNet). This model explicitly adds a Warping module, which is a mechanism that considers future predictions based on object motion in addition to implicitly learned features. The Warping module spatially warps the hierarchical features extracted by the 3D convolutional backbone based on the optical flow to obtain a feature representation that predicts the future. Compared with existing models, OFF-ViNet achieves better and competitive accuracy with state-of-the-art models on video saliency prediction datasets, particularly on UCF-Sports, which contains several videos with moving objects.

Various unsupervised anomaly detection methods using deep learning have recently been proposed, and the accuracy of the anomaly detection technique for local anomalies has been improved. However, no anomaly detection dataset includes co-occurrence-related anomalies, which are combination-related. Thus, the accuracy of anomaly detection for co-occurrence-related anomalies has not progressed. Therefore, we propose SA-PatchCore, which introduces self-attention to the state-of-the-art local anomaly detection model, PatchCore. It detects anomalies in co-occurrence relationships and anomalies in local areas with the benefit of the self-attention module, which can consider contexts between separated words introduced first in the natural language processing field. As no anomaly detection dataset includes anomalies in co-occurrence relation, we prepared a new dataset called the Co-occurrence Anomaly Detection Screw Dataset (CAD-SD). Furthermore, we performed experiments on anomaly detection using the new dataset. SA-PatchCore achieves high anomaly detection performance compared with PatchCore in the CAD-SD. Moreover, our proposed model shows almost the same anomaly detection performance as PatchCore in an MVTec Anomaly Detection dataset, which is composed of anomalies in a local area. As a contribution to the anomaly detection task, we have released the CAD-SD to the public. The code and dataset are publicly available at https://github.com/IshidaKengo/SA-PatchCore

Recently, accident prediction models, which predict the occurrence of traffic accidents through deep learning algorithms have been proposed. The application of these models demands both high precision and visualization of the decision basis applied. Current models use the motion features of objects in the surrounding environment, but they do not predict well when the motion feature of the risk factor is small. Meanwhile, drivers can avoid accidents because they utilize visual attention functions. This study focuses on the divergence between visual attention and focus of expansion (FOE), which are highly correlated in normal driving situations, as the basis for an accident prediction method. The proposed model can visualize decision basis with high accuracy, even when the motion feature of the risk factors is small, by combining it with Dynamic-Spatial-Attention, a deep-learning-based accident prediction method. In this experiment, we classified data from the Dashcam Accident Dataset, a widely used accident dataset, into categories of accidents. Using the Dashcam Accident Dataset, the proposed method achieves higher accident prediction performance in categories for which the motion feature of risk factors tends to be small while maintaining the same accident prediction performance as achieved by the baseline Dynamic-Spatial-Attention method in categories for which the motion feature of risk factors tends to be large. In addition, the proposed method visualizes the risk factors using visual attention and FOE to provide a visual explanation of the decision basis.