Multi-Scale Proposal Regression Network for Temporal Action Proposal Generation

Temporal action detection, as a branch of video analysis, aims to locate the time points when the actions start and end, and classify the actions occurred in videos into correct categories. Generating high-quality proposals is a key step in temporal action detection task. In this paper, we introduce a novel network, named multi-scale proposal regression network (MPRN), for temporal action proposal generation. First, we take encoding visual features as input and predict action scores for time points, in order to group them to generate rough proposals. Then, we regress the proposal's boundaries to obtain more precise proposals via our multi-scale proposal regression network. Compared with SSN and TURN, our multi-scale regression segments are characterized by flexible boundaries. Experiments show that 1) Our method is better than other proposal generation methods on THUMOS-14 dataset and ActivityNet-v1.3 dataset. 2) The effectiveness of our method is due to its own architecture, not the selection of visual feature encoders. 3) Our proposal generation method can generate temporal proposals for unseen action classes, which shows the good generalization ability of our proposal generation method.