1. MBGNet:Multi-branch boundary generation network with temporal context aggregation for temporal action detection.
- Author
-
Pan, Xiaoying, Zhang, Nijuan, Xie, Hewei, Li, Shoukun, and Feng, Tong
- Subjects
TIME-varying networks ,CLASS actions ,BABY strollers ,DETECTORS ,VIDEOS - Abstract
Temporal action detection is an important and fundamental video understanding task that aims to locate the temporal regions where human actions or events may occur and to identify the classes of actions in untrimmed videos. The main challenge of temporal action detection is that videos are usually of different durations and untrimmed. Although existing methods have achieved better results in recent years, there are still some challenges, such as a lack of full utilisation of video context features, insufficient accuracy of generated action boundaries and failure to consider the relationship between proposals. To address the above issues, this paper proposes a Multi-branch Boundary Generation Network (MBGNet) with temporal context aggregation. It improves the performance of temporal action proposal generation by exploiting rich temporal context features and complementary boundary generators.First, we propose a multi-path temporal context feature aggregation (MTCA) module to exploit "local and global" contextual temporal features for the generation of temporal action proposals. Second, in order to generate accurate action boundaries, we design a multi-branch temporal boundary detector (MBG) to optimise the prediction results by exploiting the complementary relationship between the two boundary detectors.In addition, to accurately predict the confidence of densely distributed proposals, we design a proposal relation-aware module (PRAM) that exploits global correlation for proposal relationship modelling. Experiments on the popular datasets ActivityNet1.3, THUMOS14, and HACS demonstrate the effectiveness of the method proposed in this paper on the task of temporal action proposal generation, which can generate action proposals with high precision and recall. Moreover, combining with existing action classifiers can also achieve better performance in temporal action detection.These results demonstrate the effectiveness of the method in this paper in improving the accuracy of temporal action proposal generation and detection. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF