1. ActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos
- Author
-
Adrien Chan-Hon-Tong, Guillaume Vaudaux-Ruth, Catherine Achard, Institut des Systèmes Intelligents et de Robotique (ISIR), Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Sorbonne Université (SU), Perception, Interaction, Robotique sociales (PIROS), Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Sorbonne Université, DTIS, ONERA, Université Paris Saclay (COmUE) [Palaiseau], ONERA-Université Paris Saclay (COmUE), DTIS, ONERA, Université Paris Saclay [Palaiseau], ONERA-Université Paris-Saclay, Vaudaux-Ruth, Guillaume, Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU), and Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU)
- Subjects
[INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI] ,FOS: Computer and information sciences ,Computer Science - Machine Learning ,Index Terms-Class ,Computer science ,Computer Science - Artificial Intelligence ,Computer Vision and Pattern Recognition (cs.CV) ,IEEEtran ,Computer Science - Computer Vision and Pattern Recognition ,02 engineering and technology ,010501 environmental sciences ,01 natural sciences ,Machine Learning (cs.LG) ,[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI] ,[INFO.INFO-CV] Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] ,0202 electrical engineering, electronic engineering, information engineering ,Reinforcement learning ,typesetting ,Reinforcement learning algorithm ,ComputingMilieux_MISCELLANEOUS ,0105 earth and related environmental sciences ,Ground truth ,business.industry ,paper ,template ,[INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] ,Pattern recognition ,Spotting ,style ,Artificial Intelligence (cs.AI) ,Action (philosophy) ,Pattern recognition (psychology) ,Key frame ,020201 artificial intelligence & image processing ,Artificial intelligence ,L A T E X ,business - Abstract
Summarizing video content is an important task in many applications. This task can be defined as the computation of the ordered list of actions present in a video. Such a list could be extracted using action detection algorithms. However, it is not necessary to determine the temporal boundaries of actions to know their existence. Moreover, localizing precise boundaries usually requires dense video analysis to be effective. In this work, we propose to directly compute this ordered list by sparsely browsing the video and selecting one frame per action instance, task known as action spotting in literature. To do this, we propose ActionSpotter, a spotting algorithm that takes advantage of Deep Reinforcement Learning to efficiently spot actions while adapting its video browsing speed, without additional supervision. Experiments performed on datasets THUMOS14 and ActivityNet show that our framework outperforms state of the art detection methods. In particular, the spotting mean Average Precision on THUMOS14 is significantly improved from 59.7% to 65.6% while skipping 23% of video.
- Published
- 2021