1. A hybrid offline-online method for sound event localization and detection.
- Author
-
Zhang, Wenjie, Yu, Peng, Wang, Zhan, Wang, Zhenhe, and Xu, Mingliang
- Subjects
DIRECTIONAL hearing ,ACOUSTIC localization - Abstract
The objective of sound event localization and detection (SELD) is to accurately identify the temporal occurrence and spatial coordinates of a specific sound category. The existing mainstream offline methods may unintentionally introduce unfavorable future feature information during the training process, thereby potentially hindering the system's performance. The utilization of online methods can lead to improved localization accuracy to a certain extent. Nevertheless, it may result in a diminished ability with the detection capability for sound events. In this paper, a hybrid offline-online method (HOOM) is proposed that involves extracting comprehensive audio information using offline network layers, while simultaneously filtering out irrelevant future information using online network layers. Based on this method, we designed two simple sub-network architectures. The first, convolution and causal convolution alternating network (CCAN), employs regular convolution along with causal convolutions to achieve the offline and online convolution features, respectively. The second, bidirectional and unidirectional alternating network (BUAN), combines bidirectional recurrent neural networks with unidirectional recurrent neural networks, capturing the offline and online contextual sequence information, respectively. Our proposed method demonstrates a 6% improvement in localization recall on the Sony-TAU Realistic Spatial Soundscapes 2023 (STARSS23) dataset. Furthermore, compared to offline or online methods, there is a 4% overall performance improvement. On the detection and classification of acoustic scenes and events 2022 (DCASE2022) synthetic dataset, the overall performance improvement is 5%. These results indicate a significant advantage and provide a novel and robust solution for the SELD task. Propose a hybrid offline-online method for SELD. The extracted hybrid features or temporal sequences enable the acquisition of a more comprehensive range of audio information. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF