1. Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception
- Author
-
Zhang, Xiang, Cui, Yufei, Fu, Chenchen, Wu, Weiwei, Wang, Zihao, Sun, Yuyang, and Liu, Xue
- Subjects
Computer Science - Computer Vision and Pattern Recognition - Abstract
Real-time object detection is critical for the decision-making process for many real-world applications, such as collision avoidance and path planning in autonomous driving. This work presents an innovative real-time streaming perception method, Transtreaming, which addresses the challenge of real-time object detection with dynamic computational delay. The core innovation of Transtreaming lies in its adaptive delay-aware transformer, which can concurrently predict multiple future frames and select the output that best matches the real-world present time, compensating for any system-induced computation delays. The proposed model outperforms the existing state-of-the-art methods, even in single-frame detection scenarios, by leveraging a transformer-based methodology. It demonstrates robust performance across a range of devices, from powerful V100 to modest 2080Ti, achieving the highest level of perceptual accuracy on all platforms. Unlike most state-of-the-art methods that struggle to complete computation within a single frame on less powerful devices, Transtreaming meets the stringent real-time processing requirements on all kinds of devices. The experimental results emphasize the system's adaptability and its potential to significantly improve the safety and reliability for many real-world systems, such as autonomous driving., Comment: Submitted to AAAI 2025
- Published
- 2024