Fast Multi-Scale Residual Fusion Network for Stereo Matching

Recent deep convolution-based stereo matching methods have shown significant progress. However, most state-of-the-art models achieve high accuracy by using 3D convolutions during cost aggregation, which come with more floating-point computations and makes it difficult for deployment in real-time applications. In this paper, we propose an effective and efficient multi-scale aggregation module (without 3D convolution) to build our fast Multi-Scale Residual Fusion Network (MSRFNet). Three lightweight components are involved in our aggregation module: combination block, attention block, and residual fusion block. The combination block combines features with different receptive fields and the attention block emphasizes salient regions in various cost volumes. The residual fusion module focuses on extracting the differences between adjacent cost volumes, using progressively residual aggregation instead of simply stacking or adding. Extensive experiments on Scene Flow and KITTI benchmarks demonstrate that our method achieves competitive accuracy com-pared with state-of-the-art methods while running at 56ms.