Progressive Depth Decoupling and Modulating for Flexible Depth Completion

Image-guided depth completion aims at generating a dense depth map from sparse light detection and ranging (LiDAR) data and the corresponding RGB image, which is crucial for applications that require 3-D scene perception, such as augmented reality and human-computer interaction. Recent methods have shown promising performance by reformulating it as a classification problem with two subtasks: depth discretization and probability prediction. They divide the depth range into several discrete depth values as depth categories, serving as priors for scene depth distributions. However, previous depth discretization methods are easy to be impacted by depth distribution variations across different scenes, resulting in suboptimal scene depth distribution priors. To address the above problem, we propose a progressive depth decoupling and modulating network, which incrementally decouples the depth range into bins and adaptively generates multiscale dense depth maps in multiple stages. Specifically, we first design a bins initializing module (BIM) to construct the seed bins by exploring the depth distribution information within a sparse depth map, adapting variations of depth distribution. Then, we devise an incremental depth decoupling branch to progressively refine the depth distribution information from global to local. Meanwhile, an adaptive depth modulating branch is developed to progressively improve the probability representation from coarse-grained to fine-grained. Also, the bidirectional information interactions are proposed to strengthen the information interaction between those two branches (subtasks) for promoting information complementation in each branch. Furthermore, we introduce a multiscale supervision mechanism to learn the depth distribution information in latent features and enhance the adaptation capability across different scenes. Experimental results on public datasets demonstrate that our method outperforms the state-of-the-art (SOTA) methods. We will release the source codes and pretrained models.