Toward intraoperative endomicroscopy with a GPU-accelerated deformable video mosaicking algorithm

Due to the limited field of view (FOV), the probe-based confocal laser endomicroscopy (pCLE) imaging system remains challenging to be widely used in clinic. Existing video mosaicking approaches are usually troubled by poor real-time capability and sensitivity to tissue deformations and intensity fluctuations. In this paper, a novel pCLE mosaicking algorithm that simultaneously implements rigid probe motion tracking and inter-frame tissue deformation correction is proposed. The sum of conditional variance (SCV) metric with low calculational complexity and intensity variation invariance is embedded into the reconstruction of pCLE mosaics. We also deduce the derivatives of the SCV metric with respect to transformation variables to enhance the robustness of numerical solutions. Moreover, a parallel acceleration mechanism that maximally exploits the resources of graphics processing unit (GPU) is designed to boost mosaicking efficiency. Experiments on lens tissue paper and swine tissue highlight the excellent mosaicking accuracy of the proposed method in this field. It is also verified that the mosaic-king rate of the proposed method exceeds 14 frames per second (fps), which can keep up with the acquisition rate of most pCLE imaging systems (8 fps~12 fps). Benefiting from finer mosaics and faster speed, the proposed algorithm has high potential to achieve large and accurate mosaicking for intraoperative endomicroscopy.