Unsupervised optical flow estimation method based on transformer and occlusion compensation.

One of the problems that still adversely affect the optical flow accuracy is the missing parts or the loss of some details in the flow map that is estimated by the flow estimation network. The main reasons are the feature maps that are extracted by the local kernels of the convolution neural networks (CNN), which cause feature loss and provide low-quality features, and the occluded parts after the image reconstruction. This loss in the encoder part becomes hard to recover in the flow estimation network. To overcome this problem, we propose a CNN with transformer architecture to emphasize the important features and model the long-range dependencies to produce a better feature representation that can provide the flow map with strong details. In addition, we propose an occlusion compensation loss to rectify the occlusion map and assist the network in learning how to predict the flow in the occluded regions. Extensive experiments are conducted on Sintel and KITTI benchmarks, and the results demonstrate the efficiency of our model in increasing the accuracy of the optical flow and improving the flow in the occluded regions. [ABSTRACT FROM AUTHOR]