Rainfall weather always degrades the quality of the images severely in the outdoor surveillance system. To improve the quality of images, different rain removal algorithms have been proposed recently. As the decomposition based methods do not need to impose any restrictions on the types of rain, they have a wider application prospect. However, they still have the problems of rain residue in the low-frequency components and information loss in high-frequency components. To solve these problems, we propose an image rain streak removal algorithm based on the depth of field (DoF) and sparse coding. Firstly, we decompose the image by using the combination of bilateral filtering and short-time Fourier transform, so that the contour in the low-frequency part of the image can be better preserved. Then the DoF saliency map of the image is used both to reduce the rain residue in the low-frequency components and to avoid mis-matching the background and the rain streaks with the same gradient in the high-frequency components. We use DoF saliency map as the weights for the weighted sum of the original image and the initial low-frequency image to obtain the corrected low-frequency images. The DoF saliency map is also used to twice weaken the rain streaks in the high-frequency image to generate the corrected high-frequency images. The algorithm includes four steps: image decomposition, dictionary learning, atomic clustering based on Principal Component Analysis and Support Vector Machine, image revising based on DoF saliency map. The experimental results demonstrate that our proposed algorithm performs better both in rain removal and high-frequency information preserving than current methods.