Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data.

Cloud masking is a key initial step in the retrieval of geophysical properties from satellite data. Despite decades of research problems still exists of over or under detection of cloud. High aerosol loadings, in particular from dust storms or fires, are often classified as cloud and vice versa. In this paper we present a cloud mask created using machine learning for the Advanced Himawari Imager (AHI) on board Himawari-8. In order to train the algorithm a parallax-corrected collocated data set was created from AHI and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar data. Artificial neural networks (ANNs) were trained on the collocated data to identify clouds in AHI scenes. The resulting NN cloud masks are validated and compared to cloud masks produced by the Japanese Meteorological Association (JMA) and the Bureau of Meteorology (BoM) for a number of different solar and viewing geometries, surface types and airmasses. 5 case studies covering a range of challenging scenarios for cloud masks are also presented to demonstrate the performance of the masking algorithm. The NN mask shows a lower false positive rate (FPR) for an equivalent true positive rate (TPR) across all categories, with FPRs of 0.106 and 0.198 for the NN and JMA masks respectively and 0.314 and 0.464 for the NN and BoM masks respectively at equivalent TPR values. This indicates the NN mask accurately identifies 1.11 and 1.28 times as many non-cloud pixels, for the equivalent hit rate when compared to the JMA and BoM masks respectively. The NN mask was shown to be particularly effective in distinguishing thick aerosol plumes from cloud, most likely due to the inclusion of the 0.47 μm and 0.51 μm bands. The NN cloud mask shows an improvement over current operational cloud masks in most scenario and it is suggested that improvements to current operational cloud masks could be made by including the 0.47 μm and 0.51 μm bands. The collocated data are made available to facilitate future research. [ABSTRACT FROM AUTHOR]