Robust and automatic beamstop shadow outlier rejection: combining crystallographic statistics with modern clustering under a semi‐supervised learning strategy.

During the automatic processing of crystallographic diffraction experiments, beamstop shadows are often unaccounted for or only partially masked. As a result of this, outlier reflection intensities are integrated, which is a known issue. Traditional statistical diagnostics have only limited effectiveness in identifying these outliers, here termed Not‐Excluded‐unMasked‐Outliers (NEMOs). The diagnostic tool AUSPEX allows visual inspection of NEMOs, where they form a typical pattern: clusters at the low‐resolution end of the AUSPEX plots of intensities or amplitudes versus resolution. To automate NEMO detection, a new algorithm was developed by combining data statistics with a density‐based clustering method. This approach demonstrates a promising performance in detecting NEMOs in merged data sets without disrupting existing data‐reduction pipelines. Re‐refinement results indicate that excluding the identified NEMOs can effectively enhance the quality of subsequent structure‐determination steps. This method offers a prospective automated means to assess the efficacy of a beamstop mask, as well as highlighting the potential of modern pattern‐recognition techniques for automating outlier exclusion during data processing, facilitating future adaptation to evolving experimental strategies. [ABSTRACT FROM AUTHOR]