Automatic signal classification in fluorescence in situ hybridization images

Background: Previous systems for dot (signal) counting in fluorescence in situ hybridization (FISH) images have relied on an auto-focusing method for obtaining a clearly defined image. Because signals are distributed in three dimensions within the nucleus and artifacts such as debris and background fluorescence can attract the focusing method, valid signals can be left unfocused or unseen. This leads to dot counting errors, which increase with the number of probes. Methods: The approach described here dispenses with auto-focusing, and instead relies on a neural network (NN) classifier that discriminates between in and out-of-focus images taken at different focal planes of the same field of view. Discrimination is performed by the NN, which classifies signals of each image as valid data or artifacts (due to out of focusing). The image that contains no artifacts is the in-focus image selected for dot count proportion estimation. Results: Using an NN classifier and a set of features to represent signals improves upon previous discrimination schemes that are based on nonadaptable decision boundaries and single-feature signal representation. Moreover, the classifier is not limited by the number of probes. Three classification strategies, two of them hierarchical, have been examined and found to achieve each between 83% and 87% accuracy on unseen data. Screening, while performing dot counting, of in and out-of-focus images based on signal classification suggests an accurate and efficient alternative to that obtained using an auto-focusing mechanism. Cytometry 43:87‐93, 2001. © 2001 Wiley-Liss, Inc.