Anomaly Detection via Context and Local Feature Matching

Unsupervised anomaly detection in medical imaging is an exciting prospect due to the option of training only on healthy data, without the need for expensive segmentation annotations of many possible variations of outliers. Most current methods rely on image reconstruction error to produce anomaly scores, which favors detection of intensity outliers. We instead propose a discriminative method based on a deep learning self-supervised pixel-level classification task. We model context and local image feature information separately and set up a pixel-level classification task to discriminate between positive (matching) and negative (mismatching) context and local feature pairs. Negative matches are created using data transformations and context/local shuffling. At test-time, the model then perceives local regions containing anomalies to be negative matches. We evaluate our method on a surrogate task of tumor segmentation in brain MRI data and show significant performance improvements over baselines.