Search

This paper explores the problem of class-generalizable anomaly detection, where the objective is to train one unified AD model that can generalize to detect anomalies in diverse classes from different domains without any retraining or fine-tuning on the target data. Because normal feature representations vary significantly across classes, this will cause the widely studied one-for-one AD models to be poorly classgeneralizable (i.e., performance drops dramatically when used for new classes). In this work, we propose a simple but effective framework (called ResAD) that can be directly applied to detect anomalies in new classes. Our main insight is to learn the residual feature distribution rather than the initial feature distribution. In this way, we can significantly reduce feature variations. Even in new classes, the distribution of normal residual features would not remarkably shift from the learned distribution. Therefore, the learned model can be directly adapted to new classes. ResAD consists of three components: (1) a Feature Converter that converts initial features into residual features; (2) a simple and shallow Feature Constraintor that constrains normal residual features into a spatial hypersphere for further reducing feature variations and maintaining consistency in feature scales among different classes; (3) a Feature Distribution Estimator that estimates the normal residual feature distribution, anomalies can be recognized as out-of-distribution. Despite the simplicity, ResAD can achieve remarkable anomaly detection results when directly used in new classes. The code is available at https://github.com/xcyao00/ResAD., Comment: This paper was accepted as a spotlight papaer by NeurIPS 2024

Unified anomaly detection (AD) is one of the most challenges for anomaly detection, where one unified model is trained with normal samples from multiple classes with the objective to detect anomalies in these classes. For such a challenging task, popular normalizing flow (NF) based AD methods may fall into a "homogeneous mapping" issue,where the NF-based AD models are biased to generate similar latent representations for both normal and abnormal features, and thereby lead to a high missing rate of anomalies. In this paper, we propose a novel Hierarchical Gaussian mixture normalizing flow modeling method for accomplishing unified Anomaly Detection, which we call HGAD. Our HGAD consists of two key components: inter-class Gaussian mixture modeling and intra-class mixed class centers learning. Compared to the previous NF-based AD methods, the hierarchical Gaussian mixture modeling approach can bring stronger representation capability to the latent space of normalizing flows, so that even complex multi-class distribution can be well represented and learned in the latent space. In this way, we can avoid mapping different class distributions into the same single Gaussian prior, thus effectively avoiding or mitigating the "homogeneous mapping" issue. We further indicate that the more distinguishable different class centers, the more conducive to avoiding the bias issue. Thus, we further propose a mutual information maximization loss for better structuring the latent feature space. We evaluate our method on four real-world AD benchmarks, where we can significantly improve the previous NF-based AD methods and also outperform the SOTA unified AD methods., Comment: This paper is accepted by ECCV2024

Humans recognize anomalies through two aspects: larger patch-wise representation discrepancies and weaker patch-to-normal-patch correlations. However, the previous AD methods didn't sufficiently combine the two complementary aspects to design AD models. To this end, we find that Transformer can ideally satisfy the two aspects as its great power in the unified modeling of patch-wise representations and patch-to-patch correlations. In this paper, we propose a novel AD framework: FOcus-the-Discrepancy (FOD), which can simultaneously spot the patch-wise, intra- and inter-discrepancies of anomalies. The major characteristic of our method is that we renovate the self-attention maps in transformers to Intra-Inter-Correlation (I2Correlation). The I2Correlation contains a two-branch structure to first explicitly establish intra- and inter-image correlations, and then fuses the features of two-branch to spotlight the abnormal patterns. To learn the intra- and inter-correlations adaptively, we propose the RBF-kernel-based target-correlations as learning targets for self-supervised learning. Besides, we introduce an entropy constraint strategy to solve the mode collapse issue in optimization and further amplify the normal-abnormal distinguishability. Extensive experiments on three unsupervised real-world AD benchmarks show the superior performance of our approach. Code will be available at https://github.com/xcyao00/FOD., Comment: Accepted by ICCV2023

Purpose: To evaluate the sensitivity of normalized blood flow index (NBFI) for detecting early diabetic retinopathy (DR). Methods: Optical coherence tomography angiography (OCTA) images of 30 eyes from 20 healthy controls, 21 eyes of diabetic patients with no DR (NoDR) and 26 eyes from 22 patients with mild non-proliferative DR (NPDR) were analyzed in this study. The OCTA images were centered on the fovea and covered a 6 mm x 6 mm area. Enface projections of the superficial vascular plexus (SVP) and the deep capillary plexus (DCP) were obtained for the quantitative OCTA feature analysis. Three quantitative OCTA features were examined: blood vessel density (BVD), blood flow flux (BFF), and normalized blood flow index (NBFI). Each feature was calculated from both the SVP and DCP and their sensitivity to distinguish the three cohorts of the study were evaluated. Results: The only quantitative feature that was capable of distinguishing between all three cohorts was NBFI in the DCP image. Comparative study revealed that both BVD and BFF were able to distinguish the controls from NoDR and mild NPDR. However, neither BVD nor BFF was sensitive enough to separate NoDR from the healthy controls. Conclusion: The NBFI has been demonstrated as a sensitive biomarker of early DR, revealing retinal blood flow abnormality better than traditional BVD and BFF. The NBFI in the DCP was verified as the most sensitive biomarker, supporting that diabetes affects the DCP earlier than SVP in DR.

Fundus photography is indispensable for clinical detection and management of eye diseases. Limited image contrast and field of view (FOV) are common limitations of conventional fundus cameras, making it difficult to detect subtle abnormalities at the early stages of eye diseases. Further improvements of image contrast and FOV coverage are important to improve early disease detection and reliable treatment assessment. We report here a portable fundus camera, with a wide FOV and high dynamic range (HDR) imaging capabilities. Miniaturized indirect ophthalmoscopy illumination was employed to achieve the portable design for nonmydriatic, widefield fundus photography. Orthogonal polarization control was used to eliminate illumination reflectance artifact. With independent power controls, three fundus images were sequentially acquired and fused to achieve HDR function for local image contrast enhancement. A 101{\deg} eye-angle (67{\deg} visual-angle) snapshot FOV was achieved for nonmydriatic fundus photography. The effective FOV can be readily expanded up to 190{\deg} eye-angle (134{\deg} visual-angle) with the aid of a fixation target, without the need of pharmacologic pupillary dilation. The effectiveness of HDR imaging was validated with both normal healthy and pathologic eyes, compared to a conventional fundus camera., Comment: 12 pages, 8 figures

Significance: Multiple eye diseases such as age-related macular degeneration, diabetic retinopathy, and retinitis pigmentosa can cause photoreceptor dysfunction. Rod photoreceptors are known to be more vulnerable than cone photoreceptors. Therefore, functional assessment of rod photoreceptors is important for early detection of eye diseases. Aim: This study is to demonstrate the feasibility of using intrinsic optical signal (IOS) optoretinography (ORG) for objective detection of dark adaptation (DA) abnormality due to rod photoreceptor degeneration. Approach: Functional optical coherence tomography (OCT) was employed for IOS ORG of wild-type (WT) and retinal degeneration 10 (rd10) mice. Six WT C57BL/6J and eight rd10 B6.CXB1-Pde6brd10/J mice at postnatal day 14 (P14) were used for this study. Dynamic OCT analysis of retinal thickness and brightness changes, corresponding to light to dark transition, was implemented Results: Comparative measurement of the retina under light and dark conditions revealed significant IOS changes within the outer retina. The thickness between external limiting membrane (ELM) and retinal pigment epithelium (RPE) reduced, and the OCT brightness of inner segment ellipsoid zone (EZ) decreased during DA, compared to light adaptation (LA). Relative EZ intensity change was observed to decrease larger in rd10, compared to WT. The relative intensity of the hyporeflective band between ELM and RPE also showed significant decrease in rd10 retinas during DA. Conclusions: DA-IOS abnormalities were observed in rd10, compared to WT at P14. The ORG measurement of DA-IOS kinetics promises valuable information for noninvasive assessment of photoreceptor function.

Most anomaly detection (AD) models are learned using only normal samples in an unsupervised way, which may result in ambiguous decision boundary and insufficient discriminability. In fact, a few anomaly samples are often available in real-world applications, the valuable knowledge of known anomalies should also be effectively exploited. However, utilizing a few known anomalies during training may cause another issue that the model may be biased by those known anomalies and fail to generalize to unseen anomalies. In this paper, we tackle supervised anomaly detection, i.e., we learn AD models using a few available anomalies with the objective to detect both the seen and unseen anomalies. We propose a novel explicit boundary guided semi-push-pull contrastive learning mechanism, which can enhance model's discriminability while mitigating the bias issue. Our approach is based on two core designs: First, we find an explicit and compact separating boundary as the guidance for further feature learning. As the boundary only relies on the normal feature distribution, the bias problem caused by a few known anomalies can be alleviated. Second, a boundary guided semi-push-pull loss is developed to only pull the normal features together while pushing the abnormal features apart from the separating boundary beyond a certain margin region. In this way, our model can form a more explicit and discriminative decision boundary to distinguish known and also unseen anomalies from normal samples more effectively. Code will be available at https://github.com/xcyao00/BGAD., Comment: Accepted to CVPR 2023

Intrinsic optical signal (IOS) imaging of the retina, also termed as optoretinography (ORG), promises a noninvasive method for objective assessment of retinal function. By providing unparalleled capability to differentiate individual layers of the retina, functional optical coherence tomography (OCT) has been actively investigated for intrinsic signal ORG measurements. However, clinical deployment of functional OCT for quantitative ORG is still challenging due to the lack of a standardized imaging protocol and the complication of IOS sources and mechanisms. This article aims to summarize recent developments of functional OCT for ORG measurement, OCT intensity- and phase-based IOS processing. Technical challenges and perspectives of quantitative IOS analysis and ORG interpretations are discussed.

Chromatic dispersion is a common problem to degrade the system resolution in optical coherence tomography (OCT). This study is to develop a deep learning network for automated dispersion compensation (ADC-Net) in OCT. The ADC-Net is based on a redesigned UNet architecture which employs an encoder-decoder pipeline. The input section encompasses partially compensated OCT B-scans with individual retinal layers optimized. Corresponding output is a fully compensated OCT B-scans with all retinal layers optimized. Two numeric parameters, i.e., peak signal to noise ratio (PSNR) and structural similarity index metric computed at multiple scales (MS-SSIM), were used for objective assessment of the ADC-Net performance. Comparative analysis of training models, including single, three, five, seven and nine input channels were implemented. The five-input channels implementation was observed as the optimal mode for ADC-Net training to achieve robust dispersion compensation in OCT, Comment: 18 pages, 5 figures

Delayed dark adaptation due to impaired rod photoreceptor homeostasis has been reported as the earliest symptom of eye diseases such as age-related macular degeneration, diabetic retinopathy, and retinitis pigmentosa. Objective measurement of dark adaptation can facilitate early diagnosis to enable prompt intervention to prevent vision losses. However, there is a lack of noninvasive methods capable of spatiotemporal monitoring of photoreceptor changes during dark adaptation. Here we demonstrate functional optical coherence tomography (OCT) for in vivo intrinsic signal optoretinography (ORG) of dark adaptation kinetics in the C57BL/6J mouse retina. Functional OCT revealed a shortening of the outer retina, a morphological change in the cone and rod photoreceptor interdigitation zone, and a reduction in intrinsic signal amplitude at the photoreceptor inner segment ellipsoid. A strong positive correlation between morphophysiological activities was also confirmed. Functional OCT of dark adaptation kinetics promises a method for rapid ORG assessment of physiological integrity of retinal photoreceptors., Comment: 18 pages, 7 figures

Purpose: This study is to test the feasibility of optical coherence tomography (OCT) detection of photoreceptor abnormality and to verify the photoreceptor abnormality is rod predominated in early DR . Methods: OCT images were acquired from normal eyes, diabetic eyes with no diabetic retinopathy (NoDR) and mild DR. Quantitative features, including length features quantifying band distances and reflectance intensity features among the external limiting membrane (ELM), inner segment ellipsoid (ISe), interdigitation zone (IZ) and retinal pigment epithelium (RPE) were determined. Comparative OCT analysis of central fovea, parafovea and perifovea were implemented to verify the photoreceptor abnormality is rod predominated in early DR. Results: Length abnormalities between the ISe and IZ also showed a decreasing trend among cohorts. Reflectance abnormalities of the ELM, IZ and ISe were observed between healthy, NoDR, and Mild DR eyes. The normalized ISe/RPE intensity ratio revealed a significant decreasing trend in the perifovea, but no detectable difference in central fovea. Conclusions: Quantitative OCT analysis consistently revealed outer retina, i.e., photoreceptor, changes in diabetic patients with NoDR and mild DR. Comparative analysis of central fovea, parafovea and perifovea confirmed the photoreceptor abnormality is rod predominated in early DR.

This study is to characterize reflectance profiles of retinal blood vessels in optical coherence tomography (OCT), and to validate these vascular features to guide artery-vein classification in OCT angiography (OCTA) of human retina. Depth-resolved OCT reveals unique features of retinal arteries and veins. Retinal arteries show hyper-reflective boundaries at both upper (inner side towards the vitreous) and lower (outer side towards the choroid) walls. In contrary, retinal veins reveal hyper-reflectivity at the upper boundary only. Uniform lumen intensity was observed in both small and large arteries. However, the vein lumen intensity was dependent on the vessel size. Small veins exhibit a hyper-reflective zone at the bottom half of the lumen, while large veins show a hypo-reflective zone at the bottom half of the lumen, Comment: 11 pages, 4 figures

Abstract: Visual-angle has been used as the conventional unit to determine the field-of-view (FOV) in traditional fundus photography. Recently emerging usage of eye-angle as the unit in wide field fundus photography creates confusions about FOV interpretation in instrumentation design and clinical application. This study is to systematically derive the relationship between the visual-angle {\theta}v and eye-angle {\theta}e, and thus to enable reliable determination of the FOV in wide field fundus photography. FOV conversion ratio {\theta}e/{\theta}v, angular conversion ratio {\Delta}{\theta}e/{\Delta}{\theta}v, retinal conversion ratio {\Delta}d/{\Delta}{\theta}v, retinal distance and area are quantitatively evaluated. Systematical analysis indicates that reliable conversion between the {\theta}v and {\theta}e requires determined nodal point and spherical radius of the eye; and the conversion ratio is not linear from the central field to peripheral region. Based on the eye model with average parameters, both angular conversion ({\Delta}{\theta}e/{\Delta}{\theta}v) and retinal conversion ({\Delta}d/{\Delta}{\theta}v) ratios are observed to have a 1.51-fold difference at the central field and far peripheral region. A conversion table, including {\theta}e/{\theta}v, {\Delta}{\theta}e/{\Delta}{\theta}v, {\Delta}d/{\Delta}{\theta}v, retinal area and percentage ratio, is created for reliable assessment of imaging systems with variable FOV.

This study is to demonstrate deep learning for automated artery-vein (AV) classification in optical coherence tomography angiography (OCTA). The AV-Net, a fully convolutional network (FCN) based on modified U-shaped CNN architecture, incorporates enface OCT and OCTA to differentiate arteries and veins. For the multi-modal training process, the enface OCT works as a near infrared fundus image to provide vessel intensity profiles, and the OCTA contains blood flow strength and vessel geometry features. A transfer learning process is also integrated to compensate for the limitation of available dataset size of OCTA, which is a relatively new imaging modality. By providing an average accuracy of 86.75%, the AV-Net promises a fully automated platform to foster clinical deployment of differential AV analysis in OCTA.

This study is to test the feasibility of using trans-pars-planar illumination for ultrawide field pediatric fundus photography. Fundus examination of the peripheral retina is essential for clinical management of pediatric eye diseases. However, current pediatric fundus cameras with traditional trans-pupillary illumination provide a limited field of view (FOV), making it difficult to access the peripheral retina adequately for a comprehensive assessment of eye conditions. Here, we report the first demonstration of trans-pars-planar illumination in ultra-wide field pediatric fundus photography. For proof-of-concept validation, all off-the-shelf optical components were selected to construct a lab prototype pediatric camera (PedCam). By freeing the entire pupil for imaging purpose only, the trans-pars-planar illumination enables a 200o FOV in a snapshot fundus image, allowing easy visualization of both the central and peripheral retina up to the ora serrata. A low-cost, easy-to-use ultra-wide field PedCam provides a unique opportunity to foster affordable telemedicine in rural and underserved areas., Comment: 9 pages, and 3 figures

Purpose: To test the feasibility of using deep learning for optical coherence tomography angiography (OCTA) detection of diabetic retinopathy (DR). Methods: A deep learning convolutional neural network (CNN) architecture VGG16 was employed for this study. A transfer learning process was implemented to re-train the CNN for robust OCTA classification. In order to demonstrate the feasibility of using this method for artificial intelligence (AI) screening of DR in clinical environments, the re-trained CNN was incorporated into a custom developed GUI platform which can be readily operated by ophthalmic personnel. Results: With last nine layers re-trained, CNN architecture achieved the best performance for automated OCTA classification. The overall accuracy of the re-trained classifier for differentiating healthy, NoDR, and NPDR was 87.27%, with 83.76% sensitivity and 90.82% specificity. The AUC metrics for binary classification of healthy, NoDR and DR were 0.97, 0.98 and 0.97, respectively. The GUI platform enabled easy validation of the method for AI screening of DR in a clinical environment. Conclusion: With a transfer leaning process to adopt the early layers for simple feature analysis and to re-train the upper layers for fine feature analysis, the CNN architecture VGG16 can be used for robust OCTA classification of healthy, NoDR, and NPDR eyes. Translational Relevance: OCTA can capture microvascular changes in early DR. A transfer learning process enables robust implementation of convolutional neural network (CNN) for automated OCTA classification of DR., Comment: 20 pages, 4 figures, 6 tables

Artificial intelligence (AI) classification holds promise as a novel and affordable screening tool for clinical management of ocular diseases. Rural and underserved areas, which suffer from lack of access to experienced ophthalmologists may particularly benefit from this technology. Quantitative optical coherence tomography angiography (OCTA) imaging provides excellent capability to identify subtle vascular distortions, which are useful for classifying retinovascular diseases. However, application of AI for differentiation and classification of multiple eye diseases is not yet established. In this study, we demonstrate supervised machine learning based multi-task OCTA classification. We sought 1) to differentiate normal from diseased ocular conditions, 2) to differentiate different ocular disease conditions from each other, and 3) to stage the severity of each ocular condition. Quantitative OCTA features, including blood vessel tortuosity (BVT), blood vascular caliber (BVC), vessel perimeter index (VPI), blood vessel density (BVD), foveal avascular zone (FAZ) area (FAZ-A), and FAZ contour irregularity (FAZ-CI) were fully automatically extracted from the OCTA images. A stepwise backward elimination approach was employed to identify sensitive OCTA features and optimal-feature-combinations for the multi-task classification. For proof-of-concept demonstration, diabetic retinopathy (DR) and sickle cell retinopathy (SCR) were used to validate the supervised machine leaning classifier. The presented AI classification methodology is applicable and can be readily extended to other ocular diseases, holding promise to enable a mass-screening platform for clinical deployment and telemedicine., Comment: Supplemental material attached at the end