Search

Previous studies have demonstrated that patients can be identified from 3-dimensional (3D) reconstructions of computed tomography (CT) or magnetic resonance imaging data of the brain or head and neck. This presents a privacy and security concern for scan data released to public data sets. It is unknown whether thermoplastic immobilization masks used for treatment planning in radiation therapy are sufficient to prevent facial recognition. Our study sought to evaluate whether patients with an immobilization mask could be identified on 3D reconstructions of scan data. Our study reconstructed 3D images from simulation CT (SIM-CT) scans of 35 patients and compared these to original patient photographs to test if the thermoplastic mask obfuscated facial features. Blind review from 4 facial recognition algorithms and a human (radiation oncologist) was evaluated for the ability to match 3D reconstructions of patients scans to patient images. The matching procedure was repeated against an expanded testing data set of the 35 patient photographs plus 13,233 facial photographs from the "Labeled Faces in the Wild" data set (13,268 photographs in total). Facial recognition algorithms were able to match a maximum of 83% (range, 60%-83%) of patients to the corresponding images. Radiation Oncologist blinded review correctly matched 80% of patients to the corresponding images. Ethnicity and facial hair were the most common reasons for patient mismatch. In the expanded testing data set, algorithms were also able to match a maximum of 83% (range, 57%-83%) of patients. The majority of patients were able to be identified through computer algorithm or human review even under a SIM-CT mask. These results suggest there is a potential privacy and security concern when SIM-CT data are released to publicly available data sets.

Image processing has contributed greatly to the clinical applications of medical imaging. Many of the major developments have been stimulated by and reported at the Image Processing (IP) conference held annually as part of the SPIE Medical Imaging meeting. The evolution, focus, and impact of the IP conference is reviewed.

A loss function has two crucial roles in training a conventional discriminant deep neural network (DNN): (i) it measures the goodness of classification and (ii) generates the gradients that drive the training of the network. In this paper, we approximate the gradients of cross-entropy loss which is the most often used loss function in the classification DNNs. The proposed approximations are noise-free, which means they depend only on the labels of the training set. They have a fixed length to avoid the vanishing gradient problem of the cross-entropy loss. By skipping the forward pass, the computational complexities of the proposed approximations are reduced to O(n) where n is the batch size. Two claims are established based on the experiments of training DNNs using the proposed approximations: (i) It is possible to train a discriminant network without explicitly defining a loss function. (ii) The success of training does not imply the convergence of network parameters to fixed values. The experiments show that the proposed gradient approximations achieve comparable classification accuracy to the conventional loss functions and can accelerate the training process on multiple datasets.

To predict the histologic grade of small clear cell renal cell carcinomas (ccRCCs) using texture analysis and machine learning algorithms. Fifty-two noncontrast (NC), 26 corticomedullary (CM) phase, and 35 nephrographic (NG) phase CTs of small (

Real-time semantic segmentation is playing a more important role in computer vision, due to the growing demand for mobile devices and autonomous driving. Therefore, it is very important to achieve a good trade-off among performance, model size and inference speed. In this paper, we propose a Channel-wise Feature Pyramid (CFP) module to balance those factors. Based on the CFP module, we built CFPNet for real-time semantic segmentation which applied a series of dilated convolution channels to extract effective features. Experiments on Cityscapes and CamVid datasets show that the proposed CFPNet achieves an effective combination of those factors. For the Cityscapes test dataset, CFPNet achieves 70.1% class-wise mIoU with only 0.55 million parameters and 2.5 MB memory. The inference speed can reach 30 FPS on a single RTX 2080Ti GPU with a 1024x2048-pixel image.

Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks (CNN) have been designed for segmentation tasks and achieved great success. Few studies, however, have fully considered the sizes of objects, and thus most demonstrate poor performance for small objects segmentation. This can have a significant impact on the early detection of diseases. This paper proposes a Context Axial Reserve Attention Network (CaraNet) to improve the segmentation performance on small objects compared with several recent state-of-the-art models. We test our CaraNet on brain tumor (BraTS 2018) and polyp (Kvasir-SEG, CVC-ColonDB, CVC-ClinicDB, CVC-300, and ETIS-LaribPolypDB) segmentation datasets. Our CaraNet achieves the top-rank mean Dice segmentation accuracy, and results show a distinct advantage of CaraNet in the segmentation of small medical objects., Accepted by SPIE Medical Imaging: Image Processing (oral presentation)

Although deep neural networks (DNNs) achieve state-of-the-art performances in a wide range of machine learning (ML) applications, they are vulnerable: when small intentional perturbations are added to inputs, the network would misclassify them with high confidence. This phenomenon attracts broad attention because it is a security issue. In this paper, we study the geometric properties of the decision boundaries in the representation space of DNNs and propose novel adversarial approaches by moving the representations of the inputs toward the decision boundaries and thus change the predictions of the DNN. Our experimental results show that the proposed algorithms are on par or better than state-of-the-art adversarial approaches in terms of the magnitude of perturbation and computation time.

Background: Brain stimulation is utilized to treat a variety of neurological disorders. Clinical brain stimulation technologies currently utilize charge-balanced pulse stimulation. The brain may better respond to other stimulation waveforms. This study was designed to evaluate the motor threshold of the brain to stimulation with various waveforms. Methods: Three stimulation waveforms were utilized on rats with surgically implanted brain electrodes: pulses, square waves, and random waveform. The peak-to-peak stimulation voltage was increased in a step-wise manner until motor signs were elicited. Results: The random waveform had the highest motor threshold with brain stimulation compared to the other waveforms. Random waveform stimulation reached maximum voltage without motor side effects while stimulating through both 1 and 8 electrodes. In contrast, the stimulation thresholds for motor side effects of the other two waveforms were on average less than half of the maximum voltage and lower for stimulation through 8 electrodes than stimulation through 1 electrode (p < 0.0005). Conclusion: The random waveform was better tolerated than the other waveforms and may allow for the use of higher stimulation voltage without side effects.

PURPOSE/OBJECTIVE(S) This study assessed trends in guideline-adherent chemoradiation therapy (GA-RT) for advanced-stage cervical cancer relative to Patient Protection and Affordable Care Act (ACA) implementation in the year 2010. MATERIALS/METHODS National Cancer Database patients treated with definitive chemoradiation for locally advanced cervical cancer (FIGO 2018 Stage IB3-IVA) from 2004-2016 were included. GA-RT was defined according to NCCN guidelines and included: 1) delivery of external beam radiation, 2) brachytherapy, and 3) chemotherapy, 4) no radical hysterectomy. Univariable and multivariable logistic regression were used to determine trends in GA-RT relative to the ACA and identify patient factors independently associated with GA-RT. Survival of GA-RT was also estimated using Kaplan-Meier analysis. RESULTS Thirty-seven thousand seven hundred and seventy-two patients met inclusion criteria (Pre-ACA: 16,169; Post-ACA: 21,673). A total of 33,116 patients had squamous cell carcinoma, and 4,626 patients had other histologies. Forty-five percent of patients were node positive. A total of 14.6% of patients were Stage I, 41.8% were Stage II, 36.4% were Stage III, and 7.9% were Stage IVA. Compared to commercial insurance, patients with Medicare had decreased odds of receiving GA-RT (OR = 0.65; 95% CI:0.61-0.69). Black patients (OR = 0.9; 95% CI:0.85-0.95) were associated with decreased odds of receiving GA-RT. Compared to squamous cell carcinoma, adenocarcinoma and carcinoma NOS were associated with decreased receipt of GA-RT (OR = 0.87; 95% CI: 0.80-0.93 and OR = 0.64; 95% CI: 0.55-0.75, respectively). Increasing T-stage was associated with greater receipt of GA-RT (stages 1B2 [OR = 1.89, 95% CI: 1.53-2.37], 2A2 [OR = 2.29, 95% CI: 1.81-2.92], 2B [OR = 2.1, 95% CI: 1.71-2.61], and 3B [OR = 1.47, 95% CI: 1.19-1.83]). Patients with nodal stage NX were less likely to receive GA-RT compared to N0 (OR = 0.6, 95% CI: 0.52-0.68). Increasing Charlson-Deyo score was associated with decreased receipt of GA-RT (OR = 0.86, 95% CI: 0.82-0.89). In multivariate analysis, all factors remained statistically significant except for race. The percentage of population that was guideline adherent significantly increased post-ACA (pre-ACA 28%; post-ACA 34%; P < 0.001). On univariate analysis, patients were 34% more likely to receive GA-RT after the ACA compared to before the ACA. In the multivariate model, receipt of GA-RT increased by 30% (P < 0.001). Adherence to treatment guidelines increased 2-year overall survival by 15% (GA 76%; Not GA 61%; P < 0.001). The post-ACA cohort was associated with increased 2-year overall survival compared to the pre-ACA cohort (pre-ACA 62%; post-ACA 69%; P < 0.001). CONCLUSION Patients were more likely to be guideline adherent and receive GA-RT after the implementation of the ACA. This increase in guideline adherence was also associated with an increase in 2-year survival, indicating implementation of the ACA may have improved patient outcomes. AUTHOR DISCLOSURE R.J. Thomas: None. D. Provenzano: None. S. Goyal: Partner; Center for Rheumatic Diseases. M. Loew: None. B. Long: None. M. Lopez-Acevedo: None. Y.J. Rao: None.

Recently, deep learning has become much more popular in computer vision applications. The Convolutional Neural Network (CNN) has brought a breakthrough in image segmentation, especially for medical images. In this regard, the UNet is the predominant approach to the medical image segmentation task. The U-Net not only performs well in segmenting multimodal medical images generally, but also in some difficult cases. We found, however, that the classical U-Net architecture has limitations in several respects. Therefore, we applied modifications: 1) designed efficient CNN architecture to replace encoder and decoder, 2) applied residual module to replace skip connection between encoder and decoder to improve, based on the-state-of-the-art U-Net model. Following these modifications, we designed a novel architecture -- DC-UNet, as a potential successor to the U-Net architecture. We created a new effective CNN architecture and built the DC-UNet based on this CNN. We have evaluated our model on three datasets with difficult cases and have obtained a relative improvement in performance of 2.90%, 1.49%, and 11.42% respectively compared with classical UNet. In addition, we used the Tanimoto similarity measure to replace the Jaccard measure for gray-to-gray image comparisons.

PURPOSE To provide an assessment of safety regarding high-dose-rate after-loading brachytherapy (HDR-BT) based on adverse events reported to the OpenFDA, an open access database maintained by the United States Food and Drug Administration (FDA). METHODS OpenFDA was queried for HDR-BT events between 1993 and 2019. A brachytherapist categorized adverse events (AEs) based on disease site, applicator, manufacturer, event type, dosimetry impact, and outcomes. Important findings are summarized. RESULTS 372 AEs were reported between 1993 and 2019, with a downwards trend after 2014. Nearly half of AEs (48.9%) were caused by a device malfunction, and 27.4% resulted in patient injury. Breast (49.2%) and Gyn (23.7%) were the most common disease sites of AEs. Applicator breaks cause the majority of AEs (64.2%) and breast balloon implants were the most common applicator to malfunction (38.7%). User error contributed to only 16.7% of events. 11.0% of events required repair of the afterloader. There were no reported staff injuries or patient deaths from an AE, however 24.7% of patients received resultant incorrect radiation dose, 16.4% required additional procedures to rectify the AE, and 3.0% resulted in unintended radiation to staff. CONCLUSION The OpenFDA database has shown a decreasing trend in AEs since 2014 for HDR-BT. Most AEs are not caused by user error and do not cause patient injury or incorrect radiation dose. Investigation into methods to prevent failures and improve applicators such as the breast balloon could improve safety. These results support the continued use of HDR-BT as a safe treatment modality for cancer.

Coronavirus disease 2019 (COVID-19) is an ongoing global pandemic that has spread rapidly since December 2019. Real-time reverse transcription polymerase chain reaction (rRT-PCR) and chest computed tomography (CT) imaging both play an important role in COVID-19 diagnosis. Chest CT imaging offers the benefits of quick reporting, a low cost, and high sensitivity for the detection of pulmonary infection. Recently, deep-learning-based computer vision methods have demonstrated great promise for use in medical imaging applications, including X-rays, magnetic resonance imaging, and CT imaging. However, training a deep-learning model requires large volumes of data, and medical staff faces a high risk when collecting COVID-19 CT data due to the high infectivity of the disease. Another issue is the lack of experts available for data labeling. In order to meet the data requirements for COVID-19 CT imaging, we propose a CT image synthesis approach based on a conditional generative adversarial network that can effectively generate high-quality and realistic COVID-19 CT images for use in deep-learning-based medical imaging tasks. Experimental results show that the proposed method outperforms other state-of-the-art image synthesis methods with the generated COVID-19 CT images and indicates promising for various machine learning applications including semantic segmentation and classification., Comment: Accepted by IEEE Journal of Biomedical and Health Informatics (J-BHI)

To predict the histologic grade and type of small papillary renal cell carcinomas (pRCCs) using texture analysis and machine learning algorithms. This was a retrospective HIPAA-compliant study. 24 noncontrast (NC), 22 corticomedullary (CM) phase, and 24 nephrographic (NG) phase CTs of small (< 4 cm) surgically resected pRCCs were identified. Surgical pathology classified the tumors as low- or high-Fuhrman histologic grade and type 1 or 2. The axial image with the largest cross-sectional tumor area was exported and segmented. Six histogram and 31 texture (20 gray-level co-occurrences and 11 gray-level run-lengths) features were calculated for each tumor in each phase. Feature values in low- versus high-grade and type 1 versus 2 pRCCs were compared. Area under the receiver operating curve (AUC) was calculated for each feature to assess prediction of histologic grade and type of pRCCs in each phase. Histogram, texture, and combined histogram and texture feature sets were used to train and test three classification algorithms (support vector machine (SVM), random forest, and histogram-based gradient boosting decision tree (HGBDT)) with stratified shuffle splits and threefold cross-validation; AUCs were calculated for each algorithm in each phase to assess prediction of histologic grade and type of pRCCs. Individual histogram and texture features did not have statistically significant differences between low- and high-grade or type 1 and type 2 pRCCs across all phases. Individual features had low predictive power for tumor grade or type in all phases (AUC < 0.70). HGBDT was highly accurate at predicting pRCC histologic grade and type using histogram, texture or combined histogram and texture feature data from the CM phase (AUCs = 0.97–1.0). All algorithms had highest AUCs using CM phase feature data sets; AUCs decreased using feature sets from NC or NG phases. The histologic grade and type of small pRCCs can be predicted with classification algorithms using CM histogram and texture features, which outperform NC and NG phase image data. The accurate prediction of pRCC histologic grade and type may be able to further guide management of patients with small (< 4 cm) pRCCs being considered for active surveillance.

Humans can count very fast by subitizing, but slow substantially as the number of objects increases. Previous studies have shown a trained deep neural network (DNN) detector can count the number of objects in an amount of time that increases slowly with the number of objects. Such a phenomenon suggests the subitizing ability of DNNs, and unlike humans, it works equally well for large numbers. Many existing studies have successfully applied DNNs to object counting, but few studies have studied the subitizing ability of DNNs and its interpretation. In this paper, we found DNNs do not have the ability to generally count connected components. We provided experiments to support our conclusions and explanations to understand the results and phenomena of these experiments. We proposed three ML-learnable characteristics to verify learnable problems for ML models, such as DNNs, and explain why DNNs work for specific counting problems but cannot generally count connected components.

Background: Gulf War Illness (GWI) and Chronic Fatigue Syndrome (CFS) are two debilitating disorders that share similar symptoms of chronic pain, fatigue, and exertional exhaustion after exercise. Many physicians continue to believe that both are psychosomatic disorders and to date no underlying etiology has been discovered. As such, uncovering objective biomarkers is important to lend credibility to criteria for diagnosis and to help differentiate the two disorders. Methods: We assessed cognitive differences in 80 subjects with GWI and 38 with CFS by comparing corresponding fMRI scans during 2-back working memory tasks before and after exercise to model brain activation during normal activity and after exertional exhaustion, respectively. Voxels were grouped by the count of total activity into the Automated Anatomical Labeling (AAL) atlas and used in an &ldquo, ensemble&rdquo, series of machine learning algorithms to assess if a multi-regional pattern of differences in the fMRI scans could be detected. Results: A K-Nearest Neighbor (70%/81%), Linear Support Vector Machine (SVM) (70%/77%), Decision Tree (82%/82%), Random Forest (77%/78%), AdaBoost (69%/81%), Na&iuml, ve Bayes (74%/78%), Quadratic Discriminant Analysis (QDA) (73%/75%), Logistic Regression model (82%/82%), and Neural Net (76%/77%) were able to differentiate CFS from GWI before and after exercise with an average of 75% accuracy in predictions across all models before exercise and 79% after exercise. An iterative feature selection and removal process based on Recursive Feature Elimination (RFE) and Random Forest importance selected 30 regions before exercise and 33 regions after exercise that differentiated CFS from GWI across all models, and produced the ultimate best accuracies of 82% before exercise and 82% after exercise by Logistic Regression or Decision Tree by a single model, and 100% before and after exercise when selected by any six or more models. Differential activation on both days included the right anterior insula, left putamen, and bilateral orbital frontal, ventrolateral prefrontal cortex, superior, inferior, and precuneus (medial) parietal, and lateral temporal regions. Day 2 had the cerebellum, left supplementary motor area and bilateral pre- and post-central gyri. Changes between days included the right Rolandic operculum switching to the left on Day 2, and the bilateral midcingulum switching to the left anterior cingulum. Conclusion: We concluded that CFS and GWI are significantly differentiable using a pattern of fMRI activity based on an ensemble machine learning model.

Gulf War Illness (GWI) is a debilitating condition characterized by dysfunction of cognition, pain, fatigue, sleep, and diverse somatic symptoms with no known underlying pathology. As such, uncovering objective biomarkers such as differential regions of activity within a Functional Magnetic Resonance Imaging (fMRI) scan is important to enhance validity of the criteria for diagnosis. Symptoms are exacerbated by mild activity, and exertional exhaustion is a key complaint amongst sufferers. We modeled this exertional exhaustion by having GWI (n = 80) and sedentary control (n = 31) subjects perform submaximal exercise stress tests on two consecutive days. Cognitive differences were assessed by comparing fMRI scans performed during 2-Back working memory tasks before and after the exercise. Machine learning algorithms were used to identify differences in brain activation patterns between the two groups on Day 1 (before exercise) and Day 2 (after exercise). The numbers of voxels with t > 3.17 (corresponding to p < 0.001 uncorrected) were determined for brain regions defined by the Automated Anatomical Labeling (AAL) atlas. Data were divided 70:30 into training and test sets. Recursive feature selection identified twenty-nine regions of interest (ROIs) that significantly distinguished GWI from control on Day 1 and 28 ROIs on Day 2. Ten regions were present in both models between the two days, including right anterior insula, orbital frontal cortex, thalamus, bilateral temporal poles, and left supramarginal gyrus and cerebellar Crus 1. The models had 70% accuracy before exercise on Day 1 and 85% accuracy after exercise on Day 2, indicating the logistic regression model significantly differentiated subjects with GWI from the sedentary control group. Exercise caused changes in these patterns that may indicate the cognitive differences caused by exertional exhaustion. A second set of predictive models was able to classify previously identified GWI exercise subgroups START, STOPP, and POTS for both Days 1 and Days 2 with 67% and 69% accuracy respectively. This study was the first of its kind to differentiate GWI and the three sub-phenotypes START, STOPP, and POTS from a sedentary control using a logistic regression estimation method.

The 2019 Novel Coronavirus (COVID-19) has caused an acute shortage of personal protective equipment (PPE) globally as well as shortage in the ability to test PPE such as respirator fit testing. This limits not only the ability to fit PPE to medical practitioners, but also the ability to rapidly prototype and produce alternative sources of PPE as it is difficult to validate fit. At the George Washington University, we evaluated an easily sourced method of qualitative fit testing using a nebulizer or “atomizer” and a sodium saccharin solution in water. If aerosolized saccharin entered candidate masks due to poor fit or inadequate filtration, then a sweet taste was detected in the mouth of the user. This method was tested against previously fit tested Milwaukee N95 and 3D Printed Reusable N95 Respirator as a positive control. A Chinese sourced KN95, cotton cloth material, and surgical mask were tested as other masks of interest. Sensitivity testing was done with no mask prior to fit test. A sweet taste was detected for both the surgical mask and cotton cloth, demonstrating a lack of seal. However, there was no sweet taste detected for the Milwaukee N95, 3D Printed Reusable N95 Respirator, or Chinese KN95. These results demonstrate this could be a valuable methodology for rapid prototyping, evaluation, and validation of fit in a non-clinical environment for use in creation of PPE. This method should be not be used without confirmation in a formal qualitative or quantitative fit test but can be used to preserve those resources until developers are confident that potential new N95 comparable respirators will pass. We strongly suggest validation of masks and respirators with Occupational Safety and Health Administration (OSHA) approved fit testing prior to use in a clinical environment.

The 2019 Novel Coronavirus (COVID-19) has caused an acute reduction in world supplies of personal protective equipment (PPE) due to increased demand. To combat the impending shortage of equipment including N95 masks, the George Washington University Hospital (GWUH) developed a 3D printed reusable N95 comparable respirator that can be used with multiple filtration units. We evaluated several candidate prototype respirator models, 3D printer filaments, and filtration units detailed here. Our most recent working model was based on a respirator found on an open source maker website and was developed with PLA (printer filament), a removable cap, a removable filtration unit consisting of two layers of MERV 16 sandwiched between MERV 13, and removable elastic bands to secure the mask. Our candidate mask passed our own suction test protocol to evaluate leakage and passed a qualitative Bitrix N95 fit test at employee health at GWUH. Further efforts are directed at improving the current model for seal against face, comfort, and sizing. The 3D model is available upon request and in the supplement of this paper. We welcome collaboration with other institutions and suggest other facilities consider mask fit for their own population when exploring this concept.

To evaluate clustering results is a significant part of cluster analysis. There are no true class labels for clustering in typical unsupervised learning. Thus, a number of internal evaluations, which use predicted labels and data, have been created. They are also named internal cluster validity indices (CVIs). Without true labels, to design an effective CVI is not simple because it is similar to create a clustering method. And, to have more CVIs is crucial because there is no universal CVI that can be used to measure all datasets, and no specific method for selecting a proper CVI for clusters without true labels. Therefore, to apply more CVIs to evaluate clustering results is necessary. In this paper, we propose a novel CVI - called Distance-based Separability Index (DSI), based on a data separability measure. We applied the DSI and eight other internal CVIs including early studies from Dunn (1974) to most recent studies CVDD (2019) as comparison. We used an external CVI as ground truth for clustering results of five clustering algorithms on 12 real and 97 synthetic datasets. Results show DSI is an effective, unique, and competitive CVI to other compared CVIs. In addition, we summarized the general process to evaluate CVIs and created a new method - rank difference - to compare the results of CVIs., Comment: 8 pages, 4 figures. Accepted by IEEE ICTAI 2020 (Long Paper & Oral Presentation)

For supervised learning models, the analysis of generalization ability (generalizability) is vital because the generalizability expresses how well a model will perform on unseen data. Traditional generalization methods, such as the VC dimension, do not apply to deep neural network (DNN) models. Thus, new theories to explain the generalizability of DNNs are required. In this study, we hypothesize that the DNN with a simpler decision boundary has better generalizability by the law of parsimony (Occam's Razor). We create the decision boundary complexity (DBC) score to define and measure the complexity of decision boundary of DNNs. The idea of the DBC score is to generate data points (called adversarial examples) on or near the decision boundary. Our new approach then measures the complexity of the boundary using the entropy of eigenvalues of these data. The method works equally well for high-dimensional data. We use training data and the trained model to compute the DBC score. And, the ground truth for model's generalizability is its test accuracy. Experiments based on the DBC score have verified our hypothesis. The DBC is shown to provide an effective method to measure the complexity of a decision boundary and gives a quantitative measure of the generalizability of DNNs., Comment: 7 pages, 11 figures. Accepted by ICMLA 2020

Automatic segmentation of infection areas in computed tomography (CT) images has proven to be an effective diagnostic approach for COVID-19. However, due to the limited number of pixel-level annotated medical images, accurate segmentation remains a major challenge. In this paper, we propose an unsupervised domain adaptation based segmentation network to improve the segmentation performance of the infection areas in COVID-19 CT images. In particular, we propose to utilize the synthetic data and limited unlabeled real COVID-19 CT images to jointly train the segmentation network. Furthermore, we develop a novel domain adaptation module, which is used to align the two domains and effectively improve the segmentation network's generalization capability to the real domain. Besides, we propose an unsupervised adversarial training scheme, which encourages the segmentation network to learn the domain-invariant feature, so that the robust feature can be used for segmentation. Experimental results demonstrate that our method can achieve state-of-the-art segmentation performance on COVID-19 CT images.

The Generative Adversarial Network (GAN) is a state-of-the-art technique in the field of deep learning. A number of recent papers address the theory and applications of GANs in various fields of image processing. Fewer studies, however, have directly evaluated GAN outputs. Those that have been conducted focused on using classification performance, e.g., Inception Score (IS) and statistical metrics, e.g., Fr\'echet Inception Distance (FID). Here, we consider a fundamental way to evaluate GANs by directly analyzing the images they generate, instead of using them as inputs to other classifiers. We characterize the performance of a GAN as an image generator according to three aspects: 1) Creativity: non-duplication of the real images. 2) Inheritance: generated images should have the same style, which retains key features of the real images. 3) Diversity: generated images are different from each other. A GAN should not generate a few different images repeatedly. Based on the three aspects of ideal GANs, we have designed the Likeness Score (LS) to evaluate GAN performance, and have applied it to evaluate several typical GANs. We compared our proposed measure with two commonly used GAN evaluation methods: IS and FID, and four additional measures. Furthermore, we discuss how these evaluations could help us deepen our understanding of GANs and improve their performance., Comment: 16 pages, 11 figures. Accepted by the Neural Computing and Applications journal

In Deep Neural Networks (DNNs), the learning process (updating the parameters) is driven by the gradients of a loss function; thus, the loss function does not only measures the "goodness" of classification, but also determines the form of representations. In this paper we propose two loss functions for Deep Neural Networks (DNNs), namely Multi-class Discriminant Analysis Loss Function and Maximum Scatter to Closest Neigh-bor Class loss function, which share the same idea, i.e., they minimize the within-class variance (scatter) and maximize the minimum between-class distance. They overcome the problem of overlapping class in the Linear Discriminant Analysis (LDA)-based loss functions and can be used to multi-class classification tasks. We show that our proposed loss functions achieve state-of-the-art accuracy in a medical image classification task. Particularly, in the Optical Coherence Tomography (OCT) image dataset, we got nearly perfect (99.6%) test accuracy.

Objectives Active dynamic thermography (ADT) is a non-contact imaging technique that characterizes non-homogeneities in thermal conductance through objects as a response to applied energy stimulus. The aim of this study was to (i) develop a heat transfer model to define the relationship between thermal stimulation and resolution and (ii) empirically quantify the resolution an ADT imaging system can detect through a range of depths of human skin. Materials and methods A heat transfer model was developed to describe a thermally non-conductive object below a sheet of skin. The size and depth of the object were varied to simulate wound conditions, while the intensity and duration of thermal stimulation were varied to define stimulation parameters. The model was solved by numerical analysis. For ex vivo experimentation, freshly excised human pannus tissue was cut into sheets of thickness 2.54-6.35 × 10-4 m (0.010-0.025vinches) for a total of 48 grafts from 12 patients. Grafts were placed over a 3D printed resolution target with objects ranging from 0.445-0.125 LP/mm. Stimulation from a 300 W halogen lamp array was applied for 0.5-14 seconds for a total of 480 experiments. Results ADT resolved a peak of 0.428 ± 0.025 LP/mm for 2.54 × 10-4 m (0.010 inches) skin thickness, 0.384 ± 0.030 LP/mm for 3.81 × 10-4 m (0.015 inches), 0.325 ± 0.042 LP/mm for 5.08 × 10-4 m (0.020 inches) and 0.249 ± 0.057 LP/mm for 6.35 × 10-4 m (0.025 inches) skin thickness. Additionally, it was determined that the ideal duration of stimulation energy with a 300 W stimulation system was 4 seconds for 2.54 × 10-4 m, 6 seconds for 3.81 × 10-4 m, 8 seconds for 5.08 × 10-4 m, and 14 seconds for 6.35 × 10-4 m skin thickness. Conclusions This study has characterized the correlation between thermal stimulus input and resolvable object size and depth for ADT. Through ex vivo experimentation it has also quantified the functional imaging depth to below the sub-cutis, beyond that of conventional imaging techniques. Lasers Surg. Med. © 2018 Wiley Periodicals, Inc.

Near-infrared diffuse reflectance imaging spectroscopy (NIR-RIS, 1000 to 2500 nm) was used to map the use of alkyd and oil paints in Jackson Pollock's Number 1, 1950 (Lavender Mist), one of his most important ‘drip’ or ‘poured’ paintings. Pollock's drip paintings were created by allowing the paint to “pour” from his brush/stick down onto the canvas. Prior analysis of micro-samples from drip paintings found Pollock was extensively using alkyd-resin paints by 1949 (in addition to oil) which led to a hypothesis that he selectively used alkyd and oil paints to obtain the fluid, gestural marks characteristic of his mature style. To determine where he used alkyd and oil paints in Lavender Mist, we utilized near-infrared spectral regions (1615–1850 and 1860–2200 nm) that allowed discrimination between these binding media, especially when the first derivative spectra were examined. The near-infrared image cubes were collected with a high-spectral resolution (2.8 nm sampling) hyperspectral camera. Using convex geometry-based multivariate analysis, three spectral endmembers were identified: oil, alkyd resin, and a third binder. The mapping results show oil binding medium was confined to the uppermost layers of the painting, corresponding with long, more-or-less straight, white skeins of paint. An alkyd binding medium maps to long white skeins of paint (both straight and curved), in addition to many irregular-shaped, disconnected regions of white and blue-green paint, indicating the mapped alkyd paint is below other layers. The third spectral endmember maps to an off-white paint that may be a mixture/layering of oil and alkyd media. X-Ray fluorescence imaging spectroscopy and site-specific reflectance spectroscopy (350–2500 nm) found evidence that the oil and alkyd white paints contain zinc and titanium whites. However, the oil paint likely contains the anatase mineral form of titanium dioxide, whereas the alkyd paint likely contains rutile titanium dioxide. As these pigments do not absorb near-infrared radiation, the mapping results demonstrate the ability to use NIR-RIS to map modern paint binders. Furthermore, the use of oil and alkyd paints seem to be intentional choices by Pollock: the oil because it has more body, and the alkyd because it is more fluid and dries more quickly.

Breast cancer is the second leading cause of death for women in the U.S. Early detection of breast cancer is key to higher survival rates to breast cancer patients. We are investigating infrared (IR) thermography as a noninvasive adjunct to mammography for breast cancer screening. IR imaging is radiation-free, pain-free, and non-contact. Automatic segmentation of the breast area from the acquired full-size breast IR images will help limit the area for tumor search, as well as reduce the time and effort costs of manual hand segmentation. Autoencoder-like convolutional and deconvolutional neural networks (C-DCNN) had been applied to automatically segment the breast area in IR images in previous studies. In this study, we applied a state-of-the-art deep-learning segmentation model, MultiResUnet, which consists of an encoder part to capture features and a decoder part for precise localization. It was used to segment the breast area by using a set of breast IR images, collected in our clinical trials by imaging breast cancer patients and normal volunteers with our infrared camera (N2 Imager). The database we used has 450 images, acquired from 14 patients and 16 volunteers. We used a thresholding method to remove interference in the raw images and remapped them from the original 16-bit to 8-bit, and then cropped and segmented the 8-bit images manually. Experiments using leave-one-out cross-validation (LOOCV) and comparison with the ground-truth images by using Tanimoto similarity show that the average accuracy of MultiResUnet is 91.47%, which is about 2% higher than that of the autoencoder. MultiResUnet offers a better approach to segment breast IR images than our previous model.

Recently, a number of papers have addressed the theory and applications of the Generative Adversarial Network (GAN) in various fields of image processing. Fewer studies, however, have directly evaluated GAN outputs. Those that have been conducted focused on using classification performance and statistical metrics. In this paper, we consider a fundamental way to evaluate GANs by directly analyzing the images they generate, instead of using them as inputs to other classifiers. We consider an ideal GAN according to three aspects: 1) Creativity: non-duplication of the real images. 2) Inheritance: generated images should have the same style, which retains key features of the real images. 3) Diversity: generated images are different from each other. Based on the three aspects, we have designed the Creativity-Inheritance-Diversity (CID) index to evaluate GAN performance. We compared our proposed measures with three commonly used GAN evaluation methods: Inception Score (IS), Frechet Inception Distance (FID) and 1-Nearest Neighbor classifier (1NNC). In addition, we discuss how the evaluation could help us deepen our understanding of GANs and improve their performance.

Purpose To use machine-learning algorithms and blur measure (BM) operators to automatically detect motion blur in mammograms. Motion blur has been reported to reduce lesion detection performance and mask small abnormalities, resulting in failure to detect them until they reach more advanced stages. Automatic detection of blur could support the clinical decision-making process during the mammography exam by allowing for an immediate retake, thereby preventing unnecessary expense, time, and patient anxiety. Methods Blur was simulated mathematically to mimic the real blur seen in clinical practice. The blur point-spread-function (PSF) mask is generated by distributing pixel intensity of an image pixel moving under random motion within the range of blur effect (the maximum amount of tissue motion allowed). The random motion trajectory vector is generated on a super-sampled image frame to accommodate smaller substeps; the vector was then sampled on a regular pixel grid using subpixel linear interpolation to generate the blur PSF mask. This randomly generated motion trajectory is constrained by several factors: the effects of variations in tissue elasticity, imaging exposure time, and size of blur effect (motion boundary in millimeters) were examined. The blur mask is convolved with a mammogram to create blur. Five motion blur magnitudes (0.1, 0.25, 0.5, 1.0, and 1.5 mm) were simulated on 244 and 434 mammograms from the INbreast and DDSM databases, respectively. Blur was quantified using nine BM operators for each mammogram and at each blur level. The mammograms were assigned to training (70%) and testing (30%) datasets to train three machine-learning classifiers: Ensemble Bagged Trees, fine Gaussian SVM, and weighted KNN, to distinguish five levels of blurred from unblurred mammograms, using six-way classification. Results For the INbreast mammograms, the average classification accuracies were 87.7%, 85.7%, and 85.7% for Ensemble Bagged Trees, fine Gaussian SVM, and weighted KNN, respectively, and the average classification accuracies for DDSM were 93.5%, 93.6%, and 92.7% for Ensemble Bagged Trees, fine Gaussian SVM, and weighted KNN, respectively. Conclusions Preliminary results show the potential to detect simulated blur automatically using those methods. Although limited work has been done to quantify the effects of motion blur on radiologists' performance, there is evidence that motion blur might not be detected visually by a human observer and could negatively affect radiologists' lesion detection performance. As of this date, no other study has investigated the ability of machine-learning classifiers and BM operators to detect motion blur in mammograms.

This paper investigates optical coherence tomography (OCT) as an advanced, non-invasive method for 2D and 3D imaging of the surface and subsurface morphology of glass cultural heritage. The OCT system used is a commercial ThorLabs Ganymede II spectral domain-Fourier domain system with a 930 nm center wavelength, axial resolution of 4-6 μm, and lateral resolution of 8 μm. Results from model alkali silicate glass artificially aged at 90°C and 90% RH allow distinction of real features from artifacts produced by the highly reflective glass, and serve as a basis for interpretation of deterioration phenomena. Analytical results from historical glass artifacts are focused on a group of musical glass flutes created in Paris by Claude Laurent between 1807 and 1848. OCT images are compared to results of destructive analysis of the same samples and objects by scanning electron microscopy with backscattered electron imaging of cross-sections, as well as non-invasive light microscopy and NIR fiber optic reflectance spectrometry, the latter of which yields complementary molecular information in terms of water vibrations in hydrated glass.

In the U.S., breast cancer is diagnosed in about 12% of women during their lifetime and it is the second leading reason for women’s death. Since early diagnosis could improve treatment outcomes and longer survival times for breast cancer patients, it is significant to develop breast cancer detection techniques. The Convolutional Neural Network (CNN) can extract features from images automatically and then perform classification. To train the CNN from scratch, however, requires a large number of labeled images, which is infeasible for some kinds of medical image data such as mammographic tumor images. In this paper, we proposed two solutions to the lack of training images. 1)To generate synthetic mammographic images for training by the Generative Adversarial Network (GAN). Adding GAN generated images made to train CNN from scratch successful and adding more GAN images improved CNN’s validation accuracy to at most (best) 98.85%. 2)To apply transfer learning in CNN. We used the pre-trained VGG-16 model to extract features from input mammograms and used these features to train a Neural Network (NN)-classifier. The stable average validation accuracy converged at about 91.48% for classifying abnormal vs. normal cases in the DDSM database. Then, we combined the two deep-learning based technologies together. That is to apply GAN for image augmentation and transfer learning in CNN for breast cancer detection. To the training set including real and GAN augmented images, although transfer learning model did not perform better than the CNN, the speed of training transfer learning model was about 10 times faster than CNN training. Adding GAN images can help training avoid over-fitting and image augmentation by GAN is necessary to train CNN classifiers from scratch. On the other hand, transfer learning is necessary to be applied for training on pure real images. To apply GAN to augment training images for training CNN classifier obtained the best classification performance.

One consensus in the machine learning community is that obtaining good representations of the data is crucial for the classification tasks. But establishing a clear objective for representation learning is an open question and difficult. In this paper, we propose the Discriminant Analysis Loss Function (DALF) for the representation learning in Deep Neural Networks (DNNs). The gradients of DALF explicitly minimize the within-class variances (scatter) and maximize the between-class variances. We use DALF to drive the training of DNNs and call them Discriminant Analysis Deep Neural Networks (DisAnDNNs). Compared to other Linear Discriminant Analysis (LDA)-based cost functions, the computational cost of DALF is drastically reduced by avoiding eigen-decomposition and matrix inversion. We used simple datasets to illustrate the geometric meaning of DALF and compared it with LDA, then experimented with DALF-driven Residual Learning Nets (ResNets) on the pediatric pneumonia (chest X-ray image) dataset. The experimental results show that the DisAnDNNs achieve state-of the-art accuracy in the binary classification task. Particularly, in the pediatric pneumonia dataset, we achieved the accuracy of 96.63%, with a sensitivity of 99.23% and a specificity of 92.30%, all of which are better than the results in the literature that published the dataset.

Breast cancer is the second leading cause of death for women in the U.S. Early detection of breast cancer has been shown to be the key to higher survival rates for breast cancer patients. We are investigating infrared thermography as a noninvasive adjunctive to mammography for breast screening. Thermal imaging is safe, radiation-free, pain-free, and non-contact. Segmentation of breast area from the acquired thermal images will help limit the area for tumor search and reduce the time and effort needed for manual hand segmentation. Autoencoder-like convolutional and deconvolutional neural networks (C-DCNN) are promising computational approaches to automatically segment breast areas in thermal images. In this study, we apply the C-DCNN to segment breast areas from our thermal breast images database, which we are collecting in our clinical trials by imaging breast cancer patients with our infrared camera (N2 Imager). For training the C-DCNN, the inputs are 132 gray-value thermal images and the corresponding manually-cropped breast area images (binary masks to designate the breast areas). For testing, we input thermal images to the trained C-DCNN and the output after post-processing are the binary breast-area images. Cross-validation and comparison with the ground-truth images show that the C-DCNN is a promising method to segment breast areas. The results demonstrate the capability of C-DCNN to learn essential features of breast regions and delineate them in thermal images.

The Convolutional Neural Network (CNN) is a promising technique to detect breast cancer based on mammograms. Training the CNN from scratch, however, requires a large amount of labeled data. Such a requirement usually is infeasible for some kinds of medical image data such as mammographic tumor images. Because improvement of the performance of a CNN classifier requires more training data, the creation of new training images -- image augmentation – could be one solution to this problem. In this study, we applied the Generative Adversarial Network (GAN) to generate synthetic mammographic images from the Digital Database for Screening Mammography (DDSM). From the DDSM, we cropped two sets of regions of interest (ROIs) from the images: normal and abnormal (cancer/tumor) Those ROIs were used to train the GAN, and the GAN then generated synthetic images. To compare the GAN with the affine transformation augmentation methods, such as rotation, shifting, scaling, etc., we used six groups of ROIs (three simple groups: affine augmented, GAN synthetic, real (original), and three mixture groups of each pair of the three simple groups) for each to train a CNN classifier from scratch. And, we used real ROIs that were not used in training to validate classification outcomes. Our results show that, to classify the normal ROIs and abnormal (tumor) ROIs from DDSM, adding GAN-generated ROIs to the training data can reduce overfitting of the classifier. But the affine transformations performed slightly better than GAN. Therefore, GAN could be an optional augmentation approach. The images augmented by GAN or affine transformation cannot substitute entirely for real images to train CNN classifiers because the absence of real images in the training set will cause serious over-fitting with more training.

Atrial fibrillation is the most common cardiac arrhythmia. It is being effectively treated using the radiofrequency ablation (RFA) procedure, which destroys culprit tissue and creates scars that prevent the spread of abnormal electrical activity. Long-term success of RFA could be improved further if ablation lesions can be directly visualized during the surgery. We have shown that autofluorescence-based hyperspectral imaging (aHSI) can help to identify lesions based on spectral unmixing. We show that use of [Formula: see text]-means clustering, an unsupervised learning method, is capable of detecting RFA lesions without a priori knowledge of the lesions’ spectral characteristics. We also show that the number of spectral bands required for successful lesion identification can be significantly reduced, enabling the use of increased spectral bandwidth. Together, these findings can help with clinical implementation of a percutaneous aHSI catheter, since by reducing the number of spectral bands one can reduce hypercube acquisition and processing times, and by increasing the spectral width of individual bands one can collect more photons. The latter is of critical importance in low-light applications such as intracardiac aHSI. The ultimate goal of our studies is to help improve clinical outcomes for atrial fibrillation patients.

Direct visualization of the ablated region in the left atrium during radiofrequency ablation (RFA) surgery for treating atrial fibrillation (AF) can improve therapy success rates. Our visualization approach is auto-fluorescence hyperspectral imaging (aHSI), which constructs each hypercube containing 31 auto-fluorescence images of the tissue. We wish to use the spectral information to characterize ablated lesions as being successful or not. In this paper, we reshaped one hypercube to a 2D matrix. Each row (sample) in the matrix represents a pixel in the spatial dimension, and the matrix has 31 columns corresponding to 31 spectral features. Then, we applied k-means clustering to detect ablated regions without a priori knowledge about the lesion. We introduced an accuracy index to evaluate the results of k-means by comparing with the reference truth images quantitatively. To speed-up the detection process, we implemented a grouping procedure to decrease the number of features. The 31 features were divided into four contiguous disjoint groups. In each group, the summation of values yielded a new feature. By the same evaluation method, we found the best 4-feature groups to adequately detect the lesions from all possible combinations. The average accuracy for detection by k-means (ke10) using 31 features was about 74p of reference truth images. And, for using the best grouped 4 features, it was about 95p of that using 31 features. The time cost of 4-feature clustering is about 41p of the 31-feature clustering. We expect that the reduction of time for both acquisition and processing will make possible intraoperative real-time display of ablation status.

This paper proposes the use of near infrared (900 to 1700 nm) transmitted light imaging with a hyperspectral camera to obtain watermarks from prints. Specifically, we show that principal component analysis applied to the hyperspectral image cube collected in the near infrared was able to separate the watermark from text printed in carbon black ink on both sides of a page from the Blaue Atlas Maior of 1662. The resulting principal component image of the watermark was compared with an image obtained using a low-energy x-ray source and a phosphor plate. Low-energy x-radiography is becoming the gold standard for imaging watermarks, replacing beta radiography. The watermark obtained by transmitted near infrared hyperspectral imaging was found to possess many of the key features of the watermark revealed by the phosphor plate radiography. The method proposed here offers an additional way to extract watermarks from works of art on paper.

Prior studies have shown the improved ability to identify artists' pigments by combining results from X-ray fluorescence (XRF), which provides elemental information, with reflectance spectroscopy in the visible to near infrared (400-1000 nm) that provides information on electronic transitions. Extending the spectral range of reflectance spectroscopy into the UV, 350-400 nm, allows identification of several white pigments since their electronic transitions occur in this region (e.g., zinc white and rutile and anatase forms of titanium white). Extending the range further into the infrared, out to 2500 nm, provides information on vibrational transitions of various functional groups, such as hydroxyl, carbonate, and methyl groups. This allows better identification of mineral-based pigments and some paint binders. The combination of elemental information with electronic and vibrational transitions provides a more robust method to identify artists' materials in situ. The collection of both sets of spectral information across works of art, such as paintings and works on paper, allows generating a more complete map of artists' materials. Here, we describe a 2-D scanner that simultaneously collects XRF spectra and reflectance spectra from 350 to 2500 nm across the surfaces of works of art. The scanner consists of a stationary, single pixel XRF spectrometer and fiber optic reflectance spectrometer along with a 2-D position-controlled easel that moves the artwork in front of the two detection systems. The dual-mode scanner has been tested on a variety of works of art from illuminated manuscripts (0.1 x 0.1 m(2)) to paintings as large as 1.7 x 1.9 m(2). The scanner is described and two sets of results are presented. The first is the XRF scanning of a large warped panel painting by Andrea del Sarto titled Charity. The second is a combined XRF and reflectance scan of Georges Seurat's painting titled Haymakers at Montfermeil. The XRF was collected at 1 mm spatial sampling and the reflectance spectral data at 3 mm. Combining the results from the data sets was found to enhance the identification of pigments as well as yield distribution maps, in spite of the relatively low reflectance spatial sampling. The elemental and reflectance maps allowed the identification and mapping of lead white, cobalt blue, viridian, ochres, and likely chrome yellow. The maps also provide information on the mixing of pigments. While the reflectance image cube has 10-20x larger spatial samples than desired, the elimination of having to use two hyperspectral cameras to cover the range from 400 to 2500 nm makes for a low cost dual modality scanner.

Active dynamic thermography (ADT) is a non-contact imaging technique that characterizes non-homogeneities in thermal conductance through objects as a response to applied energy stimulus. The aim of this study was to (i) develop a heat transfer model to define the relationship between thermal stimulation and resolution and (ii) empirically quantify the resolution an ADT imaging system can detect through a range of depths of human skin.A heat transfer model was developed to describe a thermally non-conductive object below a sheet of skin. The size and depth of the object were varied to simulate wound conditions, while the intensity and duration of thermal stimulation were varied to define stimulation parameters. The model was solved by numerical analysis. For ex vivo experimentation, freshly excised human pannus tissue was cut into sheets of thickness 2.54-6.35 × 10ADT resolved a peak of 0.428 ± 0.025 LP/mm for 2.54 × 10This study has characterized the correlation between thermal stimulus input and resolvable object size and depth for ADT. Through ex vivo experimentation it has also quantified the functional imaging depth to below the sub-cutis, beyond that of conventional imaging techniques. Lasers Surg. Med. © 2018 Wiley Periodicals, Inc.

The registration of technical art conservation images of Old Master paintings presents unique challenges. Specifically, X-radiographs and reflective infrared (1000–2500 nm) images reveal shifted, or new, compositional elements not visible on the surface of artworks. Here, we describe a new multimodal registration and mosaicking algorithm that is capable of providing accurate alignment of a variety of types of images, such as the registration of multispectral reflective infrared images, X-radiographs, hyperspectral image cubes, and X-ray fluorescence image cubes to reference color images taken at high spatial sampling (300–500 pixels per inch), even when content differences are present, and a validation study has been performed to quantify the algorithm’s accuracy. Key to the algorithm’s success is the use of subsets of wavelet images to select control points and a novel method for filtering candidate control-point pairs. The algorithm has been used to register more than 100 paintings at the National Gallery of Art, D.C. and The Art Institute of Chicago. Many of the resulting registered datasets have been published in online catalogues, providing scholars additional information to further their understanding of the paintings and the working methods of the artists who painted them.

In the U.S., breast cancer is diagnosed in about 12 % of women during their lifetime and it is the second leading reason for women's death. Since early diagnosis could improve treatment outcomes and longer survival times for breast cancer patients, it is significant to develop breast cancer detection techniques. The Convolutional Neural Network (CNN) can extract features from images automatically and then perform classification. To train the CNN from scratch, however, requires a large number of labeled images, which is infeasible for some kinds of medical image data such as mammographic tumor images. A promising solution is to apply transfer learning in CNN. In this paper, we firstly tested three training methods on the MIAS database: 1) trained a CNN from scratch, 2) applied the pre-trained VGG-16 model to extract features from input mammograms and used these features to train a Neural Network (NN)-classifier, 3) updated the weights in several final layers of the pre-trained VGG-16 model by back-propagation (fine-tuning) to detect abnormal regions. We found that method 2) is ideal for study because the classification accuracy of fine-tuning model was just 0.008 higher than that of feature extraction model but time cost of feature extraction model was only about 5% of that of the fine-tuning model. Then, we used method 2) to classify regions: benign vs. normal, malignant vs. normal and abnormal vs. normal from the DDSM database with 10-fold cross validation. The average validation accuracy converged at about 0.905 for abnormal vs. normal cases, and there was no obvious overfitting. This study shows that applying transfer learning in CNN can detect breast cancer from mammograms, and training a NN-classifier by feature extraction is a faster method in transfer learning.

A binocular tone-mapping framework can generate a binocular low-dynamic range (LDR) image pair that preserves more human-perceivable visual contents than a single LDR image. In this paper, to solve the temporal coherency problem when extending from this image-level system to a video-level system, we proposed a new binocular framework that integrates the existing image-level framework and the temporal coherency algorithm. The experimental data show that this proposed new framework can effectively solve the temporal coherency problem and generate binocular LDR videos without disturbing effects.

This research extends the Hierarchical Temporal Memory (HTM) algorithm and applies it to gait recognition. The gait sequence first is decomposed into temporal sub-sequences of spatial sub-regions. The sub-sequence are defined as the period of one step and half step, and the sub-regions are defined as the areas that correspond to body parts. Each sub-area will learn the temporal variation of the body part by constructing Markov Chains. Finally, the classification result is the concatenation of the beliefs of all sub-areas. Unlike other methods, which use gait-specific features, our method uses only image patches of sub-areas. Our extension of previous versions of HTM provides hierarchical temporal inference to cumulate the belief. This generalized new approach is evaluated on a dataset of 151 subjects and two walking conditions. It compares favorably to other current methods used with those data, without requiring problem-specific inputs.

Near infrared imaging spectroscopy (1000-2500 nm) is used to map the use of a fat-containing paint binder, likely egg yolk, in situ on a work of art for the first time. The identification of the use of egg tempera on a 15th century illuminated manuscript leaf (Praying Prophet by Lorenzo Monaco) sheds light on the relationship between painters and illuminators and can inform preservation decisions.

An abnormal thermogram has been shown to be a reliable indicator of increased risk of breast cancer. Numerical modeling techniques for thermography are proposed to quantify the complex relationships between the breast thermal behaviors and the underlying physiological/pathological conditions. Previous thermal modeling techniques did not account for gravity-induced elastic deformation arising from various body postures, nor did they suggest that a dynamic thermal procedure may be used to enhance clinical diagnosis. In this paper, 3D finite element method (FEM)-based thermal and elastic modeling techniques are developed to characterize comprehensively both the thermal and elastic properties of normal and tumorous breast tissues during static and dynamic thermography. In the steady state, gravity-induced breast deformation is found to cause an upper-lower asymmetric surface temperature contrast for sitting/standing up body posture, even though all the thermal and elastic properties are assumed uniform. Additionally, the tumor-induced surface temperature alterations are found to be caused primarily by shallow tumors and to be less sensitive to tumor size than to tumor depth. In the dynamic state, the breast exhibits distinctive temporal patterns that are associated with distinct thermal events: cold stress and thermal recovery induced by changes in the ambient temperature. Specifically, the tumor-induced thermal contrast shows an opposite initial change and delayed peak as compared with the deformation-induced thermal contrast. These findings are expected to provide a stronger foundation for, and greater specificity and precision in, thermographic diagnosis, and treatment of breast cancer.

In vivo autofluorescence hyperspectral imaging of moving objects can be challenging due to motion artifacts and to the limited amount of acquired photons. To address both limitations, we selectively reduced the number of spectral bands while maintaining accurate target identification. Several downsampling approaches were applied to data obtained from the atrial tissue of adult pigs with sites of radiofrequency ablation lesions. Standard image qualifiers such as the mean square error, the peak signal-to-noise ratio, the structural similarity index map, and an accuracy index of lesion component images were used to quantify the effects of spectral binning, an increased spectral distance between individual bands, as well as random combinations of spectral bands. Results point to several quantitative strategies for deriving combinations of a small number of spectral bands that can successfully detect target tissue. Insights from our studies can be applied to a wide range of applications.