Search

Many applications for the automated diagnosis of plant disease have been developed based on the success of deep learning techniques. However, these applications often suffer from overfitting, and the diagnostic performance is drastically decreased when used on test datasets from new environments. In this paper, we propose LeafGAN, a novel image-to-image translation system with own attention mechanism. LeafGAN generates a wide variety of diseased images via transformation from healthy images, as a data augmentation tool for improving the performance of plant disease diagnosis. Thanks to its own attention mechanism, our model can transform only relevant areas from images with a variety of backgrounds, thus enriching the versatility of the training images. Experiments with five-class cucumber disease classification show that data augmentation with vanilla CycleGAN cannot help to improve the generalization, i.e., disease diagnostic performance increased by only 0.7% from the baseline. In contrast, LeafGAN boosted the diagnostic performance by 7.4%. We also visually confirmed the generated images by our LeafGAN were much better quality and more convincing than those generated by vanilla CycleGAN. The code is available publicly at: https://github.com/IyatomiLab/LeafGAN., Accepted as a regular paper in the IEEE Transactions on Automation Science and Engineering (T-ASE)

It is often difficult to correctly infer a writer's emotion from text exchanged online, and differences in recognition between writers and readers can be problematic. In this paper, we propose a new framework for detecting sentences that create differences in emotion recognition between the writer and the reader and for detecting the kinds of expressions that cause such differences. The proposed framework consists of a bidirectional encoder representations from transformers (BERT)-based detector that detects sentences causing differences in emotion recognition and an analysis that acquires expressions that characteristically appear in such sentences. The detector, based on a Japanese SNS-document dataset with emotion labels annotated by both the writer and three readers of the social networking service (SNS) documents, detected "hidden-anger sentences" with AUC = 0.772; these sentences gave rise to differences in the recognition of anger. Because SNS documents contain many sentences whose meaning is extremely difficult to interpret, by analyzing the sentences detected by this detector, we obtained several expressions that appear characteristically in hidden-anger sentences. The detected sentences and expressions do not convey anger explicitly, and it is difficult to infer the writer's anger, but if the implicit anger is pointed out, it becomes possible to guess why the writer is angry. Put into practical use, this framework would likely have the ability to mitigate problems based on misunderstandings., 5 pages, 3 figures. Accepted at the 31st ACM International Conference on Information and Knowledge Management (CIKM '22) as a short paper

To build a robust and practical content-based image retrieval (CBIR) system that is applicable to a clinical brain MRI database, we propose a new framework -- Disease-oriented image embedding with pseudo-scanner standardization (DI-PSS) -- that consists of two core techniques, data harmonization and a dimension reduction algorithm. Our DI-PSS uses skull stripping and CycleGAN-based image transformations that map to a standard brain followed by transformation into a brain image taken with a given reference scanner. Then, our 3D convolutioinal autoencoders (3D-CAE) with deep metric learning acquires a low-dimensional embedding that better reflects the characteristics of the disease. The effectiveness of our proposed framework was tested on the T1-weighted MRIs selected from the Alzheimer's Disease Neuroimaging Initiative and the Parkinson's Progression Markers Initiative. We confirmed that our PSS greatly reduced the variability of low-dimensional embeddings caused by different scanner and datasets. Compared with the baseline condition, our PSS reduced the variability in the distance from Alzheimer's disease (AD) to clinically normal (CN) and Parkinson disease (PD) cases by 15.8-22.6% and 18.0-29.9%, respectively. These properties allow DI-PSS to generate lower dimensional representations that are more amenable to disease classification. In AD and CN classification experiments based on spectral clustering, PSS improved the average accuracy and macro-F1 by 6.2% and 10.7%, respectively. Given the potential of the DI-PSS for harmonizing images scanned by MRI scanners that were not used to scan the training data, we expect that the DI-PSS is suitable for application to a large number of legacy MRIs scanned in heterogeneous environments., Comment: 13 pages, 7 figures

Content-based image retrieval (CBIR) systems are an emerging technology that supports reading and interpreting medical images. Since 3D brain MR images are high dimensional, dimensionality reduction is necessary for CBIR using machine learning techniques. In addition, for a reliable CBIR system, each dimension in the resulting low-dimensional representation must be associated with a neurologically interpretable region. We propose a localized variational autoencoder (Loc-VAE) that provides neuroanatomically interpretable low-dimensional representation from 3D brain MR images for clinical CBIR. Loc-VAE is based on $\beta$-VAE with the additional constraint that each dimension of the low-dimensional representation corresponds to a local region of the brain. The proposed Loc-VAE is capable of acquiring representation that preserves disease features and is highly localized, even under high-dimensional compression ratios (4096:1). The low-dimensional representation obtained by Loc-VAE improved the locality measure of each dimension by 4.61 points compared to naive $\beta$-VAE, while maintaining comparable brain reconstruction capability and information about the diagnosis of Alzheimer's disease., Comment: 6 pages, 6 figures. Accepted at the International Conference on Systems, Man, and Cybernetics (IEEE SMC '22)

We propose a simple yet effective image captioning framework that can determine the quality of an image and notify the user of the reasons for any flaws in the image. Our framework first determines the quality of images and then generates captions using only those images that are determined to be of high quality. The user is notified by the flaws feature to retake if image quality is low, and this cycle is repeated until the input image is deemed to be of high quality. As a component of the framework, we trained and evaluated a low-quality image detection model that simultaneously learns difficulty in recognizing images and individual flaws, and we demonstrated that our proposal can explain the reasons for flaws with a sufficient score. We also evaluated a dataset with low-quality images removed by our framework and found improved values for all four common metrics (e.g., BLEU-4, METEOR, ROUGE-L, CIDEr), confirming an improvement in general-purpose image captioning capability. Our framework would assist the visually impaired, who have difficulty judging image quality., Comment: 6 pages, 4 figures. Accepted at 2022 IEEE 19th International Conference on Smart Communities: Improving Quality of Life Using ICT, IoT and AI (HONET) as a full paper

Although highly accurate automated diagnostic techniques for melanoma have been reported, the realization of a system capable of providing diagnostic evidence based on medical indices remains an open issue because of difficulties in obtaining reliable training data. In this paper, we propose bulk production augmentation (BPA) to generate high-quality, diverse pseudo-skin tumor images with the desired structural malignant features for additional training images from a limited number of labeled images. The proposed BPA acts as an effective data augmentation in constructing the feature detector for the atypical pigment network (APN), which is a key structure in melanoma diagnosis. Experiments show that training with images generated by our BPA largely boosts the APN detection performance by 20.0 percentage points in the area under the receiver operating characteristic curve, which is 11.5 to 13.7 points higher than that of conventional CycleGAN-based augmentations in AUC., IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES2020), Best Paper Award Student Category in Biomedical Imaging and Image Processing

Although attention mechanisms have become fundamental components of deep learning models, they are vulnerable to perturbations, which may degrade the prediction performance and model interpretability. Adversarial training (AT) for attention mechanisms has successfully reduced such drawbacks by considering adversarial perturbations. However, this technique requires label information, and thus, its use is limited to supervised settings. In this study, we explore the concept of incorporating virtual AT (VAT) into the attention mechanisms, by which adversarial perturbations can be computed even from unlabeled data. To realize this approach, we propose two general training techniques, namely VAT for attention mechanisms (Attention VAT) and "interpretable" VAT for attention mechanisms (Attention iVAT), which extend AT for attention mechanisms to a semi-supervised setting. In particular, Attention iVAT focuses on the differences in attention; thus, it can efficiently learn clearer attention and improve model interpretability, even with unlabeled data. Empirical experiments based on six public datasets revealed that our techniques provide better prediction performance than conventional AT-based as well as VAT-based techniques, and stronger agreement with evidence that is provided by humans in detecting important words in sentences. Moreover, our proposal offers these advantages without needing to add the careful selection of unlabeled data. That is, even if the model using our VAT-based technique is trained on unlabeled data from a source other than the target task, both the prediction performance and model interpretability can be improved., Comment: 18 pages, 3 figures. Accepted for publication in Springer Applied Intelligence (APIN)

Medical images have been indispensable and useful tools for supporting medical experts in making diagnostic decisions. However, taken medical images especially throat and endoscopy images are normally hazy, lack of focus, or uneven illumination. Thus, these could difficult the diagnosis process for doctors. In this paper, we propose MIINet, a novel image-to-image translation network for improving quality of medical images by unsupervised translating low-quality images to the high-quality clean version. Our MIINet is not only capable of generating high-resolution clean images, but also preserving the attributes of original images, making the diagnostic more favorable for doctors. Experiments on dehazing 100 practical throat images show that our MIINet largely improves the mean doctor opinion score (MDOS), which assesses the quality and the reproducibility of the images from the baseline of 2.36 to 4.11, while dehazed images by CycleGAN got lower score of 3.83. The MIINet is confirmed by three physicians to be satisfying in supporting throat disease diagnostic from original low-quality images.

Although attention mechanisms have been applied to a variety of deep learning models and have been shown to improve the prediction performance, it has been reported to be vulnerable to perturbations to the mechanism. To overcome the vulnerability to perturbations in the mechanism, we are inspired by adversarial training (AT), which is a powerful regularization technique for enhancing the robustness of the models. In this paper, we propose a general training technique for natural language processing tasks, including AT for attention (Attention AT) and more interpretable AT for attention (Attention iAT). The proposed techniques improved the prediction performance and the model interpretability by exploiting the mechanisms with AT. In particular, Attention iAT boosts those advantages by introducing adversarial perturbation, which enhances the difference in the attention of the sentences. Evaluation experiments with ten open datasets revealed that AT for attention mechanisms, especially Attention iAT, demonstrated (1) the best performance in nine out of ten tasks and (2) more interpretable attention (i.e., the resulting attention correlated more strongly with gradient-based word importance) for all tasks. Additionally, the proposed techniques are (3) much less dependent on perturbation size in AT. Our code is available at https://github.com/shunk031/attention-meets-perturbation, 12 pages, 4 figures. Accepted by IEEE Access on Jun. 21, 2021

The collection of high-resolution training data is crucial in building robust plant disease diagnosis systems, since such data have a significant impact on diagnostic performance. However, they are very difficult to obtain and are not always available in practice. Deep learning-based techniques, and particularly generative adversarial networks (GANs), can be applied to generate high-quality super-resolution images, but these methods often produce unexpected artifacts that can lower the diagnostic performance. In this paper, we propose a novel artifact-suppression super-resolution method that is specifically designed for diagnosing leaf disease, called Leaf Artifact-Suppression Super-Resolution (LASSR). Thanks to its own artifact removal module that detects and suppresses artifacts to a considerable extent, LASSR can generate much more pleasing, high-quality images compared to the state-of-the-art ESRGAN model. Experiments based on a five-class cucumber disease (including healthy) discrimination model show that training with data generated by LASSR significantly boosts the performance on an unseen test dataset by over 21% compared with the baseline, and that our approach is more than 2% better than a model trained with images generated by ESRGAN.

Classical and some deep learning techniques for Arabic text classification often depend on complex morphological analysis, word segmentation, and hand-crafted feature engineering. These could be eliminated by using character-level features. We propose a novel end-to-end Arabic document classification framework, Arabic document image-based classifier (AraDIC), inspired by the work on image-based character embeddings. AraDIC consists of an image-based character encoder and a classifier. They are trained in an end-to-end fashion using the class balanced loss to deal with the long-tailed data distribution problem. To evaluate the effectiveness of AraDIC, we created and published two datasets, the Arabic Wikipedia title (AWT) dataset and the Arabic poetry (AraP) dataset. To the best of our knowledge, this is the first image-based character embedding framework addressing the problem of Arabic text classification. We also present the first deep learning-based text classifier widely evaluated on modern standard Arabic, colloquial Arabic, and Classical Arabic. AraDIC shows performance improvement over classical and deep learning baselines by 12.29% and 23.05% for the micro and macro F-score, respectively.

Practical automated detection and diagnosis of plant disease from wide-angle images (i.e. in-field images containing multiple leaves using a fixed-position camera) is a very important application for large-scale farm management, in view of the need to ensure global food security. However, developing automated systems for disease diagnosis is often difficult, because labeling a reliable wide-angle disease dataset from actual field images is very laborious. In addition, the potential similarities between the training and test data lead to a serious problem of model overfitting. In this paper, we investigate changes in performance when applying disease diagnosis systems to different scenarios involving wide-angle cucumber test data captured on real farms, and propose an effective diagnostic strategy. We show that leading object recognition techniques such as SSD and Faster R-CNN achieve excellent end-to-end disease diagnostic performance only for a test dataset that is collected from the same population as the training dataset (with F1-score of 81.5% - 84.1% for diagnosed cases of disease), but their performance markedly deteriorates for a completely different test dataset (with F1-score of 4.4 - 6.2%). In contrast, our proposed two-stage systems using independent leaf detection and leaf diagnosis stages attain a promising disease diagnostic performance that is more than six times higher than end-to-end systems (with F1-score of 33.4 - 38.9%) on an unseen target dataset. We also confirm the efficiency of our proposal based on visual assessment, concluding that a two-stage model is a suitable and reasonable choice for practical applications., 7 pages, 3 figures

In image-based plant diagnosis, clues related to diagnosis are often unclear, and the other factors such as image backgrounds often have a significant impact on the final decision. As a result, overfitting due to latent similarities in the dataset often occurs, and the diagnostic performance on real unseen data (e,g. images from other farms) is usually dropped significantly. However, this problem has not been sufficiently explored, since many systems have shown excellent diagnostic performance due to the bias caused by the similarities in the dataset. In this study, we investigate this problem with experiments using more than 50,000 images of cucumber leaves, and propose an anti-overfitting pretreatment (AOP) for realizing practical image-based plant diagnosis systems. The AOP detects the area of interest (leaf, fruit etc.) and performs brightness calibration as a preprocessing step. The experimental results demonstrate that our AOP can improve the accuracy of diagnosis for unknown test images from different farms by 12.2% in a practical setting., To appear in the IEEE BigData 2019 Workshop on Big Food and Nutrition Data Management and Analysis

Accurately predicting conversions in advertisements is generally a challenging task, because such conversions do not occur frequently. In this paper, we propose a new framework to support creating high-performing ad creatives, including the accurate prediction of ad creative text conversions before delivering to the consumer. The proposed framework includes three key ideas: multi-task learning, conditional attention, and attention highlighting. Multi-task learning is an idea for improving the prediction accuracy of conversion, which predicts clicks and conversions simultaneously, to solve the difficulty of data imbalance. Furthermore, conditional attention focuses attention of each ad creative with the consideration of its genre and target gender, thus improving conversion prediction accuracy. Attention highlighting visualizes important words and/or phrases based on conditional attention. We evaluated the proposed framework with actual delivery history data (14,000 creatives displayed more than a certain number of times from Gunosy Inc.), and confirmed that these ideas improve the prediction performance of conversions, and visualize noteworthy words according to the creatives' attributes., Comment: 9 pages, 6 figures. Accepted at The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2019) as an applied data science paper

Automated plant diagnosis using images taken from a distance is often insufficient in resolution and degrades diagnostic accuracy since the important external characteristics of symptoms are lost. In this paper, we first propose an effective pre-processing method for improving the performance of automated plant disease diagnosis systems using super-resolution techniques. We investigate the efficiency of two different super-resolution methods by comparing the disease diagnostic performance on the practical original high-resolution, low-resolution, and super-resolved cucumber images. Our method generates super-resolved images that look very close to natural images with 4$\times$ upscaling factors and is capable of recovering the lost detailed symptoms, largely boosting the diagnostic performance. Our model improves the disease classification accuracy by 26.9% over the bicubic interpolation method of 65.6% and shows a small gap (3% lower) between the original result of 95.5%., Comment: Published as a conference paper at CISS 2019, Baltimore, MD, USA

Increasing numbers of MRI brain scans, improvements in image resolution, and advancements in MRI acquisition technology are causing significant increases in the demand for and burden on radiologists' efforts in terms of reading and interpreting brain MRIs. Content-based image retrieval (CBIR) is an emerging technology for reducing this burden by supporting the reading of medical images. High dimensionality is a major challenge in developing a CBIR system that is applicable for 3D brain MRIs. In this study, we propose a system called disease-oriented data concentration with metric learning (DDCML). In DDCML, we introduce deep metric learning to a 3D convolutional autoencoder (CAE). Our proposed DDCML scheme achieves a high dimensional compression rate (4096:1) while preserving the disease-related anatomical features that are important for medical image classification. The low-dimensional representation obtained by DDCML improved the clustering performance by 29.1\% compared to plain 3D-CAE in terms of discriminating Alzheimer's disease patients from healthy subjects, and successfully reproduced the relationships of the severity of disease categories that were not included in the training., Comment: To appear in the IEEE BigData 2019 Workshop on Advances in High Dimensional (AdHD) Big Data

For analysing and/or understanding languages having no word boundaries based on morphological analysis such as Japanese, Chinese, and Thai, it is desirable to perform appropriate word segmentation before word embeddings. But it is inherently difficult in these languages. In recent years, various language models based on deep learning have made remarkable progress, and some of these methodologies utilizing character-level features have successfully avoided such a difficult problem. However, when a model is fed character-level features of the above languages, it often causes overfitting due to a large number of character types. In this paper, we propose a CE-CLCNN, character-level convolutional neural networks using a character encoder to tackle these problems. The proposed CE-CLCNN is an end-to-end learning model and has an image-based character encoder, i.e. the CE-CLCNN handles each character in the target document as an image. Through various experiments, we found and confirmed that our CE-CLCNN captured closely embedded features for visually and semantically similar characters and achieves state-of-the-art results on several open document classification tasks. In this paper we report the performance of our CE-CLCNN with the Wikipedia title estimation task and analyse the internal behaviour., Comment: To appear in IEEE Applied Imagery Pattern Recognition (AIPR) 2018 workshop

Accurate extraction of lesion borders is a critical step in analysing dermoscopic skin lesion images. In this paper, we consider the problems of poor contrast and lack of colour calibration which are often encountered when analysing dermoscopy images. Different illumination or different devices will lead to different image colours of the same lesion and hence to difficulties in the segmentation stage. Similarly, low contrast makes accurate border detection difficult. We present an effective approach to improve the performance of lesion segmentation algorithms through a pre-processing step that enhances colour information and image contrast. We combine this enhancement stage with two different segmentation algorithms. One technique relies on analysis of the image background by iterative measurements of non-lesion pixels, while the other technique utilises co-operative neural networks for edge detection. Extensive experimental evaluation is carried out on a dataset of 100 dermoscopy images with known ground truths obtained from three expert dermatologists. The results show that both techniques are capable of providing good segmentation performance and that the colour enhancement step is indeed crucial as demonstrated by comparison with results obtained from the original RGB images.

An accurate, fast and low-cost automated plant diagnosis system has been called for. While several studies utilizing machine learning techniques have been conducted, significant issues remain in most cases where the dataset is not composed of field images and often includes a substantial number of inappropriate labels. In this paper, we propose a practical automated plant diagnosis system. We first build a highly reliable dataset by cultivating plants in a strictly controlled setting. We then develop a robust classifier capable of analyzing a wide variety of field images. We use a total of 9,000 original cucumber field leaf images to identify seven typical viral diseases, Downy mildew and healthy plants including initial symptoms. We also visualize the key regions of diagnostic evidence. Our system attains 93.6% average accuracy, and we confirm that our system captures important features for the diagnosis of Downy mildew.

With the breakthrough of deep learning techniques, many leaf-based automated plant diagnosis methodologies have been proposed. To the best of our knowledge, most conventional methodologies only accept narrow range images, typically one or quite a limited number of targets are in their input. This is because the appearance of leaves is diverse and leaves usually heavily overlap each other in practical situations. In this paper, we propose a basic and practical end-to-end plant disease diagnosis system for wide-angle images. Our system is principally composed of two specially designed types of convolutional neural networks. The system achieves leaf detection performance of 73.9% in F1-score, overall (detection and diagnosis) performance of 68.1% in recall and 65.8% in precision at around 3 seconds/image on 500 wide-angle on-site images which have 6,860 healthy and 6,741 infected leaves (13,601 in total).

Dermoscopy is a non-invasive skin imaging technique which permits visualization of features of pigmented melanocytic neoplasms that are not discernable by examination with the naked eye. One of the most important features for the diagnosis of melanoma in dermoscopy images is the blue-white veil (irregualar, strucuteless areas of confluent blue pigmentation with an overlying white "ground-glass" film). In this article. we present a machine learning approach to the detection of blue-white veil and related structures in dermoscopy images. The method involves contextual pixel classification using a decision tree classifier. The percentage of blue-white areas detected in a lesion combined with a simple shape descriptor yielded a sensitivity of 69.35% and a specificity of 89.97% on a set of 545 dermoscopy images. The sensitivity rises to 78.20% for detection of blue veil in those cases where it is a primary feature for melanoma recognition. (C) 2008 Elsevier Ltd. All rights reserved.

We describe a fully automated system for the classification of acral volar melanomas. We used a total of 213 acral dermoscopy images (176 nevi and 37 melanomas). Our automatic tumor area extraction algorithm successfully extracted the tumor in 199 cases (169 nevi and 30 melanomas), and we developed a diagnostic classifier using these images. Our linear classifier achieved a sensitivity (SE) of 100%, a specificity (SP) of 95.9%, and an area under the receiver operating characteristic curve (AUC) of 0.993 using a leave-one-out cross-validation strategy (81.1% SE, 92.1% SP; considering 14 unsuccessful extraction cases as false classification). In addition, we developed three pattern detectors for typical dermoscopic structures such as parallel ridge, parallel furrow, and fibrillar patterns. These also achieved good detection accuracy as indicated by their AUC values: 0.985, 0.931, and 0.890, respectively. The features used in the melanoma-nevus classifier and the parallel ridge detector have significant overlap.

非心臓手術中に発生する心事故リスクの推定を行った．これまで事故推定が難しいとされてきた，中，低リスク手術を含む1351 の手術記録から，何らかの心事故が発生した「全心事故」および，心臓死もしくは心筋梗塞の「hard event」の発生について解析した．解析因子には，手術の難易度，患者の年齢や既往歴などの臨床ファクタおよび，核医学検査結果に分類できる合計22 の項目を用い，予測モデルには線形および，サポートベクターマシン（SVM）識別器を用いた．交差検定法において，全心事故では，感度 80％，特異度 66％，hard event では，感度 85％，特異度 81％の，従来報告されているより良好な推定精度を実現した．線形，SVM識別器双方ともに，核医学検査で得られる共通のパラメータが選択されており，心臓事故の術前推定が難しい，中，低リスク手術においても核医学検査結果が重要な因子であることが確認できた．

Background Myocardial perfusion single-photon emission computed tomography (SPECT) has been used for risk stratification before non-cardiac surgery. However, few authors have used mathematical models for evaluating the likelihood of perioperative cardiac events. Methods and Results This retrospective cohort study collected data of 1,351 patients referred for SPECT before non-cardiac surgery. We generated binary classifiers using support vector machine (SVM) and conventional linear models for predicting perioperative cardiac events. We used clinical and surgical risk, and SPECT findings as input data, and the occurrence of all and hard cardiac events as output data. The area under the receiver-operating characteristic curve (AUC) was calculated for assessing the prediction accuracy. The AUC values were 0.884 and 0.748 in the SVM and linear models, respectively in predicting all cardiac events with clinical and surgical risk, and SPECT variables. The values were 0.861 (SVM) and 0.677 (linear) when not using SPECT data as input. In hard events, the AUC values were 0.892 (SVM) and 0.864 (linear) with SPECT, and 0.867 (SVM) and 0.768 (linear) without SPECT. Conclusion The SVM was superior to the linear model in risk stratification. We also found an incremental prognostic value of SPECT results over information about clinical and surgical risk. (Circ J 2008; 72: 1829 - 1835)

In this paper a methodological approach to the classification of pigmented skin lesions in dermoscopy images is presented. First, automatic border detection is performed to separate the lesion from the background skin. Shape features are then extracted from this border. For the extraction of color and texture related features, the image is divided into various clinically significant regions using the Euclidean distance transform. This feature data is fed into an optimization framework, which ranks the features using various feature selection algorithms and determines the optimal feature subset size according to the area under the ROC curve measure obtained from support vector machine classification. The issue of class imbalance is addressed using various sampling strategies, and the classifier generalization error is estimated using Monte Carlo cross validation. Experiments on a set of 564 images yielded a specificity of 92.34% and a sensitivity of 93.33%.

This paper proposes a new computer-aided method for the skin lesion classification applicable to both melanocytic skin lesions (MSLs) and nonmelanocytic skin lesions (NoMSLs). The computer-aided skin lesion classification has drawn attention as an aid for detection of skin cancers. Several researchers have developed methods to distinguish between melanoma and nevus, which are both categorized as MSL. However, most of these studies did not focus on NoMSLs such as basal cell carcinoma (BCC), the most common skin cancer and seborrheic keratosis (SK) despite their high incidence rates. It is preferable to deal with these NoMSLs as well as MSLs especially for the potential users who are not enough capable of diagnosing pigmented skin lesions on their own such as dermatologists in training and physicians with different expertise. We developed a new method to distinguish among melanomas, nevi, BCCs, and SKs. Our method calculates 828 candidate features grouped into three categories: color, subregion, and texture. We introduced two types of classification models: a layered model that uses a task decomposition strategy and flat models to serve as performance baselines. We tested our methods on 964 dermoscopy images: 105 melanomas, 692 nevi, 69 BCCs, and 98 SKs. The layered model outperformed the flat models, achieving detection rates of 90.48%, 82.51%, 82.61%, and 80.61% for melanomas, nevi, BCCs, and SKs, respectively. We also identified specific features effective for the classification task including irregularity of color distribution. The results show promise for enhancing the capability of the computer-aided skin lesion classification.

Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, automated analysis of dermoscopy images has become an important research area. Border detection is often the first step in this analysis. In many cases, the lesion can be roughly separated from the background skin using a thresholding method applied to the blue channel. However, no single thresholding method appears to be robust enough to successfully handle the wide variety of dermoscopy images encountered in clinical practice. In this paper, we present an automated method for detecting lesion borders in dermoscopy images using ensembles of thresholding methods. Experiments on a difficult set of 90 images demonstrate that the proposed method is robust, fast, and accurate when compared to nine state-of-the-art methods., 8 pages, 3 figures, 2 tables. arXiv admin note: substantial text overlap with arXiv:1009.1362

Ductal carcinoma in situ (DCIS) is a pre-invasive carcinoma of the breast that exhibits several distinct morphologies but the link between morphology and patient outcome is not clear. We hypothesize that different mechanisms of growth may still result in similar 2D morphologies, which may look different in 3D. To elucidate the connection between growth and 3D morphology, we reconstruct the 3D architecture of cribriform DCIS from resected patient material. We produce a fully automated algorithm that aligns, segments, and reconstructs 3D architectures from microscopy images of 2D serial sections from human specimens. The alignment algorithm is based on normalized cross correlation, the segmentation algorithm uses histogram equilization, Otsu's thresholding, and morphology techniques to segment the duct and cribra. The reconstruction method combines these images in 3D. We show that two distinct 3D architectures are indeed found in samples whose 2D histological sections are similarly identified as cribriform DCIS. These differences in architecture support the hypothesis that luminal spaces may form due to different mechanisms, either isolated cell death or merging fronds, leading to the different architectures. We find that out of 15 samples, 6 were found to have ‘bubble-like’ cribra, 6 were found to have ‘tube-like’ criba and 3 were ‘unknown.’ We propose that the 3D architectures found, ‘bubbles’ and ‘tubes’, account for some of the heterogeneity of the disease and may be prognostic indicators of different patient outcomes.

Background: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, computerized analysis of dermoscopy images has become an important research area. One of the most important steps in dermoscopy image analysis is the automated detection of lesion borders. Methods: In this article, we present a systematic overview of the recent border detection methods in the literature paying particular attention to computational issues and evaluation aspects. Conclusion: Common problems with the existing approaches include the acquisition, size, and diagnostic distribution of the test image set, the evaluation of the results, and the inadequate description of the employed methods. Border determination by dermatologists appears to depend upon higher-level knowledge, therefore it is likely that the incorporation of domain knowledge in automated methods will enable them to perform better, especially in sets of images with a variety of diagnoses., 10 pages, 1 figure, 3 tables

Background: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, dermoscopy image analysis has become an important research area. One of the most important steps in dermoscopy image analysis is the automated detection of lesion borders. Although numerous methods have been developed for the detection of lesion borders, very few studies were comprehensive in the evaluation of their results. Methods: In this paper, we evaluate five recent border detection methods on a set of 90 dermoscopy images using three sets of dermatologist-drawn borders as the ground-truth. In contrast to previous work, we utilize an objective measure, the Normalized Probabilistic Rand Index, which takes into account the variations in the ground-truth images. Conclusion: The results demonstrate that the differences between four of the evaluated border detection methods are in fact smaller than those predicted by the commonly used XOR measure.

Accurate color information in dermoscopy images is very important for melanoma diagnosis since inappropriate white balance or brightness in the images adversely affects the diagnostic performance. In this paper, we present an automated color calibration method for dermoscopy images of skin lesions. On a set of 319 dermoscopy images, we develop color calibration filters based on the HSV color system. We determined that the color characteristics of the peripheral part of the tumors have significant influence on the color calibration filters and confirmed that the presented filters achieved satisfactory calibration performance as evaluated by cross-validation. We also confirmed that our method successfully modifies the color distribution of a given image to make it closer to the color distribution of the training image set.

Background: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, automated analysis of dermoscopy images has become an important research area. Border detection is often the first step in this analysis. Methods: In this article, we present an approximate lesion localization method that serves as a preprocessing step for detecting borders in dermoscopy images. In this method, first the black frame around the image is removed using an iterative algorithm. The approximate location of the lesion is then determined using an ensemble of thresholding algorithms. Results: The method is tested on a set of 428 dermoscopy images. The localization error is quantified by a metric that uses dermatologist determined borders as the ground truth. Conclusion: The results demonstrate that the method presented here achieves both fast and accurate localization of lesions in dermoscopy images.

Malignant melanoma has consistently had one of the most rapidly increasing incidence of all cancers, with 59,940 new cases and 8110 deaths estimated in the United States in 2007 (1). Early diagnosis is particularly important because melanoma can be cured with a simple excision if detected early. Dermoscopy is a non-invasive skin imaging technique that uses optical magnification and either liquid immersion and low angle-of-incidence lighting or cross-polarized lighting to make the contact area translucent, making subsurface structures more easily visible when compared with conventional macroscopic (clinical) images. Dermoscopy allows the identification of dozens of morphological features such as pigment networks, dots/globules, streaks, blue-white areas, and blotches (2). This reduces screening errors, and provides greater differentiation between difficult lesions such as pigmented Spitz nevi and small, clinically equivocal lesions (3). However, it has been demonstrated that dermoscopy may actually lower the diagnostic accuracy in the hands of inexperienced dermatologists (4). Therefore, due to the lack of reproducibility and subjectivity of human interpretation, the development of computerized image analysis techniques is of paramount importance (5). The first step in the computerized analysis of skin lesion images is the detection of the lesion borders. The importance of border detection for the analysis is two-fold. First, the border structure provides important information for accurate diagnosis. Many clinical features such as asymmetry, border irregularity, and abrupt border cutoff are calculated from the border. Second, the extraction of other important clinical features such as atypical pigment networks, globules, and blue-white areas critically depends on the accuracy of border detection. Automated border detection in dermoscopy images is a challenging task due to several reasons: (i) low contrast between the lesion and the surrounding skin; (ii) irregular and fuzzy lesion borders; (iii) artifacts such as skin lines, air bubbles and hairs; and (iv) variegated coloring inside the lesion. Numerous methods have been developed for border detection in pigmented skin lesion images earlier; most of these dealt with clinical images (6). However, recent research has focused more on dermoscopy images. Gao et al. (7) proposed two methods: one based on stabilized inverse diffusion equations, a form of non-linear diffusion and another one based on Markov random fields in which the model parameters are estimated using the mean field theory. Pagadala (8) described a method based on optimized histogram thresholding. Schmid (6) developed a technique based on color clustering. First, a 2D histogram is calculated from the first two principal components of the CIE L*u*v* color space. The histogram is then smoothed and initial cluster centers are determined from the peaks using a perceptron classifier. Finally, the lesion image is segmented using a modified version of the fuzzy c-means (FCM) clustering algorithm. Donadey et al. (9) presented a supervised method based on intensity radial profiles calculated from the I (intensity) component of the HSI space. Cucchiara et al. (10) presented a recursive FCM clustering technique that augments Schmid’s method using topological information. Erkol et al. (11) proposed a method based on the gradient vector flow (GVF) snakes with an automatic initialization. Iyatomi et al. (12) described a method called the dermatologist-like tumor extraction algorithm (DTEA) that is based on thresholding followed by iterative region growing. Melli et al. (13) compared four different color clustering algorithms: median cut, k-means, FCM, and meanshift. They concluded that the meanshift algorithm gave the best results. Celebi et al. (14) developed a method based on the JSEG segmentation algorithm. Their method involves an algorithm for approximate lesion localization that reduces the computational time and improves the accuracy by focusing the border detection process on the immediate neighborhood of the lesion rather than the whole image. In this paper, we present an unsupervised approach to border detection in dermoscopy images based on the statistical region merging (SRM) algorithm (15). The SRM algorithm is adapted to this problem due to its simplicity, computational efficiency, and excellent performance in a variety of image domains.

Malignant melanoma has consistently been one of the most rapidly increasing cancers of all, with an estimated incidence of 59,580 and an estimated total of 7770 deaths in the United States in 2005 (1). Early diagnosis is particularly important as melanoma can be cured with a simple excision if detected early. This study uses contact dermoscopy images. Contact dermoscopy is a non-invasive imaging technique using a glass plate and alcohol, gel or oil immersion, which renders the skin translucent, thus allowing a better visualization of the surface and subsurface structures. A systematic review covering Medline entries from 1983 to 1997 revealed that dermoscopy had 10–27% higher sensitivity (2). In a more recent meta-analysis (3), dermoscopy experts achieved a 49% increase in diagnostic accuracy when compared with clinical assessment, with results confirmed by other studies (4). However, it has been demonstrated that dermoscopy may actually lower the diagnostic accuracy in the hands of inexperienced dermatologists (5). Therefore, due to the lack of reproducibility and subjectivity of human interpretation, the development of computerized techniques is of utmost importance (6). The first step in the computerized analysis of skin lesion images is the detection of the lesion borders. The importance of the border detection for the analysis is twofold. First, the border structure provides important information for accurate diagnosis. Many clinical features such as asymmetry, border irregularity, and abrupt border cutoff are calculated from the border. Second, the extraction of other important clinical features such as atypical pigment networks, globules, and blue-white areas critically depends on the accuracy of the border detection. Automated border detection in dermoscopy images is a challenging task due to several reasons: (a) low contrast between the lesion and the surrounding skin, (b) irregular and fuzzy lesion borders, (c) artifacts such as skin texture, air bubbles, and hairs, and (d) variegated coloring inside the lesion. Several methods have been developed for border detection in skin lesion images, with many previous efforts dealing with clinical images (7). Here, we review the border detection methods applied to dermoscopy images. Gao et al. (8) have proposed a method based on a stabilized inverse diffusion equation, a form of non-linear diffusion. Schmid (7) proposed a technique based on color clustering. First, a 2D histogram is calculated from the first two principal components of the CIE L*u*v* color space. Then, the histogram is smoothed and initial cluster centers are determined from the peaks using a perceptron classifier. Finally, the lesion image is segmented using a modified version of the fuzzy c-means clustering algorithm. Donadey et al. (9) presented a supervised method based on intensity radial profiles calculated from the I (intensity) component of the HSI space. Haeghen et al. (10) proposed a very simple supervised method that exploits the CIE L*a*b* color space. In this method, the background skin color is determined from a rough outer border drawn by the user. This initial border is then shrunk as long as the local color is similar to the background skin color. In this paper, we present an unsupervised approach to border detection in dermoscopy images based on a modified version of the JSEG algorithm (11). We adapted the JSEG algorithm to this problem due to its flexibility and good performance in a variety of domains such as natural scenery (11), colonoscopy images (12), tongue images (13), etc.

In this paper, we present an Internet-based melanoma screening system. Our web server is accessible from all over the world and performs the following procedures when a remote user uploads a dermoscopy image: separates the tumor area from the surrounding skin using highly accurate dermatologist-like tumor area extraction algorithm, calculates a total of 428 features for the characterization of the tumor, classifies the tumor as melanoma or nevus using a neural network classifier, and presents the diagnosis. Our system achieves a sensitivity of 85.9% and a specificity of 86.0% on a set of 1258 dermoscopy images using cross-validation.