Search

The phenomenon of Adversarial Examples has become one of the most intriguing topics associated to deep learning. The so-called adversarial attacks have the ability to fool deep neural networks with inappreciable perturbations. While the effect is striking, it has been suggested that such carefully selected injected noise does not necessarily appear in real-world scenarios. In contrast to this, some authors have looked for ways to generate adversarial noise in physical scenarios (traffic signs, shirts, etc.), thus showing that attackers can indeed fool the networks. In this paper we go beyond that and show that adversarial examples also appear in the real-world without any attacker or maliciously selected noise involved. We show this by using images from tasks related to microscopy and also general object recognition with the well-known ImageNet dataset. A comparison between these natural and the artificially generated adversarial examples is performed using distance metrics and image quality metrics. We also show that the natural adversarial examples are in fact at a higher distance from the originals that in the case of artificially generated adversarial examples.

Fast automatic handgun detection can be very useful to avoid or mitigate risks in public spaces. Detectors based on deep learning methods have been proposed in the literature to trigger an alarm if a handgun is detected in the image. However, those detectors are solely based on the weapon appearance on the image. In this work, we propose to combine the detector with the individual’s pose information in order to improve overall performance. To this end, a model that integrates grayscale images from the output of the handgun detector and heatmap-like images that represent pose is proposed. The results show an improvement over the original handgun detector. The proposed network provides a maximum improvement of a 17.5% in AP of the proposed combinational model over the baseline handgun detector.

Closed-circuit television (CCTV) systems are essential nowadays to prevent security threats or dangerous situations, in which early detection is crucial. Novel deep learning-based methods have allowed to develop automatic weapon detectors with promising results. However, these approaches are mainly based on visual weapon appearance only. For handguns, body pose may be a useful cue, especially in cases where the gun is barely visible. In this work, a novel method is proposed to combine, in a single architecture, both weapon appearance and human pose information. First, pose keypoints are estimated to extract hand regions and generate binary pose images, which are the model inputs. Then, each input is processed in different subnetworks and combined to produce the handgun bounding box. Results obtained show that the combined model improves the handgun detection state of the art, achieving from 4.23 to 18.9 AP points more than the best previous approach., 17 pages, 18 figures

In an object detection system, the main objective during training is to maintain the detection and false positive rates under acceptable levels when the model is run over the test set. However, this typically translates into an unacceptable rate of false alarms when the system is deployed in a real surveillance scenario. To deal with this situation, which often leads to system shutdown, we propose to add a filter step to discard part of the new false positive detections that are typical of the new scenario. This step consists of a deep autoencoder trained with the false alarm detections generated after running the detector over a period of time in the new scenario. Therefore, this step will be in charge of determining whether the detection is a typical false alarm of that scenario or whether it is something anomalous for the autoencoder and, therefore, a true detection. In order to decide whether a detection must be filtered, three different approaches have been tested. The first one uses the autoencoder reconstruction error measured with the mean squared error to make the decision. The other two use the k-NN (k-nearest neighbors) and one-class SVMs (support vector machines) classifiers trained with the autoencoder vector representation. In addition, a synthetic scenario has been generated with Unreal Engine 4 to test the proposed methods in addition to a dataset with real images. The results obtained show a reduction in the number of false positives between 22.5% and 87.2% and an increase in the system’s precision of 1.2%$$-47$$ - 47 % when the autoencoder is applied.

Deep learning (henceforth DL) has become most powerful machine learning methodology. Under specific circumstances recognition rates even surpass those obtained by humans. Despite this, several works have shown that deep learning produces outputs that are very far from human responses when confronted with the same task. This the case of the so-called “adversarial examples” (henceforth AE). The fact that such implausible misclassifications exist points to a fundamental difference between machine and human learning. This paper focuses on the possible causes of this intriguing phenomenon. We first argue that the error in adversarial examples is caused by high bias, i.e. by regularization that has local negative effects. This idea is supported by our experiments in which the robustness to adversarial examples is measured with respect to the level of fitting to training samples. Higher fitting was associated to higher robustness to adversarial examples. This ties the phenomenon to the trade-off that exists in machine learning between fitting and generalization.

There is a great need to implement preventive mechanisms against shootings and terrorist acts in public spaces with a large influx of people. While surveillance cameras have become common, the need for monitoring 24/7 and real-time response requires automatic detection methods. This paper presents a study based on three convolutional neural network (CNN) models applied to the automatic detection of handguns in video surveillance images. It aims to investigate the reduction of false positives by including pose information associated with the way the handguns are held in the images belonging to the training dataset. The results highlighted the best average precision (96.36%) and recall (97.23%) obtained by RetinaNet fine-tuned with the unfrozen ResNet-50 backbone and the best precision (96.23%) and F1 score values (93.36%) obtained by YOLOv3 when it was trained on the dataset including pose information. This last architecture was the only one that showed a consistent improvement—around 2%—when pose information was expressly considered during training.

One of the most intriguing phenomenons related to deep learning is the so-called adversarial examples. These samples are visually equivalent to normal inputs, undetectable for humans, yet they cause the networks to output wrong results. The phenomenon can be framed as a symmetry/asymmetry problem, whereby inputs to a neural network with a similar/symmetric appearance to regular images, produce an opposite/asymmetric output. Some researchers are focused on developing methods for generating adversarial examples, while others propose defense methods. In parallel, there is a growing interest in characterizing the phenomenon, which is also the focus of this paper. From some well known datasets of common images, like CIFAR-10 and STL-10, a neural network architecture is first trained in a normal regime, where training and validation performances increase, reaching generalization. Additionally, the same architectures and datasets are trained in an overfitting regime, where there is a growing disparity in training and validation performances. The behaviour of these two regimes against adversarial examples is then compared. From the results, we observe greater robustness to adversarial examples in the overfitting regime. We explain this simultaneous loss of generalization and gain in robustness to adversarial examples as another manifestation of the well-known fitting-generalization trade-off.

Introducing efficient automatic violence detection in video surveillance or audiovisual content monitoring systems would greatly facilitate the work of closed-circuit television (CCTV) operators, rating agencies or those in charge of monitoring social network content. In this paper we present a new deep learning architecture, using an adapted version of DenseNet for three dimensions, a multi-head self-attention layer and a bidirectional convolutional long short-term memory (LSTM) module, that allows encoding relevant spatio-temporal features, to determine whether a video is violent or not. Furthermore, an ablation study of the input frames, comparing dense optical flow and adjacent frames subtraction and the influence of the attention layer is carried out, showing that the combination of optical flow and the attention mechanism improves results up to 4.4%. The conducted experiments using four of the most widely used datasets for this problem, matching or exceeding in some cases the results of the state of the art, reducing the number of network parameters needed (4.5 millions), and increasing its efficiency in test accuracy (from 95.6% on the most complex dataset to 100% on the simplest one) and inference time (less than 0.3 s for the longest clips). Finally, to check if the generated model is able to generalize violence, a cross-dataset analysis is performed, which shows the complexity of this approach: using three datasets to train and testing on the remaining one the accuracy drops in the worst case to 70.08% and in the best case to 81.51%, which points to future work oriented towards anomaly detection in new datasets. Ministerio de Economía y Competitividad TIN2017-82113-C2-1-R MInisterio de Economía y Competitividad TIN2017-82113-C2-2-R

Adversarial examples are one of the most intriguing topics in modern deep learning. Imperceptible perturbations to the input can fool robust models. In relation to this problem, attack and defense methods are being developed almost on a daily basis. In parallel, efforts are being made to simply pointing out when an input image is an adversarial example. This can help prevent potential issues, as the failure cases are easily recognizable by humans. The proposal in this work is to study how chaos theory methods can help distinguish adversarial examples from regular images. Our work is based on the assumption that deep networks behave as chaotic systems, and adversarial examples are the main manifestation of it (in the sense that a slight input variation produces a totally different output). In our experiments, we show that the Lyapunov exponents (an established measure of chaoticity), which have been recently proposed for classification of adversarial examples, are not robust to image processing transformations that alter image entropy. Furthermore, we show that entropy can complement Lyapunov exponents in such a way that the discriminating power is significantly enhanced. The proposed method achieves 65% to 100% accuracy detecting adversarials with a wide range of attacks (for example: CW, PGD, Spatial, HopSkip) for the MNIST dataset, with similar results when entropy-changing image processing methods (such as Equalization, Speckle and Gaussian noise) are applied. This is also corroborated with two other datasets, Fashion-MNIST and CIFAR 19. These results indicate that classifiers can enhance their robustness against the adversarial phenomenon, being applied in a wide variety of conditions that potentially matches real world cases and also other threatening scenarios.

The data presented in this article is part of the whole slide imaging (WSI) datasets generated in European project AIDPATH 2 This data is also related to the research paper entitle “Glomerulosclerosis Identification in Whole Slide Images using Semantic Segmentation”, published in Computer Methods and Programs in Biomedicine Journal [1]. In that article, different methods based on deep learning for glomeruli segmentation and their classification into normal and sclerotic glomerulous are presented and discussed. The raw data used is described and provided here. In addition, the detected glomeruli are also provided as individual image files. These data will encourage research on artificial intelligence (AI) methods, create and compare fresh algorithms, and measure their usability in quantitative nephropathology. Keywords: Glomeruli identification, Normal glomerulus, Global sclerotic glomerulus, Whole slide image, Digital pathology

21 pags., 11 figs., 7 tabs., A microscope is an essential tool in biosciences and production quality laboratories for unveiling the secrets of microworlds. This paper describes the development of MicroHikari3D, an affordable DIY optical microscopy platform with automated sample positioning, autofocus and several illumination modalities to provide a high-quality flexible microscopy tool for labs with a short budget. This proposed optical microscope design aims to achieve high customization capabilities to allow whole 2D slide imaging and observation of 3D live specimens. The MicroHikari3D motion control system is based on the entry level 3D printer kit Tronxy X1 controlled from a server running in a Raspberry Pi 4. The server provides services to a client mobile app for video/image acquisition, processing, and a high level classification task by applying deep learning models., Junta de Comunidades de Castilla-La Mancha (SBPLY/19/180501/000273).

While action recognition has become an important line of research in computer vision, the recognition of particular events such as aggressive behaviors, or fights, has been relatively less studied. These tasks may be exceedingly useful in some video surveillance scenarios such as psychiatric centers, prisons or even in personal camera smartphones. Their potential usability has caused a surge of interest in developing fight or violence detectors. The key aspect in this case is efficiency, that is, these methods should be computationally very fast. In this paper, spatio-temporal elastic cuboid trajectories are proposed for fight recognition. This method is based on the use of blob movements to create trajectories that capture and model the different motions that are specific to a fight. The proposed method is robust to the specific shapes and positions of the individuals. Additionally, the standard Hough forests classifier is adapted in order to use it with this descriptor. This method is compared to other nine related methods on four datasets. The results show that the proposed method obtains the best accuracy for each dataset and is also computationally efficient.

This chapter presents the most relevant image processing techniques and algorithms related to computing features that are able to characterize diatoms as objects in the computer vision field for further analysis and classification. For this purpose, a wide revision of the most important contributions to diatom classification is performed. Moreover, features that have been found to be suitable for this task are covered. Later on, the reader will find the main techniques for diatom classification for the two paradigms that are used nowadays: machine learning with classical methods that rely on previously selected features, or deep learning, which learns the features from the images automatically.

15 pags., 21 figs., 4 tabs., Microscopic algae segmentation, specifically of diatoms, is an essential procedure for water quality assessment. The segmentation of these microalgae is still a challenge for computer vision. This paper addresses for the first time this problem using deep learning approaches to predict exactly those pixels that belong to each class, i.e., diatom and non diatom. A comparison between semantic segmentation and instance segmentation is carried out, and the performance of these methods is evaluated in the presence of different types of noise. The trained models are then evaluated with the same raw images used for manual diatom identification. A total of 126 images of the entire field of view at 60x magnification, with a size of 2592x1944 pixels, are analyzed. The images contain 10 different taxa plus debris and fragments. The best results were obtained with instance segmentation achieving an average precision of 85% with 86% sensitivity and 91% specificity (up to 92% precision with 98%, both sensitivity and specificity for some taxa). Semantic segmentation was able to improve the average sensitivity up to 95% but decreasing the specificity down to 60% and precision to 57%. Instance segmentation was also able to properly separate diatoms when overlap occurs, which helps estimate the number of diatoms, a key requirement for water quality grading., The authors received financial support from the Spanish Government under the Aqualitas-retos project (Ref. CTM2014-51907-C2-2-R-MINECO).

IbPRIA 2019,Madrid, Spain. July 1-4 2019. -- http://www.ibpria.org/2019/. -- Part of the Lecture Notes in Computer Science book series (LNCS, volume 11867) Also part of the Image Processing, Computer Vision, Pattern Recognition, and Graphics book sub series (LNIP, volume 11867), Diatom identification is a crucial process to estimate water quality, which is essential in biological studies. This process can be auto- mated with machine learning algorithms. For this purpose, a dataset with 10 common taxa is collected, with annotations provided by an expert diatomist. In this work, a comparison of the classical state-of-the-art gen- eral purpose methods along with two different deep learning approaches is carried out. The classical methods are based on Viola-Jones and scale and curvature invariant ridge object detectors. The deep learning based methods are Semantic Segmentation and YOLO. This is the first time that Viola-Jones and Semantic Segmentation techniques are applied and compared for diatom segmentation in microscopic images containing sev- eral taxon shells. While all methods provide relatively good results in specific species, the deep learning approaches are consistently better in terms of sensitivity and specificity (up to 0.99 for some taxa) and up to 0.86 precision., The authors acknowledge financial support of the Spanish Government under the Aqualitas-retos project (Ref. CTM2014-51907-C2-R-MINECO).

In recent scientific literature, some studies have been published where recognition rates obtained with Deep Learning (DL) surpass those obtained by humans on the same task. In contrast to this, other studies have shown that DL networks have a somewhat strange behavior which is very different from human responses when confronted with the same task. The case of the so-called “adversarial examples” is perhaps the best example in this regard. Despite the biological plausibility of neural networks, the fact that they can produce such implausible misclassifications still points to a fundamental difference between human and machine learning. This paper delves into the possible causes of this intriguing phenomenon. We first contend that, if adversarial examples are pointing to an implausibility it is because our perception of them relies on our capability to recognise the classes of the images. For this reason we focus on what we call cognitively adversarial examples, which are those obtained from samples that the classifier can in fact recognise correctly. Additionally, in this paper we argue that the phenomenon of adversarial examples is rooted in the inescapable trade-off that exists in machine learning (including DL) between fitting and generalization. This hypothesis is supported by experiments carried out in which the robustness to adversarial examples is measured with respect to the degree of fitting to the training samples.

Public safety in public areas is nowadays one of the main concerns for governments and companies around the world. Video surveillance systems can take advantage from the emerging techniques of deep learning to improve their performance and accuracy detecting possible threats. This paper presents a system for gun and knife detection based on the Faster R-CNN methodology. Two approaches have been compared taking as CNN base a GoogleNet and a SqueezeNet architecture respectively. The best result for gun detection was obtained using a SqueezeNet architecture achieving a 85.44% \(AP_{50}\). For knife detection, the GoogleNet approach achieved a 46.68% \(AP_{50}\). Both results improve upon previous literature results evidencing the effectiveness of our detectors.

The development of object detection systems is normally driven to achieve both high detection and low false positive rates in a certain public dataset. However, when put into a real scenario the result is generally an unacceptable rate of false alarms. In this context we propose to add an additional step that models and filters the typical false alarms of the new scenario while roughly maintaining the ability to detect the objects of interest. We propose to use the false alarms of the new scenario to train a deep autoencoder and to model them. The latter will act as a filter that checks whether the output of the detector is one of its typical false positives or not based on the reconstruction error measured with the Mean Squared Error (MSE) and the Peak Signal-to-Noise Ratio (PSNR). We test the system using an entirely synthetic novel dataset for training and testing the autoencoder generated with Unreal Engine 4. Results show a reduction in the number of FPs of up to 37.9% in combination with the PSNR error while maintaining the same detection capability.

[Abstract] Breast biopsies are crucial in the process of detec ing a wide range of diseases such as breast cancer. The evaluation of these biopsies is performed by trained pathologists that are often overworked due to the increasing number of pathologies requested. Automatic tumour detection techniques have been developed, achieving very good results. In this work, we propose to classify breast biopsies in all the different types of tissue present in them. The tissue types were identified by hand-labeling them following the indications of an expert pathologist. Afterward, they were trained with diffeerent convolutional neural networks such as GoogleNet, AlexNet, SqueezeNet and DenseNet. Out of these four networks, GoogleNet outperformed all of them achieving 95.4% of accuracy. Finally, we tried to identify why the networks were underperforming while also suggesting how results could be improved. AIDPATH EC; 612471

Background and Objective: Glomeruli identification, i.e., detection and characterization, is a key procedure in many nephropathology studies. In this paper, semantic segmentation based on convolutional neural networks (CNN) is proposed to detect glomeruli using Whole Slide Imaging (WSI) follows by a classification CNN to divide the glomeruli into normal and sclerosed. Methods: Comparison between U-Net and SegNet CNNs is performed for pixel-level segmentation considering both a two and three class problem, that is, a) non-glomerular and glomerular structures and b) non-glomerular normal glomerular and sclerotic structures. The two class semantic segmentation result is then used for a CNN classification where glomerular regions are divided into normal and global sclerosed glomeruli. Results: These methods were tested on a dataset composed of 47 WSIs belonging to human kidney sections stained with Periodic Acid Schiff (PAS). The best approach was the SegNet for two class segmentation follows by a fine-tuned AlexNet network to characterize the glomeruli. 98.16% of accuracy was obtained with this process of consecutive CNNs (SegNet-AlexNet) for segmentation and classification. Conclusion: The results obtained demonstrate that the sequential CNN segmentation-classification strategy achieves higher accuracy reducing misclassified cases and therefore being the methodology proposed for glomerulosclerosis detection.

While action recognition has become an important line of research in computer vision, the recognition of particular events, such as aggressive behaviors, or fights, has been relatively less studied. These tasks may be extremely useful in several video surveillance scenarios, such as psychiatric wards, prisons, or even in personal camera smartphones. Their potential usability has led to a surge of interest in developing fight or violence detectors. One of the key aspects in this case is efficiency, that is, these methods should be computationally fast. “Handcrafted” spatio-temporal features that account for both motion and appearance information can achieve high accuracy rates, albeit the computational cost of extracting some of those features is still prohibitive for practical applications. The deep learning paradigm has been recently applied for the first time to this task too, in the form of a 3D convolutional neural network that processes the whole video sequence as input. However, results in human perception of other’s actions suggest that, in this specific task, motion features are crucial. This means that using the whole video as input may add both redundancy and noise in the learning process. In this paper, we propose a hybrid “handcrafted/learned” feature framework which provides better accuracy than the previous feature learning method, with similar computational efficiency. The proposed method is compared to three related benchmark data sets. The method outperforms the different state-of-the-art methods in two of the three considered benchmark data sets.

SPIE Photonics Europe 2018. 22-26 April, Strasbourg, France. ..-https://doi.org/10.1117/12.2309488, Diatom detection has been a challenging task for computer scientist and biologist during past years. In this work, the new state of art techniques based on the deep learning framework have been tested, in order to check whether they are suitable for this purpose. On the one hand, RCNNs (Region based Convolutional Neural Networks), which select candidate regions and applies a convolutional neural network and, on the other hand, YOLO (You Only Look Once), which applies a single neural network over the whole image, have been tested. The first one is able to reach poor results in out experimentation, with an average of 0.68 recall and some tricky aspects, as for example it is needed to apply a bounding box merging algorithm to get stable detections; but the second one gets remarkable results, with an average of 0.84 recall in the evaluation that have been carried out, and less aspects to take into account after the detection has been performed. Future work related to parameter tuning and processing are needed to increase the performance of deep learning in the detection task. However, as for classification it has been probed to provide succesfully performance.

The goal of this challenge 1 was to evaluate new and existing algorithms for automated detection of metastases in hematoxylin and eosin (H&E) stained whole-slide images of lymph node sections. To this end, both slide-based and lesion-based evaluation was made. Several classification methods were tested including classical and deep learning techniques. The most efficient one for the dataset tested was Bagging Tree classifiers using texture features. In the slide-based classification an AUC equal to 0.9952 was obtained, with 98.13% of TP and 1.28% of FP. The TP result decreases in the lesion-based evaluation. Two methodologies were proposed for this second evaluation. Method 1 was based on the convex area of the regions and method 2 based on morphophone- mic, geometric and statistical features. The sensitivity for lesion detection was 39.83% and 36.66% respectively, though the false positive average is kept low about 12.28 in method 1 and 10.71 in method 2.

Advances in digital pathology are generating huge volumes of whole slide (WSI) and tissue microarray images (TMA) which are providing new insights into the causes of cancer. The challenge is to extract and process effectively all the information in order to characterize all the heterogeneous tissue-derived data. This study aims to identify an optimal set of features that best separates different classes in breast TMA. These classes are: stroma, adipose tissue, benign and benign anomalous structures and ductal and lobular carcinomas. To this end, we propose an exhaustive assessment on the utility of textons and colour for automatic classification of breast TMA. Frequential and spatial texton maps from eight different colour models were extracted and compared. Then, in a novel way, the TMA is characterized by the 1st and 2nd order Haralick statistical descriptors obtained from the texton maps with a total of 241 × 8 features for each original RGB image. Subsequently, a feature selection process is performed to remove redundant information and therefore to reduce the dimensionality of the feature vector. Three methods were evaluated: linear discriminant analysis, correlation and sequential forward search. Finally, an extended bank of classifiers composed of six techniques was compared, but only three of them could significantly improve accuracy rates: Fisher, Bagging Trees and AdaBoost. Our results reveal that the combination of different colour models applied to spatial texton maps provides the most efficient representation of the breast TMA. Specifically, we found that the best colour model combination is Hb, Luv and SCT for all classifiers and the classifier that performs best for all colour model combinations is the AdaBoost. On a database comprising 628 TMA images, classification yields an accuracy of 98.1% and a precision of 96.2% with a total of 316 features on spatial textons maps.

14 pags., 15 figs., 3 tabs., We study the effectiveness of several low-cost oblique illumination filters to improve overall image quality, in comparison with standard bright field imaging. For this purpose, a dataset composed of 3360 diatom images belonging to 21 taxa was acquired. Subjective and objective image quality assessments were done. The subjective evaluation was performed by a group of diatom experts by psychophysical test where resolution, focus, and contrast were assessed. Moreover, some objective nonreference image quality metrics were applied to the same image dataset to complete the study, together with the calculation of several texture features to analyze the effect of these filters in terms of textural properties. Both image quality evaluation methods, subjective and objective, showed better results for images acquired using these illumination filters in comparison with the no filtered image. These promising results confirm that this kind of illumination filters can be a practical way to improve the image quality, thanks to the simple and low cost of the design and manufacturing process., The authors acknowledge financial support of the Spanish Government under the Aqualitas-retos project (Ref. CTM2014-51907-C2-2-R-MINECO).

IMPORTANCE: Application of deep learning algorithms to whole-slide pathology imagescan potentially improve diagnostic accuracy and efficiency. OBJECTIVE: Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists' diagnoses in a diagnostic setting. DESIGN, SETTING, AND PARTICIPANTS: Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). EXPOSURES: Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. MAIN OUTCOMES AND MEASURES: The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. RESULTS: The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P

The field of anatomic pathology has experienced major changes over the last decade. Virtual microscopy (VM) systems have allowed experts in pathology and other biomedical areas to work in a safer and more collaborative way. VMs are automated systems capable of digitizing microscopic samples that were traditionally examined one by one. The possibility of having digital copies reduces the risk of damaging original samples, and also makes it easier to distribute copies among other pathologists. This article describes the development of an automated high-resolution whole slide imaging (WSI) system tailored to the needs and problems encountered in digital imaging for pathology, from hardware control to the full digitization of samples. The system has been built with an additional digital monochromatic camera together with the color camera by default and LED transmitted illumination (RGB). Monochrome cameras are the preferred method of acquisition for fluorescence microscopy. The system is able to digitize correctly and form large high resolution microscope images for both brightfield and fluorescence. The quality of the digital images has been quantified using three metrics based on sharpness, contrast and focus. It has been proved on 150 tissue samples of brain autopsies, prostate biopsies and lung cytologies, at five magnifications: 2.5×, 10×, 20×, 40×, and 63×. The article is focused on the hardware set-up and the acquisition software, although results of the implemented image processing techniques included in the software and applied to the different tissue samples are also presented.

This paper describes a specific tool for automatically segmenting and archiving of tissue microarray (TMA) cores in microscopy images at different magnifications. TMA enables researchers to extract the small cylinders of a single tissue (core sections) from histological sections and arrange them in an array on a paraffin block such that hundreds can be analyzed simultaneously. A crucial step to improve the speed and quality of this process is the correct localization of each tissue core in the array. However, usually the tissue cores are not aligned in the microarray, the TMA cores are incomplete and the images are noisy and with distorted colors. We develop a robust framework to handle core sections under these conditions. The algorithms are able to detect, stitch, and archive the TMA cores at different magnifications. Once the TMA cores are segmented they are stored in a relational database allowing their processing for further studies of benign-malignant classification. The method was shown to be reliable for handling the TMA cores and therefore enabling further large-scale molecular pathology research.

The Bonseyes EU H2020 collaborative project aims to develop a platform consisting of a Data Marketplace, a Deep Learning Toolbox, and Developer Reference Platforms for organizations wanting to adopt Artificial Intelligence. The project will be focused on using artificial intelligence in low power Internet of Things (IoT) devices ("edge computing"), embedded computing systems, and data center servers ("cloud computing"). It will bring about orders of magnitude improvements in efficiency, performance, reliability, security, and productivity in the design and programming of systems of artificial intelligence that incorporate Smart Cyber-Physical Systems (CPS). In addition, it will solve a causality problem for organizations who lack access to Data and Models. Its open software architecture will facilitate adoption of the whole concept on a wider scale. To evaluate the effectiveness, technical feasibility, and to quantify the real-world improvements in efficiency, security, performance, effort and cost of adding AI to products and services using the Bonseyes platform, four complementary demonstrators will be built. Bonseyes platform capabilities are aimed at being aligned with the European FI-PPP activities and take advantage of its flagship project FIWARE. This paper provides a description of the project motivation, goals and preliminary work.

Immunohistochemical (IHC) biomarkers in breast tissue microarray (TMA) samples are used daily in pathology departments. In recent years, automatic methods to evaluate positive staining have been investigated since they may save time and reduce errors in the diagnosis. These errors are mostly due to subjective evaluation. The aim of this work is to develop a density tool able to automatically quantify the positive brown IHC stain in breast TMA for different biomarkers. To avoid the problem of colour variation and make a robust tool independent of the staining process, several colour standardization methods have been analysed. Four colour standardization methods have been compared against colour model segmentation. The standardization methods have been compared by means of NBS colour distance. The use of colour standardization helps to reduce noise due to stain and histological sample preparation. However, the most reliable and robust results have been obtained by combining the HSV and RGB colour models for segmentation with the HSB channels. The segmentation provides three outputs based on three saturation values for weak, medium and strong staining. Each output image can be combined according to the type of biomarker staining. The results with 12 biomarkers were evaluated and compared to the segmentation and density calculation done by expert pathologists. The Hausdorff distance, sensitivity and specificity have been used to quantitative validate the results. The tests carried out with 8000 TMA images provided an average of 95.94% accuracy applied to the total tissue cylinder area. Colour standardization was used only when the tissue core had blurring and fading stain and the expert could not evaluate them without a pre-processing.

In this work we present a study for assessing and comparing the fidelity of biopsy and cytology images captured with two different devices, that is optical microscopes and scanners, at 40× magnification in bright field. The devices use different ways to magnify images. Microscopes use a set of lenses while scanners capture light through arrays of micro-photoreceptors. The objective is to carry out a quantitative evaluation to discern which of the two devices provides better image quality in terms of contrast, colour and stain. Since there is no unanimous consensus on quality metrics, we will make use of both an objective metric based on perceptual features, together with a subjective psychophysical test as the International Telecommunications Union (ITU) recommends in ITU-R BT.500 for such type of tests. Both techniques indicate a slight preference for the scanner over the microscope in terms of better image quality, considering defocus as the main problem followed by colour distortions. However, the image quality of both devices is suitable for clinical, educational and research purposes. © 2011 Elsevier Ltd., This work has been carried out with the support of COST ActionIC0604, and the research projects DPI2008-06071, TEC2010-20307, TEC2010-09834-E of the Spanish Research Ministry, PI-2010/040 of the FISCAM and PAI08-0283-9663 of JCCM.

Video understanding has attracted significant research attention in recent years, motivated by interest in video surveillance, rich media retrieval and vision-based gesture interfaces. Typical methods focus on analyzing both the appearance and motion of objects in video. However, the apparent motion induced by a moving camera can dominate the observed motion, requiring sophisticated methods for compensating for camera motion without a priori knowledge of scene characteristics. This paper introduces two new methods for global motion compensation that are both significantly faster and more accurate than state of the art approaches. The first employs RANSAC to robustly estimate global scene motion even when the scene contains significant object motion. Unlike typical RANSAC-based motion estimation work, we apply RANSAC not to the motion of tracked features but rather to a number of segments of image projections. The key insight of the second method involves reliably classifying salient points into foreground and background, based upon the entropy of a motion inconsistency measure. Extensive experiments on established datasets demonstrate that the second approach is able to remove camera-based observed motion almost completely while still preserving foreground motion.

Face recognition has been a long standing problem in computer vision. Recently, Histograms of Oriented Gradients (HOGs) have proven to be an effective descriptor for object recognition in general and face recognition in particular. In this paper, we investigate a simple but powerful approach to make robust use of HOG features for face recognition. The three main contributions of this work are: First, in order to compensate for errors in facial feature detection due to occlusions, pose and illumination changes, we propose to extract HOG descriptors from a regular grid. Second, fusion of HOG descriptors at different scales allows to capture important structure for face recognition. Third, we identify the necessity of performing dimensionality reduction to remove noise and make the classification process less prone to overfitting. This is particularly important if HOG features are extracted from overlapping cells. Finally, experimental results on four databases illustrate the benefits of our approach.

This paper describes a method based on an energy minimizing deformable model applied to the 3D biomechanical modeling of a set of organs considered as regions of interest (ROI) for radiotherapy. The initial model consists of a quadratic surface that is deformed to the exact contour of the ROI by means of the physical properties of a mass-spring system. The exact contour of each ROI is first obtained using a geodesic active contour model. The ROI is then parameterized by the vibration modes resulting from the deformation process. Once each structure has been defined, the method provides a 3D global model including the whole set of ROIs. This model allows one to describe statistically the most significant variations among its structures. Statistical ROI variations among a set of patients or through time can be analyzed. Experimental results are presented using the pelvic zone to simulate anatomical variations among structures and its application in radiotherapy treatment planning.

The widespread availability of portable computing power and inexpensive digital cameras are opening up new possibilities for retailers in some markets. One example is in optical shops, where a number of systems exist that facilitate eyeglasses selection. These systems are now more necessary as the market is saturated with an increasingly complex array of lenses, frames, coatings, tints, photochromic and polarizing treatments, etc. Research challenges encompass Computer Vision, Multimedia and Human-Computer Interaction. Cost factors are also of importance for widespread product acceptance. This paper describes a low-cost system that allows the user to visualize different glasses models in live video. The user can also move the glasses to adjust its position on the face. The system, which runs at 9.5 frames/s on general-purpose hardware, has a homeostatic module that keeps image parameters controlled. This is achieved by using a camera with motorized zoom, iris, white balance, etc. This feature can be specially useful in environments with changing illumination and shadows, like in an optical shop. The system also includes a face and eye detection module and a glasses management module.

Organ motion should be taken into account for image-guided fractionated radiotherapy. A deformable segmentation and registration method was developed for inter-and intra-fraction organ motion planning and evaluation.Energy minimizing active models were synthesized for tracking a set of organs delineated by regions of interest (ROI) in radiotherapy treatment. The initial model consists of a surface deformed to match the ROI contour by geometrical properties, following a heat flow model. The deformable segmentation model was tested using a Shepp-Logan head CT simulation, and different quantitative metrics were applied such as ROC analysis, Jaccard index, Dice coefficient and Hausdorff distance.Experimental evaluation of automated versus manual segmentation was done for the cardiac, thoracic and pelvic regions. The method has been quantitatively validated, obtaining an average of 93.3 and 99.2% for the sensitivity and specificity, respectively, 90.79% for the Jaccard index, 95.15% for the Dice coefficient and 0.96% mm for the Hausdorff distance.Model-based deformable segmentation was developed and tested for image-guided radiotherapy treatment planning. The method is efficient, robust and has sufficient accuracy for 2D CT data without markers.

Robotics researchers and cognitive scientists are becoming more and more interested in so-called sociable robots. These machines normally have expressive power (facial features, voice,...) as well as abilities for locating, paying attention to, and addressing people. The design objective is to make robots which are able to sustain natural interactions with people. This capacity falls within the range classed as social intelligence in humans. This position paper argues that the reproduction of social intelligence, as opposed to other types of human ability, may lead to fragile performance, in the sense that tested cases may produce rather different performances to future (untested) cases and situations. This limitation stems from the fact that our social abilities, which appear early in life, are mainly unconscious in origin. This is in contrast with other human abilities that we carry out using conscious effort, and for which we can easily conceive algorithms and representations. This novel perspective is deemed useful for defining the obstacles and limitations of a field that is generating increasing interest. Taking into account the mentioned issues, a development approach suited to the problem is proposed. The use of this approach is demonstrated in the development of CASIMIRO, a robotic head with basic interaction abilities.

This paper describes a face detection system which goes beyond traditional face detection approaches normally designed for still images. The system described in this paper has been designed taking into account the temporal coherence contained in a video stream in order to build a robust detector. Multiple and real-time detection is achieved by means of cue combination. The resulting system builds a feature based model for each detected face, and searches them using the various model information in the next frame. The experiments have been focused on video streams, where our system can actually exploit the benefits of the temporal coherence integration. The results achieved for video stream processing outperform Rowley-Kanade's and Viola-Jones' solutions providing eye and face data in real-time with a notable correct detection rate, approx. 99.9% faces and 87.5% eye pairs on 26338 images.

The Eyes of Things (EoT) EU H2020 project envisages a computer vision platform that can be used both standalone and embedded into more complex artifacts, particularly for wearable applications, robotics, home products, surveillance etc. The core hardware will be based on a Software on Chip (SoC) that has been designed for maximum performance of the always-demanding vision applications while keeping the lowest energy consumption. This will allow "always on" and truly mobile vision processing. This demo presents the first prototype applications developed within EoT. First, example vision processing applications will be shown. Additionally, an RTSP server implemented in the device will be demonstrated. This server can capture and stream images. Finally, connectivity will be shown using a minimal MQTT broker specifically implemented for the device.

In Digital Pathology, one of the most simple and yet most useful feature is the ability to view serial sections of tissue simultaneously on a computer monitor. This enables the pathologist to evaluate the histology and expression of multiple markers for a patient in a single review. However, the rate limiting step in this process is the time taken for the pathologist to open each individual image, align the sections within the viewer, with a maximum of four slides at a time, and then manually move around the section. In addition, due to tissue processing and pre-analytical steps, sections with different stains have non-linear variations between the two acquisitions, that is, they will stretch and change shape from section to section. To date, no solution has come close to a workable solution to automatically align the serial sections into one composite image. This research work address this problem to obtain an automated serial section alignment tool enabling the pathologists to simply scroll through the various sections in a single viewer. To this aim a multi-resolution intensity-based registration method using mutual information as a similarity metric, an optimizer based on an evolutionary process and a bilinear transformation has been used. To characterize the performance of the algorithm 40 cases x 5 different serial sections stained with hematoxiline-eosine (HE), estrogen receptor (ER), progesterone receptor (PR), Ki67 and human epidermal growth factor receptor 2 (Her2), have been considered. The qualitative results obtained are promising, with average computation time of 26.4s for up to 14660x5799 images running interpreted code.

Automatic detection systems usually require large and representative training datasets in order to obtain good detection and false positive rates. Training datasets are such that the positive set has few samples and/or the negative set should represent anything except the object of interest. In this respect, the negative set typically contains orders of magnitude more images than the positive set. However, imbalanced training databases lead to biased classifiers. In this paper, we focus our attention on a negative sample selection method to properly balance the training data for cascade detectors. The method is based on the selection of the most informative false positive samples generated in one stage to feed the next stage. The results show that the proposed cascade detector with sample selection obtains on average better partial AUC and smaller standard deviation than the other compared cascade detectors.

Support vector machines (SVM) and independent component analysis (ICA) are two powerful and relatively recent techniques. SVMs are classifiers which have demonstrated high generalization capabilities in many different tasks, including the object recognition problem. ICA is a feature extraction technique which can be considered a generalization of principal component analysis (PCA). ICA has been mainly used on the problem of blind signal separation. In this paper we combine these two techniques for the face recognition problem. Experiments were made on two different face databases, achieving very high recognition rates. As the results using the combination PCA/SVM were not very far from those obtained with ICA/SVM, our experiments suggest that SVMs are relatively insensitive to the representation space. Thus as the training time for ICA is much larger than that of PCA, this result indicates that the best practical combination is PCA with SVM.

Breast cancer diagnosis is still done by observation of biopsies under the microscope. The development of automated methods for breast TMA classification would reduce diagnostic time. This paper is a step towards the solution for this problem and shows a complete study of breast TMA classification based on colour models and texture descriptors. The TMA images were divided into four classes: i) benign stromal tissue with cellularity, ii) adipose tissue, iii) benign and benign anomalous structures, and iv) ductal and lobular carcinomas. A relevant set of features was obtained on eight different colour models from first and second order Haralick statistical descriptors obtained from the intensity image, Fourier, Wavelets, Multiresolution Gabor, M-LBP and textons descriptors. Furthermore, four types of classification experiments were performed using six different classifiers: (1) classification per colour model individually, (2) classification by combination of colour models, (3) classification by combination of colour models and descriptors, and (4) classification by combination of colour models and descriptors with a previous feature set reduction. The best result shows an average of 99.05% accuracy and 98.34% positive predictive value. These results have been obtained by means of a bagging tree classifier with combination of six colour models and the use of 1719 non-correlated (correlation threshold of 97%) textural features based on Statistical, M-LBP, Gabor and Spatial textons descriptors.

This paper deals with automatic taxa identification based on machine learning methods. The aim is therefore to automatically classify diatoms, in terms of pattern recognition terminology. Diatoms are a kind of algae microorganism with high biodiversity at the species level, which are useful for water quality assessment. The most relevant features for diatom description and classification have been selected using an extensive dataset of 80 taxa with a minimum of 100 samples/taxon augmented to 300 samples/taxon. In addition to published morphological, statistical and textural descriptors, a new textural descriptor, Local Binary Patterns (LBP), to characterize the diatom’s valves, and a log Gabor implementation not tested before for this purpose are introduced in this paper. Results show an overall accuracy of 98.11% using bagging decision trees and combinations of descriptors. Finally, some phycological features of diatoms that are still difficult to integrate in computer systems are discussed for future work., The authors acknowledge financial support of the Spanish Government under the Aqualitas-retos project (Ref. CTM2014-51907-C2-2-R-MINECO), We acknowledge support by the CSIC Open Access Publication Initiative through its Unit of Information Resources for Research (URICI).

Diatoms, a kind of algae microorganisms with several species, are quite useful for water quality determination, one of the hottest topics in applied biology nowadays. At the same time, deep learning and convolutional neural networks (CNN) are becoming an extensively used technique for image classification in a variety of problems. This paper approaches diatom classification with this technique, in order to demonstrate whether it is suitable for solving the classification problem. An extensive dataset was specifically collected (80 types, 100 samples/type) for this study. The dataset covers different illumination conditions and it was computationally augmented to more than 160,000 samples. After that, CNNs were applied over datasets pre-processed with different image processing techniques. An overall accuracy of 99% is obtained for the 80-class problem and different kinds of images (brightfield, normalized). Results were compared to previous presented classification techniques with different number of samples. As far as the authors know, this is the first time that CNNs are applied to diatom classification., We acknowledge support by the CSIC Open Access Publication Initiative through its Unit of Information Resources for Research (URICI)., The authors acknowledge financial support of the Spanish Government under the Aqualitas-retos project (Ref. CTM2014-51907-C2-2-R-MINECO) http://aqualitas-retos.es/en/.

This paper shows a complete and novel study for breast TMA classification based on texture, frequential features and color models information. Thus a relevant set of features for automatic breast TMA classification is found. These features are obtained from 1st and 2nd order Haralick statistical descriptors as well as by filtering with the Fourier and Daubechies Wavelet transforms. Moreover, the discriminant value of the features has been analyzed under different color models, that is, RGB, Lab*, HSV, Hb* and b* channel. Thus, a total of 3133 features with 5 color models were analyzed. The TMA images were divided into four classes: i) benign stromal tissue with cellularity, ii) adipose tissue, iii) benign structures but anomalous and iv) ductal and lobular carcinomas. A statistical study of the features was carried out to provide information about their influence in the tissue classes. Finally, the classification was performed with 10 classifiers using 10-fold and leave-one-out cross validation for all color models selected. Furthermore, the classification results were also analyzed by means of a principal component analysis (PCA). A 95% accuracy and 90% precision with PCA and a Fisher classifier on Hb* image channels is obtained with the selected features.

14 págs.; 10 figs.; 7 tabs.; 1 app., © 2014 Elsevier B.V. Besides the well-established healthy properties of pollen, palynology and apiculture are of extreme importance to avoid hard and fast unbalances in our ecosystems. To support such disciplines computer vision comes to alleviate tedious recognition tasks. In this paper we present an applied study of the state of the art in pattern recognition techniques to describe, analyze, and classify pollen grains in an extensive dataset specifically collected (15 types, 120 samples/type). We also propose a novel contour-inner segmentation of grains, improving 50% of accuracy. In addition to published morphological, statistical, and textural descriptors, we introduce a new descriptor to measure the grain's contour profile and a logGabor implementation not tested before for this purpose. We found a significant improvement for certain combinations of descriptors, providing an overall accuracy above 99%. Finally, some palynological features that are still difficult to be integrated in computer systems are discussed., This work has been supported by the European project APIFRESH FP7-SME-2008-2 ‘‘Developing European standards for bee pollen and royal jelly: quality, safety and authenticity’’ and we would like to thank to Mr. Walter Haefeker, President of the European Professional Beekeepers Association (EPBA). J. Victor Marcos is a ‘‘Juan de la Cierva’’ research fellow funded by the Spanish Ministry of Economy and Competitiveness. Rodrigo Nava thanks Consejo Nacional de Ciencia y Tecnología (CONACYT) and PAPIIT Grant IG100814.