Search

En aquesta tesi ens centrem en la recuperació i re-identificació d'imatges. L'entrenament de xarxes neuronals profundes usant funcions de pèrdua basades en rànquing ha esdevingut un estàndard de facto per a les tasques de recuperació i re-identificació. Hi analitzem i aportem propostes de respostes a tres qüestions principals: 1) Quines són les estratègies més rellevants dels mètodes de l'estat de l'art i com es poden combinar per obtenir un millor rendiment? 2) Es pot realitzar un mostreig de mostres negatives restrictiu de manera eficient (O(1)) mentre es proporciona un rendiment millorat respecte al mostreig aleatori simple? 3) Es poden aconseguir objectius de reconeixement i recuperació mitjançant una funció de pèrdua basada en el reconeixement? En primer lloc, en el capítol 4 analitzem la importància d'algunes estratègies de l'estat de l'art relacionades amb la formació d'un model d'aprenentatge profund que abasta l'augment d'imatges, l'arquitectura vertebral i la mineria de tripletes restrictives. A continuació, combinem les millors estratègies per dissenyar una arquitectura profunda senzilla, a més d'una metodologia d'entrenament per a una identificació de persones efectiva i d'alta qualitat. Avaluem àmpliament cada opció de disseny, donant lloc a una llista de bones pràctiques per a la re-identificació de persones. Seguint aquestes pràctiques, el nostre enfocament supera l'estat de l'art, inclosos mètodes més complexos amb components auxiliars, de forma amplia en quatre conjunts de dades de referència. També proporcionem una anàlisi qualitativa de la nostra representació entrenada que indica que, tot i ser compacta, és capaç de captar informació de regions focalitzades i discriminatives, d'una manera semblant a un mecanisme d'atenció implícita. En segon lloc, al capítol 5 abordem el problema del mostreig de mostres negatives restrictiu quan s'entrena un model amb funcions del tipus pèrdua per tripletes. En aquest capítol presentem"Bag of Negatives (BoN)", un, En esta tesis nos centramos en la recuperación y re-identificación de imágenes. El entrenamiento de redes neuronales profundas usando funciones de pérdida basadas en ranking se ha convertido en un estándar de facto para las tareas de recuperación y re-identificación. Analizamos y aportamos propuestas de respuestas a tres cuestiones principales: 1) ¿Cuáles son las estrategias más relevantes de los métodos del estado del arte y cómo se pueden combinar para obtener un mejor rendimiento? 2) ¿Se puede realizar unmuestreo de muestras negativas restrictivo de manera eficiente (O(1)) mientras se proporciona un rendimiento mejorado respecto almuestreo aleatorio simple? 3) ¿Se pueden conseguir objetivos de reconocimiento y recuperación mediante una función de pérdida basada en el reconocimiento? En primer lugar, en el capítulo 4 analizamos la importancia de algunas estrategias del estado del arte relacionadas con la formación de un modelo de aprendizaje profundo que abarca el aumento de imágenes, la arquitectura vertebral y la minería de tripletas restrictivas. A continuación, combinamos las mejores estrategias para diseñar una arquitectura profunda sencilla, además de una metodología de entrenamiento para una identificación de personas efectiva y de alta calidad. Evaluamos ampliamente cada opción de diseño, dando lugar a una lista de buenas prácticas para la re-identificación de personas. Siguiendo estas prácticas, nuestro enfoque supera el estado del arte, incluidos métodos más complejos con componentes auxiliares, de forma amplia en cuatro conjuntos de datos de referencia. También proporcionamos un análisis cualitativo de nuestra representación entrenada que indica que, a pesar de ser compacta, es capaz de captar información de regiones focalizadas y discriminativas, de una manera similar a un mecanismo de atención implícita. En segundo lugar, el capítulo 5 abordamos el problema del muestreo demuestras negativas restrictivo cuando se entrena un modelo con funciones del tip, In this thesis we focus on image retrieval and re-identification. Training a deep architecture using a ranking loss has become standard for the retrieval and re-identification tasks. We analyze and propose answers on three main issues: 1) What are the most relevant strategies of state-of-the-art methods and how can they be combined in order to obtain a better performance? 2) Can hard negative sampling be performed efficiently (O(1)) while providing improved performance over naïve random sampling? 3) Can recognition and retrieval objectives be achieved by using a recognition-based loss? First, in chapter 4 we analyze the importance of some state of the art strategies related to the training of a deep model such as image augmentation, backbone architecture and hard triplet mining. We then combine the best strategies to design a simple deep architecture plus a training methodology for effective and high quality person re-identification. We extensively evaluate each design choice, leading to a list of good practices for person re-identification. By following these practices, our approach outperforms the state of the art, including more complex methods with auxiliary components, by large margins on four benchmark datasets. We also provide a qualitative analysis of our trained representation which indicates that, while compact, it is able to capture information from localized and discriminative regions, in a manner akin to an implicit attention mechanism. Second, in chapter 5 we address the problem of hard negative sampling when training a model with triplet-like loss. In this chapter we present Bag of Negatives (BoN), a fast hard negative mining method, that provides a set, triplet or pair of potentially relevant training samples. BoN is an efficient method that selects a bag of hard negatives based on a novel online hashing strategy. We show the superiority of BoN against state-of-the-art hard negative mining methods in terms of accuracy and training time over three lar, Universitat Autònoma de Barcelona. Programa de Doctorat en Informàtica

Training a Siamese architecture for re-identification with a large number of identities is a challenging task due to the difficulty of finding relevant negative samples efficiently. In this work we present Bag of Negatives (BoN), a method for accelerated and improved training of Siamese networks that scales well on datasets with a very large number of identities. BoN is an efficient and loss-independent method, able to select a bag of high quality negatives, based on a novel online hashing strategy., Comment: accepted for BMVC2019

In this paper, we discuss the scientific outcomes of the 2015 data fusion contest organized by the Image Analysis and Data Fusion Technical Committee (IADF TC) of the IEEE Geoscience and Remote Sensing Society (IEEE GRSS). As for previous years, the IADF TC organized a data fusion contest aiming at fostering new ideas and solutions for multisource studies. The 2015 edition of the contest proposed a multiresolution and multisensorial challenge involving extremely high-resolution RGB images and a three-dimensional (3-D) LiDAR point cloud. The competition was framed in two parallel tracks, considering 2-D and 3-D products, respectively. In this paper, we discuss the scientific results obtained by the winners of the 2-D contest, which studied either the complementarity of RGB and LiDAR with deep neural networks (winning team) or provided a comprehensive benchmarking evaluation of new classification strategies for extremely high-resolution multimodal data (runner-up team). The data and the previously undisclosed ground truth will remain available for the community and can be obtained at http://www.grss-ieee.org/community/technicalcommittees/data-fusion/2015-ieee-grss-data-fusion-contest/. The 3-D part of the contest is discussed in the Part-B paper [1].

This paper introduces the use of single layer and deep convolutional networks for remote sensing data analysis. Direct application to multi- and hyper-spectral imagery of supervised (shallow or deep) convolutional networks is very challenging given the high input data dimensionality and the relatively small amount of available labeled data. Therefore, we propose the use of greedy layer-wise unsupervised pre-training coupled with a highly efficient algorithm for unsupervised learning of sparse features. The algorithm is rooted on sparse representations and enforces both population and lifetime sparsity of the extracted features, simultaneously. We successfully illustrate the expressive power of the extracted representations in several scenarios: classification of aerial scenes, as well as land-use classification in very high resolution (VHR), or land-cover classification from multi- and hyper-spectral images. The proposed algorithm clearly outperforms standard Principal Component Analysis (PCA) and its kernel counterpart (kPCA), as well as current state-of-the-art algorithms of aerial classification, while being extremely computationally efficient at learning representations of data. Results show that single layer convolutional networks can extract powerful discriminative features only when the receptive field accounts for neighboring pixels, and are preferred when the classification requires high resolution and detailed results. However, deep architectures significantly outperform single layers variants, capturing increasing levels of abstraction and complexity throughout the feature hierarchy.

Person description is a challenging problem in computer vision. We investigated two major aspects of person description: 1) gender and 2) action recognition in still images. Most state-of-the-art approaches for gender and action recognition rely on the description of a single body part, such as face or full-body. However, relying on a single body part is suboptimal due to significant variations in scale, viewpoint, and pose in real-world images. This paper proposes a semantic pyramid approach for pose normalization. Our approach is fully automatic and based on combining information from full-body, upper-body, and face regions for gender and action recognition in still images. The proposed approach does not require any annotations for upper-body and face of a person. Instead, we rely on pretrained state-of-the-art upper-body and face detectors to automatically extract semantic information of a person. Given multiple bounding boxes from each body part detector, we then propose a simple method to select the best candidate bounding box, which is used for feature extraction. Finally, the extracted features from the full-body, upper-body, and face regions are combined into a single representation for classification. To validate the proposed approach for gender recognition, experiments are performed on three large data sets namely: 1) human attribute; 2) head-shoulder; and 3) proxemics. For action recognition, we perform experiments on four data sets most used for benchmarking action recognition in still images: 1) Sports; 2) Willow; 3) PASCAL VOC 2010; and 4) Stanford-40. Our experiments clearly demonstrate that the proposed approach, despite its simplicity, outperforms state-of-the-art methods for gender and action recognition., Funding Agencies|Swedish Foundation for Strategic Research through the Collaborative Unmanned Aircraft Systems Project; Swedish Research Council through the ETT Project; Strategic Area for Information and Communication Technology research ELLIIT; CADICS; Academy of Finland, through the Finnish Centre of Excellence in Computational Inference Research [251170]; Ministerio de Ciencia e Innovacion through the Ramon y Cajal Fellowship

Person description is a challenging problem in computer vision. We investigated two major aspects of person description: 1) gender and 2) action recognition in still images. Most state-of-the-art approaches for gender and action recognition rely on the description of a single body part, such as face or full-body. However, relying on a single body part is suboptimal due to significant variations in scale, viewpoint, and pose in real-world images. This paper proposes a semantic pyramid approach for pose normalization. Our approach is fully automatic and based on combining information from full-body, upper-body, and face regions for gender and action recognition in still images. The proposed approach does not require any annotations for upper-body and face of a person. Instead, we rely on pretrained state-of-the-art upper-body and face detectors to automatically extract semantic information of a person. Given multiple bounding boxes from each body part detector, we then propose a simple method to select the best candidate bounding box, which is used for feature extraction. Finally, the extracted features from the full-body, upper-body, and face regions are combined into a single representation for classification. To validate the proposed approach for gender recognition, experiments are performed on three large data sets namely: 1) human attribute; 2) head-shoulder; and 3) proxemics. For action recognition, we perform experiments on four data sets most used for benchmarking action recognition in still images: 1) Sports; 2) Willow; 3) PASCAL VOC 2010; and 4) Stanford-40. Our experiments clearly demonstrate that the proposed approach, despite its simplicity, outperforms state-of-the-art methods for gender and action recognition., Funding Agencies|Swedish Foundation for Strategic Research through the Collaborative Unmanned Aircraft Systems Project; Swedish Research Council through the ETT Project; Strategic Area for Information and Communication Technology research ELLIIT; CADICS; Academy of Finland, through the Finnish Centre of Excellence in Computational Inference Research [251170]; Ministerio de Ciencia e Innovacion through the Ramon y Cajal Fellowship

Person description is a challenging problem in computer vision. We investigated two major aspects of person description: 1) gender and 2) action recognition in still images. Most state-of-the-art approaches for gender and action recognition rely on the description of a single body part, such as face or full-body. However, relying on a single body part is suboptimal due to significant variations in scale, viewpoint, and pose in real-world images. This paper proposes a semantic pyramid approach for pose normalization. Our approach is fully automatic and based on combining information from full-body, upper-body, and face regions for gender and action recognition in still images. The proposed approach does not require any annotations for upper-body and face of a person. Instead, we rely on pretrained state-of-the-art upper-body and face detectors to automatically extract semantic information of a person. Given multiple bounding boxes from each body part detector, we then propose a simple method to select the best candidate bounding box, which is used for feature extraction. Finally, the extracted features from the full-body, upper-body, and face regions are combined into a single representation for classification. To validate the proposed approach for gender recognition, experiments are performed on three large data sets namely: 1) human attribute; 2) head-shoulder; and 3) proxemics. For action recognition, we perform experiments on four data sets most used for benchmarking action recognition in still images: 1) Sports; 2) Willow; 3) PASCAL VOC 2010; and 4) Stanford-40. Our experiments clearly demonstrate that the proposed approach, despite its simplicity, outperforms state-of-the-art methods for gender and action recognition., Funding Agencies|Swedish Foundation for Strategic Research through the Collaborative Unmanned Aircraft Systems Project; Swedish Research Council through the ETT Project; Strategic Area for Information and Communication Technology research ELLIIT; CADICS; Academy of Finland, through the Finnish Centre of Excellence in Computational Inference Research [251170]; Ministerio de Ciencia e Innovacion through the Ramon y Cajal Fellowship

Person description is a challenging problem in computer vision. We investigated two major aspects of person description: 1) gender and 2) action recognition in still images. Most state-of-the-art approaches for gender and action recognition rely on the description of a single body part, such as face or full-body. However, relying on a single body part is suboptimal due to significant variations in scale, viewpoint, and pose in real-world images. This paper proposes a semantic pyramid approach for pose normalization. Our approach is fully automatic and based on combining information from full-body, upper-body, and face regions for gender and action recognition in still images. The proposed approach does not require any annotations for upper-body and face of a person. Instead, we rely on pretrained state-of-the-art upper-body and face detectors to automatically extract semantic information of a person. Given multiple bounding boxes from each body part detector, we then propose a simple method to select the best candidate bounding box, which is used for feature extraction. Finally, the extracted features from the full-body, upper-body, and face regions are combined into a single representation for classification. To validate the proposed approach for gender recognition, experiments are performed on three large data sets namely: 1) human attribute; 2) head-shoulder; and 3) proxemics. For action recognition, we perform experiments on four data sets most used for benchmarking action recognition in still images: 1) Sports; 2) Willow; 3) PASCAL VOC 2010; and 4) Stanford-40. Our experiments clearly demonstrate that the proposed approach, despite its simplicity, outperforms state-of-the-art methods for gender and action recognition., Funding Agencies|Swedish Foundation for Strategic Research through the Collaborative Unmanned Aircraft Systems Project; Swedish Research Council through the ETT Project; Strategic Area for Information and Communication Technology research ELLIIT; CADICS; Academy of Finland, through the Finnish Centre of Excellence in Computational Inference Research [251170]; Ministerio de Ciencia e Innovacion through the Ramon y Cajal Fellowship

Person description is a challenging problem in computer vision. We investigated two major aspects of person description: 1) gender and 2) action recognition in still images. Most state-of-the-art approaches for gender and action recognition rely on the description of a single body part, such as face or full-body. However, relying on a single body part is suboptimal due to significant variations in scale, viewpoint, and pose in real-world images. This paper proposes a semantic pyramid approach for pose normalization. Our approach is fully automatic and based on combining information from full-body, upper-body, and face regions for gender and action recognition in still images. The proposed approach does not require any annotations for upper-body and face of a person. Instead, we rely on pretrained state-of-the-art upper-body and face detectors to automatically extract semantic information of a person. Given multiple bounding boxes from each body part detector, we then propose a simple method to select the best candidate bounding box, which is used for feature extraction. Finally, the extracted features from the full-body, upper-body, and face regions are combined into a single representation for classification. To validate the proposed approach for gender recognition, experiments are performed on three large data sets namely: 1) human attribute; 2) head-shoulder; and 3) proxemics. For action recognition, we perform experiments on four data sets most used for benchmarking action recognition in still images: 1) Sports; 2) Willow; 3) PASCAL VOC 2010; and 4) Stanford-40. Our experiments clearly demonstrate that the proposed approach, despite its simplicity, outperforms state-of-the-art methods for gender and action recognition., Funding Agencies|Swedish Foundation for Strategic Research through the Collaborative Unmanned Aircraft Systems Project; Swedish Research Council through the ETT Project; Strategic Area for Information and Communication Technology research ELLIIT; CADICS; Academy of Finland, through the Finnish Centre of Excellence in Computational Inference Research [251170]; Ministerio de Ciencia e Innovacion through the Ramon y Cajal Fellowship

While depth tends to improve network performances, it also makes gradient-based training more difficult since deeper networks tend to be more non-linear. The recently proposed knowledge distillation approach is aimed at obtaining small and fast-to-execute models, and it has shown that a student network could imitate the soft output of a larger teacher network or ensemble of networks. In this paper, we extend this idea to allow the training of a student that is deeper and thinner than the teacher, using not only the outputs but also the intermediate representations learned by the teacher as hints to improve the training process and final performance of the student. Because the student intermediate hidden layer will generally be smaller than the teacher's intermediate hidden layer, additional parameters are introduced to map the student hidden layer to the prediction of the teacher hidden layer. This allows one to train deeper students that can generalize better or run faster, a trade-off that is controlled by the chosen student capacity. For example, on CIFAR-10, a deep student network with almost 10.4 times less parameters outperforms a larger, state-of-the-art teacher network.

We propose a meta-parameter free, off-the-shelf, simple and fast unsupervised feature learning algorithm, which exploits a new way of optimizing for sparsity. Experiments on STL-10 show that the method presents state-of-the-art performance and provides discriminative features that generalize well.

X-ray angiography is the most common imaging modality employed in the diagnosis of coronary diseases prior or during a catheter-based intervention. The analysis of the patient X-Ray sequence can provide useful information about the degree of arterial stenosis, the myocardial perfusion and other clinical parameters. If the sequence has been acquired to evaluate the perfusion grade, the opacity due to the diaphragm could potentially hinder any kind of visual inspection and make more difficult a computer aided measurements. In this thesis we propose an accurate and robust method to automatically identify the diaphragm border in each frame. Quantitative evaluation on a set of 11 sequences shows that the proposed algorithm outperforms previous methods.

This work has been supported in part by the projects La Marató de TV3 082131, TIN2009-14404-C02, and CONSOLIDER-INGENIO CSD 2007-00018., The segmentation of tubular structures is still an open eld of investigation, particularly in medical imaging, where the quality of the image is poor with respect to natural images. Despite the quality of state-of-the-art segmentation methods, little effort has been devoted to the computational effi ciency of the algorithms. E fficiency is an important topic, since intra-operative computer assisted interventions require near real-time performance. In this master thesis, we present a simple, yet effective, algorithm that e fficiently segments vessels in 2D images and 3D volumes. The algorithm requires no initialization and has a computational cost of O(SN logN), where S is the number of scales and N is the number of image pixels. Results on the DRIVE dataset show that the proposed method has near state-of-theart performance with very little computational burden in the 2-dimensional case. Qualitative results on the Rotterdam Coronary Artery dataset show that the method is easily extendable to 3-dimensions.

We present a novel approach to simultaneously reconstruct the 3D structure of a non-rigid coronary tree and estimate point correspondences between an input X-ray image and a reference 3D shape. At the core of our approach lies an optimization scheme that iteratively fits a generative 3D model of increasing complexity and guides the matching process. As a result, and in contrast to existing approaches that assume rigidity or quasi-rigidity of the structure, our method is able to retrieve large non-linear deformations even when the input data is corrupted by the presence of noise and partial occlusions. We extensively evaluate our approach under synthetic and real data and demonstrate a remarkable improvement compared to state-of-the-art., Peer Reviewed, Postprint (author’s final draft)

We present a novel approach to simultaneously reconstruct the 3D structure of a non-rigid coronary tree and estimate point correspondences between an input X-ray image and a reference 3D shape. At the core of our approach lies an optimization scheme that iteratively fits a generative 3D model of increasing complexity and guides the matching process. As a result, and in contrast to existing approaches that assume rigidity or quasi-rigidity of the structure, our method is able to retrieve large non-linear deformations even when the input data is corrupted by the presence of noise and partial occlusions. We extensively evaluate our approach under synthetic and real data and demonstrate a remarkable improvement compared to state-of-the-art.