Search

In traditional semantic segmentation, knowing about all existing classes is essential to yield effective results with the majority of existing approaches. However, these methods trained in a Closed Set of classes fail when new classes are found in the test phase, not being able to recognize that an unseen class has been fed. This means that they are not suitable for Open Set scenarios, which are very common in real-world computer vision and remote sensing applications. In this paper, we discuss the limitations of Closed Set segmentation and propose two fully convolutional approaches to effectively address Open Set semantic segmentation: OpenFCN and OpenPCS. OpenFCN is based on the well-known OpenMax algorithm, configuring a new application of this approach in segmentation settings. OpenPCS is a fully novel approach based on feature-space from DNN activations that serve as features for computing PCA and multi-variate gaussian likelihood in a lower dimensional space. In addition to OpenPCS and aiming to reduce the RAM memory requirements of the methodology, we also propose a slight variation of the method (OpenIPCS) that uses an iteractive version of PCA able to be trained in small batches. Experiments were conducted on the well-known ISPRS Vaihingen/Potsdam and the 2018 IEEE GRSS Data Fusion Challenge datasets. OpenFCN showed little-to-no improvement when compared to the simpler and much more time efficient SoftMax thresholding, while being some orders of magnitude slower. OpenPCS achieved promising results in almost all experiments by overcoming both OpenFCN and SoftMax thresholding. OpenPCS is also a reasonable compromise between the runtime performances of the extremely fast SoftMax thresholding and the extremely slow OpenFCN, being able to run close to real-time. Experiments also indicate that OpenPCS is effective, robust and suitable for Open Set segmentation, being able to improve the recognition of unknown class pixels without reducing the accuracy on the known class pixels. We also tested the scenario of hiding multiple known classes to simulate multimodal unknowns, resulting in an even larger gap between OpenPCS/OpenIPCS and both SoftMax thresholding and OpenFCN, implying that gaussian modeling is more robust to settings with greater openness.

Accurately mapping farmlands is important for precision agriculture practices. Unmanned aerial vehicles (UAV) embedded with multispectral cameras are commonly used to map plants in agricultural landscapes. However, separating plantation fields from the remaining objects in a multispectral scene is a difficult task for traditional algorithms. In this connection, deep learning methods that perform semantic segmentation could help improve the overall outcome. In this study, state-of-the-art deep learning methods to semantic segment citrus-trees in multispectral images were evaluated. For this purpose, a multispectral camera that operates at the green (530–570 nm), red (640–680 nm), red-edge (730–740 nm) and also near-infrared (770–810 nm) spectral regions was used. The performance of the following five state-of-the-art pixelwise methods were evaluated: fully convolutional network (FCN), U-Net, SegNet, dynamic dilated convolution network (DDCN) and DeepLabV3 + . The results indicated that the evaluated methods performed similarly in the proposed task, returning F1-Scores between 94.00% (FCN and U-Net) and 94.42% (DDCN). It was also determined the inference time needed per area and, although the DDCN method was slower, based on a qualitative analysis, it performed better in highly shadow-affected areas. This study demonstrated that the semantic segmentation of citrus orchards is highly achievable with deep neural networks. The state-of-the-art deep learning methods investigated here proved to be equally suitable to solve this task, providing fast solutions with inference time varying from 0.98 to 4.36 min per hectare. This approach could be incorporated into similar research, and contribute to decision-making and accurate mapping of plantation fields.

It is undeniable that aerial/satellite images can provide useful information for a large variety of tasks. But, since these images are always taken from above, some applications can benefit from complementary information provided by other perspective views of the scene, such as ground-level images. Despite a large number of public repositories for both georeferenced photographs and aerial images, there is a lack of benchmark datasets that allow the development of approaches that exploit the benefits and complementarity of aerial/ground imagery. In this article, we present two new publicly available datasets named AiRound and CV-BrCT. The first one contains triplets of images from the same geographic coordinate with different perspectives of view extracted from various places around the world. Each triplet is composed of an aerial RGB image, a ground-level perspective image, and a Sentinel-2 sample. The second dataset contains pairs of aerial and street-level images extracted from southeast Brazil. We design an extensive set of experiments concerning multiview scene classification, using early and late fusion. Such experiments were conducted to show that image classification can be enhanced using multiview data.

Made available in DSpace on 2020-12-11T00:48:55Z (GMT). No. of bitstreams: 0 Previous issue date: 2019-10-01 Pro-Reitoria de Pesquisa da UFMG Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG) Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) Cedro Textil, Reserva Vellozia, Parque Nacional da Serra do Cipo Plant phenology studies rely on long-term monitoring of life cycles of plants. High-resolution unmanned aerial vehicles (UAVs) and near-surface technologies have been used for plant monitoring, demanding the creation of methods capable of locating, and identifying plant species through time and space. However, this is a challenging task given the high volume of data, the constant data missing from temporal dataset, the heterogeneity of temporal profiles, the variety of plant visual patterns, and the unclear definition of individuals' boundaries in plant communities. In this letter, we propose a novel method, suitable for phenological monitoring, based on convolutional networks (ConvNets) to perform spatio-temporal vegetation pixel classification on high-resolution images. We conducted a systematic evaluation using high-resolution vegetation image datasets associated with the Brazilian Cerrado biome. Experimental results show that the proposed approach is effective, overcoming other spatio-temporal pixel-classification strategies. Univ Fed Minas Gerais, Dept Comp Sci, BR-31270901 Belo Horizonte, MG, Brazil Univ Estadual Campinas, Inst Comp, BR-13083970 Campinas, SP, Brazil Univ Estadual Paulista, Inst Biociencias, BR-13506900 Sao Paulo, Brazil Univ Estadual Paulista, IGCE, BR-13506900 Sao Paulo, Brazil Univ Stirling, Biol & Environm Sci, Fac Nat Sci, Stirling FK9 4LA, Scotland Univ Estadual Paulista, Inst Biociencias, BR-13506900 Sao Paulo, Brazil Univ Estadual Paulista, IGCE, BR-13506900 Sao Paulo, Brazil FAPEMIG: APQ-00449-17 FAPESP: 2013/50155-0 FAPESP: 2013/50169-1 FAPESP: 2009/54208-6 FAPESP: 2016/26170-8 FAPESP: 2018/06918-3 CNPq: 424700/2018-2 CAPES: 001 CAPES: 88881.145912/2017-01 Cedro Textil, Reserva Vellozia, Parque Nacional da Serra do Cipo: PELD-CRSC-17

Made available in DSpace on 2022-04-29T08:30:21Z (GMT). No. of bitstreams: 0 Previous issue date: 2021-07-01 Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) Urban forests contribute to maintaining livability and increase the resilience of cities in the face of population growth and climate change. Information about the geographical distribution of individual trees is essential for the proper management of these systems. RGB high-resolution aerial images have emerged as a cheap and efficient source of data, although detecting and mapping single trees in an urban environment is a challenging task. Thus, we propose the evaluation of novel methods for single tree crown detection, as most of these methods have not been investigated in remote sensing applications. A total of 21 methods were investigated, including anchor-based (one and two-stage) and anchor-free state-of-the-art deep-learning methods. We used two orthoimages divided into 220 non-overlapping patches of 512 × 512 pixels with a ground sample distance (GSD) of 10 cm. The orthoimages were manually annotated, and 3382 single tree crowns were identified as the ground-truth. Our findings show that the anchor-free detectors achieved the best average performance with an AP50 of 0.686. We observed that the two-stage anchor-based and anchor-free methods showed better performance for this task, emphasizing the FSAF, Double Heads, CARAFE, ATSS, and FoveaBox models. RetinaNet, which is currently commonly applied in remote sensing, did not show satisfactory performance, and Faster R-CNN had lower results than the best methods but with no statistically significant difference. Our findings contribute to a better understanding of the performance of novel deep-learning methods in remote sensing applications and could be used as an indicator of the most suitable methods in such applications. Faculty of Engineering Architecture and Urbanism and Geography Federal University of Mato Grosso do Sul Faculty of Computer Science Federal University of Mato Grosso do Sul Department of Cartography São Paulo State University (UNESP) Computing Science and Mathematics Division Faculty of Natural Sciences University of Stirling Department of Cartography São Paulo State University (UNESP) CNPq: 303559/2019-5 CNPq: 304052/2019-1 CNPq: 433783/2018-4 CAPES: 88881.311850/2018-01

Informacoes podem ser extraida da superficie terrestre atraves de imagens adquiridas por sensores aereos. Mais que isso, hoje em dia, informacoes aprimoradas e detalhadas podem ser extraidas de imagens de alta resolucao espacial, atraves de novas e avancadas tecnologias de sensores. Essa informacao pode auxiliar tarefas de diferentes aplicacoes, como urbanizacao, manejo de culturas e florestas, etc. Embora muito usadas, e complexo minerar as informacoes sobre essas imagens, ja que o primeiro e mais importante passo em direcao a tal objetivo e baseado em codificar as caracteristicas das imagens, uma tarefa desafiadora dadas as propriedades distintas de cada imagem. De fato, independentemente da tarefa, codificar as caracteristicas espaciais de forma eficiente e robusta e fundamental para gerar bons modelos discriminativos. Mesmo que muitos descritores tenham sido propostos e usados com sucesso para codificar caracteristicas espaciais de imagens de sensoriamento remoto, a maioria das aplicacoes exigem tecnicas mais especificas. Aprendizado profundo e capaz de aprender, ao mesmo tempo, caracteristicas e classificadores especificos e ajustando-os em tempo de processamento atendendo as necessidades de cada problema. Tal tecnica conseguiu bons resultados, principalmente impulsionados pelo aprendizado de caracteristicas, que permite ao metodo extrair caracteristicas especificas e adaptaveis dependendo dos dados. Neste projeto, abordamos duas principais tarefas, a classificacao de cenas e de pixels, usando aprendizado profundo para codificar caracteristicas espaciais em imagens de sensoriamento remoto. Inicialmente, enfrentamos o problema de classificacao de cenas, em que o desempenho final do modelo e dependente da qualidade das caracteristicas extraidas. Com base em obras recentemente publicadas e apos um conjunto de experimentos, a primeira contribuicao foi uma arquitetura de rede convolucional, treinada especificamente para sensoriamento remoto. Experimentos mostraram que a arquitetura proposta superou algoritmos de ultima geracao, demonstrando a eficacia de metodos de aprendizado profundo para codificar caracteristicas, mesmo para o dominio de sensoriamento remoto. Em seguida, numa tentativa de amenizar as adversidades encontradas ao explorar redes convolucionais, foram avaliadas diferentes estrategias para melhor aproveitar os beneficios dessas tecnicas no dominio de sensoriamento remoto. Tres estrategias especificas foram avaliadas em tres conjuntos distintos de dados. Experimentos mostraram que treinar a rede a partir de um ponto pre-treinado e ajustar seus filtros para o dominio especifico e a melhor estrategia para explorar a rede convolucional. A segunda tarefa abordada e a classificacao de pixels. Essa tarefa e interessante para a comunidade de sensoriamento remoto, visto que trata-se da criacao automatica de mapas tematicos que, por sua vez, podem ajudar a melhor compreender situacoes, como novas areas urbanas e desmatamento. Considerando tal tarefa, a primeira contribuicao e uma nova arquitetura da rede convolucional para realizar a classificacao de pixels. Essa tecnica e baseada em janelas de contexto, ou seja, pequenas imagens que carregam o contexto dos pixels centralizados e ajudam a entender os padroes espaciais em torno deles. Experimentos mostraram que a abordagem proposta proporciona melhorias significativas quando comparada as outros metodos. Com base nos problemas da tecnica anterior e dos metodos existentes, propusemos uma nova abordagem que emprega uma versao evoluida da rede proposta anteriormente, bem como outras arquiteturas, para executar a classificacao de todos os pixels da imagem de entrada (ao inves de classificar apenas o central). Esta nova tecnica e capaz de agregar informacao multicontexto sem aumentar o numero de parâmetros. Os resultados mostraram que o algoritmo proposto fornece melhorias na classificacao de pixel quando comparado aos metodos de estado da arte. Finalmente, a ultima contribuicao e um novo metodo que, quando comparado a outras abordagens existentes, e capaz de melhor aprender algumas caracteristicas visuais, como bordas e cantos, uteis para algumas aplicacoes e dominios, incluindo o sensoriamento remoto.Este metodo, chamado Rede Morfologica Profunda (DeepMorphNet), e capaz de realizar e otimizar operacoes morfologicas, que podem ser capazes de lidar melhor com algumas propriedades visuais dos objetos. Essa DeepMorphNet pode ser treinada e otimizada de ponta a ponta, utilizando tecnicas tradicionais existentes, comumente empregadas no treinamento de abordagens de aprendizagem profunda, e pode ser usada em diversas aplicacoes, como classificacao de imagens e pixels, deteccao e outros. Uma avaliacao sistematica do algoritmo proposto foi conduzida. Resultados mostraram que a DeepMorphNet e uma tecnica promissora capaz de aprender caracteristicas distintas quando comparadas aquelas aprendidas pelos atuais metodos de aprendizagem profunda.

Soil erosion is considered one of the most expensive natural hazards with a high impact on several infrastructure assets. Among them, railway lines are one of the most likely constructions for the appearance of erosion and, consequently, one of the most troublesome due to the maintenance costs, risks of derailments, and so on. Therefore, it is fundamental to identify and monitor erosion in railway lines to prevent major consequences. Currently, erosion identification is manually performed by humans using huge image sets, a time-consuming and slow task. Hence, automatic machine learning methods appear as an appealing alternative. A crucial step for automatic erosion identification is to create a good feature representation. Towards such objective, deep learning can learn data-driven features and classifiers. In this paper, we propose a novel deep learning-based framework capable of performing erosion identification in railway lines. Six techniques were evaluated and the best one, Dynamic Dilated ConvNet, was integrated into this framework that was then encapsulated into a new ArcGIS plugin to facilitate its use by non-programmer users. To analyze such techniques, we also propose a new dataset, composed of almost 2000 high-resolution images.

Classical and more recently deep computer vision methods are optimized for visible spectrum images, commonly encoded in grayscale or RGB colorspaces acquired from smartphones or cameras. A more uncommon source of images exploited in the remote sensing field are satellite and aerial images. However, the development of pattern recognition approaches for these data is relatively recent, mainly due to the limited availability of this type of images, as until recently they were used exclusively for military purposes. Access to aerial imagery, including spectral information, has been increasing mainly due to the low cost of drones, cheapening of imaging satellite launch costs, and novel public datasets. Usually remote sensing applications employ computer vision techniques strictly modeled for classification tasks in closed set scenarios. However, real-world tasks rarely fit into closed set contexts, frequently presenting previously unknown classes, characterizing them as open set scenarios. Focusing on this problem, this is the first paper to study and develop semantic segmentation techniques for open set scenarios applied to remote sensing images. The main contributions of this paper are: 1) a discussion of related works in open set semantic segmentation, showing evidence that these techniques can be adapted for open set remote sensing tasks; 2) the development and evaluation of a novel approach for open set semantic segmentation. Our method yielded competitive results when compared to closed set methods for the same dataset.

Informações podem ser extraída da superfície terrestre através de imagens adquiridas por sensores aéreos. Mais que isso, hoje em dia, informações aprimoradas e detalhadas podem ser extraídas de imagens de alta resolução espacial, através de novas e avançadas tecnologias de sensores. Essa informação pode auxiliar tarefas de diferentes aplicações, como urbanização, manejo de culturas e florestas, etc. Embora muito usadas, é complexo minerar as informações sobre essas imagens, já que o primeiro e mais importante passo em direção a tal objetivo é baseado em codificar as características das imagens, uma tarefa desafiadora dadas as propriedades distintas de cada imagem. De fato, independentemente da tarefa, codificar as características espaciais de forma eficiente e robusta é fundamental para gerar bons modelos discriminativos. Mesmo que muitos descritores tenham sido propostos e usados com sucesso para codificar características espaciais de imagens de sensoriamento remoto, a maioria das aplicações exigem técnicas mais específicas. Aprendizado profundo é capaz de aprender, ao mesmo tempo, características e classificadores específicos e ajustando-os em tempo de processamento atendendo às necessidades de cada problema. Tal técnica conseguiu bons resultados, principalmente impulsionados pelo aprendizado de características, que permite ao método extrair características específicas e adaptáveis dependendo dos dados. Neste projeto, abordamos duas principais tarefas, a classificação de cenas e de pixels, usando aprendizado profundo para codificar características espaciais em imagens de sensoriamento remoto. Inicialmente, enfrentamos o problema de classificação de cenas, em que o desempenho final do modelo é dependente da qualidade das características extraídas. Com base em obras recentemente publicadas e após um conjunto de experimentos, a primeira contribuição foi uma arquitetura de rede convolucional, treinada especificamente para sensoriamento remoto. Experimentos mostraram que a arquitetura proposta superou algoritmos de última geração, demonstrando a eficácia de métodos de aprendizado profundo para codificar características, mesmo para o domínio de sensoriamento remoto. Em seguida, numa tentativa de amenizar as adversidades encontradas ao explorar redes convolucionais, foram avaliadas diferentes estratégias para melhor aproveitar os benefícios dessas técnicas no domínio de sensoriamento remoto. Três estratégias específicas foram avaliadas em três conjuntos distintos de dados. Experimentos mostraram que treinar a rede a partir de um ponto pré-treinado e ajustar seus filtros para o domínio específico é a melhor estratégia para explorar a rede convolucional. A segunda tarefa abordada é a classificação de pixels. Essa tarefa é interessante para a comunidade de sensoriamento remoto, visto que trata-se da criação automática de mapas temáticos que, por sua vez, podem ajudar a melhor compreender situações, como novas áreas urbanas e desmatamento. Considerando tal tarefa, a primeira contribuição é uma nova arquitetura da rede convolucional para realizar a classificação de pixels. Essa técnica é baseada em janelas de contexto, ou seja, pequenas imagens que carregam o contexto dos pixels centralizados e ajudam a entender os padrões espaciais em torno deles. Experimentos mostraram que a abordagem proposta proporciona melhorias significativas quando comparada as outros métodos. Com base nos problemas da técnica anterior e dos métodos existentes, propusemos uma nova abordagem que emprega uma versão evoluída da rede proposta anteriormente, bem como outras arquiteturas, para executar a classificação de todos os pixels da imagem de entrada (ao invés de classificar apenas o central). Esta nova técnica é capaz de agregar informação multicontexto sem aumentar o número de parâmetros. Os resultados mostraram que o algoritmo proposto fornece melhorias na classificação de pixel quando comparado aos métodos de estado da arte. Finalmente, a última contribuição é um novo método que, quando comparado a outras abordagens existentes, é capaz de melhor aprender algumas características visuais, como bordas e cantos, úteis para algumas aplicações e domínios, incluindo o sensoriamento remoto.Este método, chamado Rede Morfológica Profunda (DeepMorphNet), é capaz de realizar e otimizar operações morfológicas, que podem ser capazes de lidar melhor com algumas propriedades visuais dos objetos. Essa DeepMorphNet pode ser treinada e otimizada de ponta a ponta, utilizando técnicas tradicionais existentes, comumente empregadas no treinamento de abordagens de aprendizagem profunda, e pode ser usada em diversas aplicações, como classificação de imagens e pixels, detecção e outros. Uma avaliação sistemática do algoritmo proposto foi conduzida. Resultados mostraram que a DeepMorphNet é uma técnica promissora capaz de aprender características distintas quando comparadas àquelas aprendidas pelos atuais métodos de aprendizagem profunda. A lot of information may be extracted from the Earth's surface through images acquired by airborne sensors. Even more, nowadays, enhanced information may be extracted from high spatial resolution images obtained from new sensor technologies. This information may be exploited in several tasks (including, classification and segmentation) assisting in different applications, such as urban planning, disaster relief, phenological studies, etc. Although widely used, the process of distilling this information is strongly based on efficiently encoding features, a challenging task given the distinct properties of the images. In fact, independently of the task, encoding the spatial features in an efficient and robust fashion is the key to generating good discriminative models. Even though many descriptors have been proposed and successfully used to encode spatial features of remote sensing images, most applications demand specific description techniques. Deep Learning, an emergent machine learning approach based on neural networks, is capable of learning specific features and classifiers at the same time and adjust at each step, in processing time, to better fit the need of each problem. For several tasks, it has achieved very good results, mainly boosted by the feature learning performed which allows the method to extract specific and adaptable features depending on the data. In this project, we tackled two main tasks, image and pixel classification, using Deep Learning to encode spatial features over high-resolution remote sensing images. Initially, we faced the scene classification problem, in which the final model performance is highly dependent on the quality of extracted features. Based on recently published works and after a systematic set of experiments, our first contribution was a novel Convolutional Network architecture fully trained specifically for the remote sensing domain. Experiments showed that the proposed architecture outperformed state-of-the-art algorithms, demonstrating the effectiveness of deep learning methods to encode features even for remote sensing domain. Then, based on adversities to explore deep learning-based techniques (such as small data), we proposed to evaluate different strategies to better exploit Convolutional Networks in the remote sensing domain. Specifically, three strategies were evaluated in three very distinctive datasets. Experiments demonstrate that fine-tuning a network into the specific domain is the best strategy to exploit Convolutional Networks. A second task tackled by this project is the pixelwise classification (also known as semantic segmentation). This task is of great interest to the remote sensing community since it involves the automatic creation of thematic maps from labeled samples. Such maps may help to understand events over a specific region, such as new urban areas, deforestation, etc. Considering such task, our first contribution is a new Convolutional Network architecture to perform pixel classification of remote sensing images. This technique is based on context windows, i.e., overlapping patches that carry the context of the centered pixels and help to understand the spatial patterns around them. Experiments showed that the proposed approach provides significant improvements when compared to traditional and state-of-the-art methods. Then, based on the problems of the previous technique and of other existing methods, we proposed a novel approach that employs an evolved version of the previously proposed network (as well as other architectures) to perform pixel labeling of the entire input image (instead of classifying only the centered pixel). Moreover, this new technique is able to aggregate multi-context information without increasing the number of parameters. Results showed that the proposed algorithm provides improvements in pixelwise classification accuracy when compared to state-of-the-art methods. Finally, our last contribution is a novel method that, when compared to other existing approaches, is capable of better learning some visual characteristics (such as edges and corners), which are very useful for some applications and domains, including the remote sensing one. Precisely, this method, called Deep Morphological Network (DeepMorphNet), is able to perform and optimize non-linear morphological operations, that may be able to better cope with some interesting visual properties of the objects. These DeepMorphNets can be trained and optimized end-to-end using traditional existing techniques commonly employed in the training of deep learning approaches and can be used in several applications, such as image and pixel classification, detection, and so on. A systematic evaluation of the proposed algorithm was conducted. Experimental results showed that the DeepMorphNets is a promising technique that can learn distinct features when compared to the ones learned by current deep learning methods.

We present an analysis of three possible strategies for exploiting the power of existing convolutional neural networks (ConvNets or CNNs) in different scenarios from the ones they were trained: full training, fine tuning, and using ConvNets as feature extractors. In many applications, especially including remote sensing, it is not feasible to fully design and train a new ConvNet, as this usually requires a considerable amount of labeled data and demands high computational costs. Therefore, it is important to understand how to better use existing ConvNets. We perform experiments with six popular ConvNets using three remote sensing datasets. We also compare ConvNets in each strategy with existing descriptors and with state-of-the-art baselines. Results point that fine tuning tends to be the best performing strategy. In fact, using the features from the fine-tuned ConvNet with linear SVM obtains the best results. We also achieved state-of-the-art results for the three datasets used.

With the advances on science, a powerful computational infrastructure is desirable to increase the performance of experiments. The same holds true for research on the Computer Vision field, which deals with large amounts of data and also requires intensive computation. Nevertheless, the administration of a computational infrastructure involves many tasks, such as system configuration, preventive maintenance and storage management, which becomes very challenging for many research groups. With that in mind, this work proposes an infrastructure model to assist researchers with focus on Computer Vision. We conducted a series of tests to evaluate the performance of our model and the processing power of our infrastructure.

In this paper, we evaluate the generalization power of deep features (ConvNets) in two new scenarios: aerial and remote sensing image classification. We evaluate experimentally ConvNets trained for recognizing everyday objects for the classification of aerial and remote sensing images. ConvNets obtained the best results for aerial images, while for remote sensing, they performed well but were outperformed by low-level color descriptors, such as BIC. We also present a correlation analysis, showing the potential for combining/fusing different ConvNets with other descriptors or even for combining multiple ConvNets. A preliminary set of experiments fusing ConvNets obtains state-of-the-art results for the well-known UCMerced dataset.

Esta dissertação apresenta algoritmos eficientes para anotar e decompor peças de roupas a partir de dados provindos de redes sociais, como Facebook e Instagram. Anotação de roupas pode ser informalmente descrito como reconhecer, o mais precisamente possível, cada peça do traje que aparece em uma imagem. A decomposição, por sua vez, procura além de anotar as peças de roupa, também localizá-las na imagem. Tais tarefas tem papel importante em áreas como vigilância, reconhecimento de ações, busca por pessoas, sistemas de recomendação e de comércio eletrônico. Estes problemas trazem desafios interessantes vinculados à visão computacional e ao reconhecimento de padrões como, por exemplo, distinguir roupas visualmente parecidas mas conceitualmente diferentes, ou identificar um padrão para uma peça específica, já que esta pode ter diferentes cores, formas, texturas e aparência. Inicialmente, o problema de anotação de roupas foi analisado considerando métodos estatísticos de aprendizado de máquina. Para isso, uma extensa avaliação das técnicas de extração de características visuais, incluindo descritores locais e globais, foi feita. Em seguida, formulamos a tarefa de anotação como um problema de classificação multi-modal e multi-rótulo, isto é: (i) conteúdo visual e textual (tags relacionadas às imagens) estão disponíveis para os classificadores, (ii) os classificadores precisam predizer um conjunto de rótulos (um conjunto de peças de roupas) e, (iii) a decisão sobre quais os rótulos devem ser atribuídas à imagem ocorre através de uma função, construída a partir de um conjunto de instâncias. Com esta configuração, propomos duas abordagens: (i) a pontual, chamada neste trabalho de MMCA, que usa uma única imagem como entrada para o classificador, e (ii) a pareada, chamada de M3CA, que usa pares de imagens como entrada para seus classificadores. Comparamos ambos os métodos para definir qual o melhor para o problema em questão. Para cada uma, aplicamos um algoritmo de classificação que usa regras de associação para construir modelos de reconhecimento que combina informações visuais e textuais. Também usamos uma estratégia de minimização de entropia para encontrar quais rótulosdevem ser associados à cada imagem. Realizamos uma avaliação sistemática dos métodos propostos usando fotos coletadas de duas grandes mídias sociais relacionadas à moda, \url{pose.com} e \url{chictopia.com}. Os resultados mostram que os métodos propostos fornecem melhorias quando comparados a algoritmos popularemente utilizados que variam entre 20\% to 30\% em termos de acurácia.Em um segundo momento, analisamos o problema de decomposição de imagens utilizando aprendizado profundo. Propomos um modelo de redes de convolução utilizando uma estratégia multi-escala. Mais especificamente, empregamos diferentes níveis de redes onde cada nível processa imagens de dimensões diferentes, ou seja, a cada nível as imagens são decompostas em pedaços menores, possibilitando assim com que a rede classifique pequenos detalhes. No primeiro nível, imagens com maiores dimensões são processadas em uma rede mais robusta. As imagens com entropia baixa já adquirem sua classificação neste nível, enquanto as imagens com entropia alta (não classificadas perfeitamente) são subdivididas e passam para o segundo nível.No terceiro patamar, as imagens não classificadas no segundo nível são novamente subdivididas em pedaços ainda menores e, enfim, classificadas. Ao final, teremos as classes de cada pedaço da imagem, e podemos recompo-la. Para avaliar esta abordagem, utilizamos um conjunto de imagens coletadas do site \url{chictopia.com}, e nossos experimentos mostram que nossa abordagem fornecem resultados promissores. In this work, we present effective algorithms to automatically annotate and parse clothes from social media data, such as Facebook and Instagram. Clothing annotation can be informally stated as recognizing, as accurately as possible, each garment item that appears in a photo.Clothing parsing, in turn, locates and annotate each garment item in a photo. These tasks play important roles in several areas, including surveillance, action recognition, person search, recommender systems and e-commerce. They also pose interesting challenges for existing vision and recognition algorithms, such as distinguishing between similar but conceptually different types of clothes or identifying a pattern of a specific item, since it can have different colors, shapes, textures and appearance. Initially, the clothing annotation problem was analyzed considering statistical methods of machine learning. For this purpose, we perform an extensive evaluation of the visual feature extraction techniques, including global and local descriptors. Then, we formulate the annotation task as a multi-label and multi-modal classification problem (i) both image and textual content (i.e., tags related to the image) are available for learning classifiers, (ii) the classifiers must predict a set of labels (i.e., a set of garment items), and (iii) the decision on which labels to assign to the query photo comes from instances (or {\em bag} of instances) that are used to build a function, which separates labels that should be assigned to the query photo, from those that should not be assigned. Using this configuration, we propose two approaches: (i) the pointwise one, called MMCA, which uses a single image as input to the classifiers, and (ii) a multi-instance classification, called M3CA, also known as pairwise approach, that uses pair of images as input to the classifiers. We compare both approaches in order to define the best one for the problem. For both of them, we propose a classification algorithm that employs association rules in order to build a recognition model that combines textual and visual information. We also adopt an entropy-minimization strategy in order to find the best set of labels that should be assigned to the query photo. We conduct a systematic evaluation of the proposed algorithms using everyday photos collected from two major fashion-related social media, namely \url{pose.com} and \url{chictopia.com}. Our results show that the proposed approaches provide improvements when compared to popular first choice multi-label, multi-modal, multi-instance algorithms that range from 20\% to 30\% in terms of accuracy. In a second phase, we analyzed the clothing parsing problem using deep learning. We propose a multi-scale convolutional neural network model. Specifically, we use different network levels where each level processes images with different dimensions, i.e., after every level the images are decomposed into smaller patches, allowing the network to capture minimal details. In the first level, bigger images are processed in a robust network. Images with low entropy already get their final class in this level, while the others with high entropy (classification still undefined) are splitted into smaller patches and go to the next one. In the third and last level, images without final classification in the second level are again divided into even smaller patches and, finally, classified.At the end, we have a class associated with each patch of the image and we can recompose it. To evaluate this approach, we use a dataset crawled from \url{chictopia.com}. Our experiments shows that our proposed approach achieves promising results.

Submitted by Guilherme Lemeszenski (guilherme@nead.unesp.br) on 2013-08-22T18:49:19Z No. of bitstreams: 1 S0100-46702007000200010.pdf: 140050 bytes, checksum: 5393a5c0c0d64109c9b8d7e7468c851f (MD5) Made available in DSpace on 2013-08-22T18:49:19Z (GMT). No. of bitstreams: 1 S0100-46702007000200010.pdf: 140050 bytes, checksum: 5393a5c0c0d64109c9b8d7e7468c851f (MD5) Previous issue date: 2007-01-01 Made available in DSpace on 2013-09-30T19:39:38Z (GMT). No. of bitstreams: 2 S0100-46702007000200010.pdf: 140050 bytes, checksum: 5393a5c0c0d64109c9b8d7e7468c851f (MD5) S0100-46702007000200010.pdf.txt: 21318 bytes, checksum: f475392c82ef48d11a3b46c96123cd06 (MD5) Previous issue date: 2007-01-01 Submitted by Vitor Silverio Rodrigues (vitorsrodrigues@reitoria.unesp.br) on 2014-05-20T14:18:53Z No. of bitstreams: 2 S0100-46702007000200010.pdf: 140050 bytes, checksum: 5393a5c0c0d64109c9b8d7e7468c851f (MD5) S0100-46702007000200010.pdf.txt: 21318 bytes, checksum: f475392c82ef48d11a3b46c96123cd06 (MD5) Made available in DSpace on 2014-05-20T14:18:53Z (GMT). No. of bitstreams: 2 S0100-46702007000200010.pdf: 140050 bytes, checksum: 5393a5c0c0d64109c9b8d7e7468c851f (MD5) S0100-46702007000200010.pdf.txt: 21318 bytes, checksum: f475392c82ef48d11a3b46c96123cd06 (MD5) Previous issue date: 2007-01-01 Muitos métodos analíticos estão sendo desenvolvidos visando à determinação de contaminantes orgânicos, especialmente alteradores endócrinos. Tais métodos baseiam-se geralmente na extração em fase sólida (SPE) seguida por determinação cromatográfica (CG ou HPLC). No presente trabalho utilizou-se ferramentas quimiométricas no processo de SPE para avaliar os principais fatores que influenciam tal processo e as interações entre os mesmos. Foram analisadas matrizes de água subterrânea fortificada com hormônios (17 b estradiol, estrona e 17 b etinilestradiol) e a determinação analítica foi feita por HPLC/Fluorescência. Um planejamento fatorial completo foi utilizado. Os fatores escolhidos incluíram: condicionamento da fase sólida, concentração dos analitos, volume da amostra e solvente de eluição. As melhores condições obtidas foram: 500 mL da amostra, condicionamento da fase sólida (C18) com acetona (4mL), metanol (6 mL) e água pH 3(10 mL), e eluição dos analitos com 4 mL de acetona. Several analytical methods have been proposed for organic pollutants determination in water, specially endocrine disrupters. These methods are frequently based in solid phase extraction (SPE) followed by chromatographic determination (GC or HPLC). In this work it was used chemometric tools on experimental design and experimental data treatment to evaluate the principal factors in SPE process and their interactions. It was used water matrices fortified with hormones (17 b estradiol, estrone and 17 b ethinylestradiol) and HPLC/Fluorescence did analytical determination. It was used a factorial complete design. The choosed factors included: solid phase type, analytes concentration, sample volumen, eluent composition, analytes concentration and solvents to solid phase conditioning. The best condition obtained in this initial procedure included: 500 mL sample, conditioning SPE cartridge with acetone (4mL), methanol (6 mL) and water pH 3(10 mL), and analytes elution with acetone. Unesp Instituto de Química Unesp Instituto de Química

CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico FAPEMIG - Fundação de Amparo à Pesquisa do Estado de Minas Gerais CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior It is undeniable that aerial and orbital images can provide useful information for a large variety of tasks, such as disaster relief and urban planing. But, since these images only see the Earth from one point of view, some applications can benefit from complementary information provided by other perspective views of the scene, such as ground-level images. Despite a large number of public image repositories for both georeferenced photographs and aerial images (such as Google Maps and Google Street View), there is a lack of public datasets that allow the development of approaches that exploit the benefits and complementarity of aerial/ground imagery. Because of that, in this dissertation, we present two new publicly available datasets named AiRound and CV-BrCT (Cross-View Brazilian Construction Types). The first one contains triplets of images from the same geographic coordinate with different perspectives, obtained at various places around the world. Each triplet is composed of an aerial RGB image, a ground-level perspective image, and a Sentinel-2 sample. The second dataset contains pairs of aerial and street-level images extracted from the southeast of Brazil. For this dissertation, we conducted a series of experiments involving both proposed datasets with the main objectives of (i) explore the complementary information that aerial and ground images have by using multi-modal machine learning models to enhance results, (ii) compare different feature fusion approaches applied in several state-of-the-art Convolutional Neural Network architectures, and (iii) investigate alternatives to handle missing data in a multi-modal scenario. Experiments show that, when compared to networks trained using only a single view, feature fusion algorithms achieved gains up to 0.15 and 0.20 in F1-Score for the AiRound and CV-BrCT datasets, respectively. Since it is not always possible to obtain the paired aerial/ground samples of a place, we also designed a framework to handle scenarios with missing samples. Comparing the results of a single-view network classification to the use of our framework integrated with a multi-view model, we achieved gains up to 0.03 in F1-Score for both datasets. Thus, our missing data completion framework has proven to be a more effective approach than just classifying images using a single-view model. É inegável que imagens aéreas e orbitais fornecem uma grande variedade de informações para muitos tipos de aplicações, tais como logística humanitária para desastres naturais e planejamento urbano. Porém, devido ao fato dessas imagens sempre terem a mesma perspectiva, algumas aplicações podem ter grandes benefícios, caso sejam complementadas com fotos de outros ângulos, como por exemplo, imagens tomadas ao nível do solo. Apesar do grande número de repositórios de imagens públicos que permitem a aquisição de fotos e imagens aéreas georreferenciadas (tais como Google Maps e Google Street View), existe uma falta de datasets públicos com imagens pareadas de múltiplas visões. Devido a essa escassez, nesta dissertação foram produzidos dois novos datasets. O primeiro deles foi nomeado AiRound, e para cada amostra possui triplas de imagens de uma mesma coordenada geográfica. Cada tripla do AiRound contém uma imagem aérea, uma foto a nível do solo e uma imagem multi-espectral do satélite Sentinel-2. O segundo dataset foi nomeado CV-BrCT (Cross-View Brazilian Construction Types). Este é composto por pares de imagens (nível aéreo e nível do solo) coletados do Sudeste do Brasil. Para esta dissertação, conduzimos uma série de experimentos envolvendo ambos os datasets e visando os seguintes objetivos: (i) explorar a complementariedade de informação que imagens aéreas e a nível de solo possuem, usando modelos de aprendizado de máquina multimodais, (ii) comparar diferentes técnicas de fusão de características aplicadas em arquiteturas de redes neurais convolucionais, e (iii) investigar formas de lidar com atributos ausentes em um cenário multi-modal, no qual sempre existirá falta de dados em um determinado domínio. Experimentos demonstram que se comparados a modelos treinados/avaliados em um único domínio, algoritmos de fusão de informação atingem ganhos de até 0.15 e 0.20 no F1-Score para os datasets AiRound e CV-BrCT, respectivamente. Como nem sempre é possível obter imagens pareadas (em níveis aéreo e de solo) do mesmo local, projetamos um framework para lidar com cenários que utilizam algoritmos multimodais, e que nem sempre exigem pares de imagens para todas as amostras. Comparando resultados de classificações usando imagens de um único domínio com o uso do nosso framework integrado a um modelo multimodal, atingimos um ganho de 0.03 no F1-Score para ambos os datasets. Portanto, demonstramos que utilizar o nosso framework é mais eficaz do que apenas classificar usando dados e classificadores de um único domínio.