Search

This thesis will aim to design and implement an ADS-B integral and distributed system for detecting aircrafts broadcasts messages. Those messages will be processed and stored in a SQL database the project will rely on an MariaDB server installed on a home build server computer. After the data processing, clients through a web application developed with Python and Django framework will be able to consume on real time the information which will be displayed in an interactive map thanks to an API REST. Additionally, it will be also possible to access historic and real time data for researchers through downloadable files or through a direct SQL connection to the server with read-only privilege dedicated account. The system will be developed at a first stage with a raspberry pi 4B 8GB and then migrated to a dedicated server, the system is gives only support to GNU-LINUX Debian based distributions such as Ubuntu and Raspbian. The hardware for receiving messages will be an RTL-SDR receiver and will rely on the following open-source projects: dump1090 for Raspbian OS or dump1090-mutability for Ubuntu OS where data will be collected through an UDP socket. Scalability will be taken into account allowing to have multiple base stations feeding to the server at the same time. It will involve from the radio frequency system conditioning to the front-end development. The applications for such system will be the promotion of the EETAC, a tool for researchers in ATM, for performing some basic statistics with the data harvested and for estimating the radio propagation in the 1090 MHz band

The booming of AI tools capable of modifying images has equipped fake media producers with strong tools in their arsenal. Complementary to the efforts of implementing fake media detectors, research organizations are designing a standardized way of describing the modification history of digital media in a cryptographically secure way, ensuring that this information cannot be tampered with. This thesis proposes a specification which focuses on JPEG images and specifies a data model based on the JPEG Universal Metadata Box Format (JUMBF) standard. Furthermore, it proposes the encryption of a subset of provenance metadata that could pose privacy-related risks to the users. Along with the specification, a library has been developed to manage provenance information of JPEG images. To that extent, a set of libraries that handle JUMBF information is required to be implemented. These libraries have been submitted as a proposed reference software contributing to the JUMBF standard.

In the financial documents processing, especially in the case of accounts payable, as the number of documents increases, manually extracting data becomes time-consuming, repetitive, and allows for mistakes to occur. Automating this process has been a field of research in document understanding for decades, as it became a need to help companies save resources and optimize existing processes in their digital transformation journey. Since most of these documents such as invoices are confidential documents, there is not much data publicly available to take advantage of and improve the data extraction via Machine Learning (ML) models. In the scope of this work, we were able to get that data from the existing customer to create ML models to extract information from their data. This thesis investigates three main strategies to handle automatic document processing. First, an automatic strategy for document annotation based on Microsoft Azure Cognitive services. This technique allows faster data preparation with high key field annotation precision to be used for developing ML models for data extraction. Second, a comparison of two methods to extract the logo and identify its issuer; one is based on an Object Detection model, a VGG16 architecture is fine-tuned on invoices and their corresponding logo bounding box, while the other approach consists of extracting words lying on the top of an invoice, then detecting organization entities using Flair NER model. The third and final strategy consists of implementing a state-of-the-art Transformer model, called LayoutLM, to identify a couple of fields from a document, such as the total amount, date of issue, and company name. This strategy is introduced to avoid the costs of Azure services. Results are later presented and discussed evaluating the performance of each approach. Finally, a conclusion summarizes the work and the achievements realized, along with a snippet describing future work. En el procesamiento de documentos financieros, especialmente en el caso de las cuentas por pagar, a medida que aumenta la cantidad de documentos, la extracción manual de datos se vuelve lenta, repetitiva y permite que ocurran errores. La automatización de este proceso ha sido un campo de investigación en la comprensión de documentos durante décadas, ya que se convirtió en una necesidad para ayudar a las empresas a ahorrar recursos y optimizar los procesos existentes en su proceso de transformación digital. Dado que la mayoría de estos documentos, como las facturas, son documentos confidenciales, no hay muchos datos disponibles públicamente para aprovechar y mejorar la extracción de datos a través de modelos de aprendizaje automático (AA). En el ámbito de este trabajo, hemos podido obtener estos datos de un cliente para crear modelos AA para extraer información de sus datos. Esta tesis investiga tres estrategias principales para manejar el procesamiento automático de documentos. Primero, una estrategia automática para la anotación de documentos basada en los servicios cognitivos de Microsoft Azure. Esta técnica permite una preparación de datos más rápida con alta precisión de anotación de campo clave para el desarrollo de modelos AA para la extracción de datos. En segundo lugar, una comparación de dos métodos para extraer el logotipo e identificar a su emisor; uno se basa en un modelo de detección de objetos, una arquitectura VGG16 se ajusta con precisión en las facturas y su cuadro delimitador de logotipo correspondiente, mientras que el otro enfoque consiste en extraer palabras que se encuentran en la parte superior de una factura y luego detectar entidades de la organización utilizando el modelo Flair NER. La tercera y última estrategia consiste en implementar un modelo transformador de última generación, llamado LayoutLM, para identificar un par de campos de un documento, como el importe total, la fecha de emisión y el nombre de la empresa. Esta estrategia se introduce para evitar los costes de los servicios de Azure. Los resultados se presentan y discuten más adelante para evaluar el desempeño de cada enfoque. Finalmente, una conclusión resume el trabajo y los logros alcanzados, junto con un fragmento que describe el trabajo futuro. En el processament de documents financers, especialment en el cas de comptes a pagar, a mesura que augmenta el nombre de documents, l'extracció manual de dades es fa llarga, repetitiva i permet que es produeixin errors. L'automatització d'aquest procés ha estat un camp d'investigació en la comprensió de documents durant dècades, ja que es va convertir en una necessitat per ajudar les empreses a estalviar recursos i optimitzar els processos existents en el seu viatge de transformació digital. Com que la majoria d'aquests documents, com ara les factures, són documents confidencials, no hi ha moltes dades disponibles públicament per aprofitar i millorar l'extracció de dades mitjançant models d'aprenentatge automàtic (AA). En l'àmbit d'aquest treball hem pogut obtenir aquestes dades d’un client per crear models d’AA per extreure informació de les seves dades. Aquesta tesi investiga tres estratègies principals per gestionar el processament automàtic de documents. En primer lloc, una estratègia automàtica per a l'anotació de documents basada en els serveis cognitius de Microsoft Azure. Aquesta tècnica permet una preparació de dades més ràpida amb una alta precisió d'anotació de camp clau per desenvolupar models d’AA per a l'extracció de dades. En segon lloc, una comparació de dos mètodes per extreure el logotip i identificar el seu emissor; un es basa en un model de detecció d'objectes, una arquitectura VGG16 s'ajusta a les factures i el seu corresponent quadre delimitador del logotip, mentre que l'altre enfocament consisteix a extreure paraules que es troben a la part superior d'una factura i, a continuació, detectar les entitats de l'organització mitjançant el model Flair NER. La tercera i última estratègia consisteix a implementar un model de transformador d'última generació, anomenat LayoutLM, per identificar un parell de camps d'un document, com ara l'import total, la data d'emissió i el nom de l'empresa. Aquesta estratègia s'introdueix per evitar els costos dels serveis Azure. Més endavant es presenten i es discuteixen els resultats per avaluar el rendiment de cada enfocament. Finalment, una conclusió resumeix el treball i els assoliments aconseguits, juntament amb un fragment que descriu el treball futur.

Trabajo presentado en la 9th International Conference on 3D Vision, celebrada online del 1 al 3 de diciembre de 2021, We address the problem of multi-person 3D body pose and shape estimation from a single image. While this problem can be addressed by applying single-person approaches multiple times for the same scene, recent works have shown the advantages of building upon deep architectures that simultaneously reason about all people in the scene in a holistic manner by enforcing, e.g., depth order constraints or minimizing interpenetration among reconstructed bodies. However, existing approaches are still unable to capture the size variability of people caused by the inherent body scale and depth ambiguity. In this work we tackle this challenge by devising a novel optimization scheme that learns the appropriate body scale and relative camera pose, by enforcing the feet of all people to remain on the ground floor. A thorough evaluation on MuPoTS-3D and 3DPW datasets demonstrates that our approach is able to robustly estimate the body translation and shape of multiple people while retrieving their spatial arrangement, consistently improving current state-of-the-art, especially in scenes with people of very different heights., This work is partly supported by the Spanish government with the project MoHuCo PID2020- 120049RB-I00, the ERA-Net Chistera project IPALM PCI2019- 103386 and Mar´ıa de Maeztu Seal of Excellence MDM-2016- 0656. Adria Ruiz acknowledges financial support from MICINN (Spain) through the program Juan de la Cierva.

In recent years, the introduction of Machine Learning together with the use of conventional image processing techniques has given a boost to automated document processing. One of the main issues that has compromised the performance of these automated systems has been the degradation the documents may present. Some kinds of degradation, such as document artifacts (watermarks, strips, stains?) or scan distortions can be difficult to remove with conventional methods, resulting in the deterioration of OCR systems. This study attempts to provide an end-to-end framework based on conditional GANs (cGANs) that restore degraded document images. Specifically, it intends to improve the text quality in documents to enhance the performance of subsequent automation systems. The outcome has been very satisfactory as the framework has been able to significantly increase the automation rate of the projects it has been tested in. Recientemente, la aparición del Machine Learning conjuntamente con el uso de técnicas convencionales de procesamiento de imagen han propiciado un considerable salto de calidad en los sistemas de procesado automático de documentos. Sin embargo, uno de los factores que ha comprometido el rendimiento de estos sistemas son las diversas formas de degradación presentes en los documentos a tratar. La presencia de artefactos no deseados, tales como marcas de agua, manchas, garabatos u otros tipos de distorsiones que se pueden generar a la hora de escanear los documentos pueden resultar extremadamente difíciles de eliminar mediante técnicas convencionales. Este hecho conlleva una degradación en los sistemas OCR encargados de analizar el texto. Este estudio presenta un sistema basado en cGANs (conditional GANs) que restaura las imágenes de documentos degradados. El sistema es capaz de eliminar ciertos tipos de degradaciones, mejorando la calidad del texto presente en los documentos. El resultado ha sido sumamente satisfactorio, ya que se ha observado una considerable mejora en el rendimiento de los sistemas en los que se ha incluido el módulo diseñado, hecho que se ha traducido en un aumento en las tasas de automatización de los proyectos testados. Recentment, l’aparició del Machine Learning conjuntament amb l’ús de tècniques convencionals de processament d’imatge han propiciat un considerable salt de qualitat pel que fa als sistemes de processat automàtic de documents. No obstant això, un dels factors que ha compromès el rendiment d’aquests sistemes són les diverses formes de degradació presents en els documents a tractar. La presència d’artefactes no desitjats, com ara marques d’aigua, taques, gargots o d’altres tipus de distorsions que es poden generar a l’hora d’escanejar els documents poden resultar extremadament difícils d’eliminar mitjançant tècniques convencionals. Aquest fet comporta una degradació en els sistemes OCR encarregats d’analitzar el text. Aquest estudi presenta un sistema basat en cGANs (conditional GANs) que restaura les imatges de documents degradats. El sistema és capaç d’eliminar certs tipus de degradacions, millorant la qualitat del text present en els documents. El resultat ha estat summament satisfactori, ja que s’ha observat una considerable millora en el rendiment dels sistemes en els quals s’ha inclòs el mòdul dissenyat, fet que s’ha traduït en un augment en les taxes d’automatització dels projectes testats.

This project arises from the collaboration with a contemporary art production that is looking for a system capable of generating low-resolution images on a thermochromic sheet as if it was an image display. Therefore, a research of alternatives has been carried out in order to find the best way to generate heat in controlled points. Among all the alternatives, the usage of resistors arranged in matrix form has been selected. The obtained system can be modular, where each module measures 8x8 cm and comprises 128 resistors (16x8), which makes it very comfortable due to the easy image size and resolution adaptation. Regarding the final image result, the goals set by the artist and art production are met: the progressive appearance of a low-but-enough resolution image capable of being interpreted without distinguishing the pixels because of a felt that acts as an insulator between the resistors and the thermochromic sheet. Este proyecto es fruto de la colaboración con una productora de arte contemporáneo que busca un sistema capaz de generar imágenes de baja resolución en una lámina de material termocrómico como si de un display se tratara, al que se le pasa una imagen y este la muestra. Así, en este proyecto se ha hecho un estudio de alternatives para generar puntos de calor controlados de donde se ha seleccionado el uso de resistencias puestas en forma matricial. El sistema obtenido tiene la capacidad de ser modular, midiendo 8x8 cm cada módulo y teniendo 128 resistencias (16x8), que lo hace muy cómodo a la hora de poderse adaptar a la medida y resolución de la imagen que se busca. En cuanto al resultado de la imagen final, se cumple con los objetivos marcados por la artista y la productora de arte, la aparición progresiva de una imagen de resolución baja, pero con la suficiente para poder interpretarla sin que en esta se le puedan distinguir los pixeles gracias a un fieltro que hace de aislante entre las resistencias y la lámina termocrómica. Aquest projecte és fruit de la col·laboració amb una productora d'art contemporani que busca un sistema capaç de generar imatges de baixa resolució en una làmina de material termocròmic com si es tractés d'un display al qual se li passa una imatge i aquest la mostra. Així, en aquest projecte s'ha fet un estudi d'alternatives per generar calor en punts controlats d'on s'ha acabat seleccionant l'ús de resistències disposades en forma de matriu. El sistema obtingut té la capacitat de ser modular, mesurant cada mòdul 8x8 cm i constant de 128 resistències (16x8), el que el fa molt còmode a l'hora de poder-se adaptar a la mida i resolució de la imatge que es busca. Pel que fa al resultat d'imatge final, es compleix amb els objectius marcats per l'artista i productora d'art, l'aparició progressiva d'una imatge de baixa resolució però la suficient com interpretar-la i sense que a aquesta se li puguin distingir els píxels gràcies a un feltre que fa d'aïllant entre les resistències i la làmina termocròmica.

Automatic image colourisation is a complex and ambiguous task due to having multiple correct solutions. Previous approaches have resulted in desaturated results unless relying on a significant user interaction. In this thesis we study the state of the art for colourisation and we propose an automatic colourisation approaches based on generative adversarial networks that incorporates a feature reconstruction loss during training. The generative network is framed in an adversarial model that learns how to colourise by incorporating perceptual understanding of the colour. Qualitative and quantitative results show the capacity of the proposed method to colourise images in a realistic way, boosting the colourfulness and perceptual realism of previous GAN-based methodologies. We also study and propose a second approach that incorporates segmentation information in the GAN framework and obtain quantitative and qualitative results.

Face is one of the most common biometric modalities which is used for identification. In this context, face recognition has gained an important role in biometric applications based on identification systems during the last few decades. Since there is no physical interaction required during recognition or identification, it's easy to deploy and implement. In face recognition, a face is categorized as known or unknown by comparing a face with all the faces in a database. Due to inherent distinct features, human face analysis is one of the most effective methods of identifying individuals. Nowadays, utilizing hyperspectral images in face recognition is one of the most important research topics in biometrics, since they contain additional significant spectral information compared to 2D images which have only information in spatial dimensions (texture and structure). A hyperspectral image is a data cube containing two spatial and one spectral dimension. Hyperspectral image samples are captured by a hyperspectral camera which operates at multiple narrow bands within the visible spectrum and neighboring near-infrared spectra. Hyperspectral imaging provides new prospects for improving face recognition accuracy since they contain information in both space and spectral axes. Hence significant information for each person regarding the skin based on reflected, absorbed and released electromagnetic energy at different wavelengths can be extracted. Additional spectral information which is not embedded in traditional grey/color facial images provides an opportunity to improve the recognition accuracy. Hyperspectral imaging employs spatial and spectral relationship simultaneously, which improves segmentation and classification in the respective applications. Difficulties encountered in visible light-based face recognition systems, such as the variance in orientation, illumination or expressions can be minimized by employing hyperspectral imaging. Besides these opportunities, hyperspectral images pose some challenges such as low signal to noise ratios, high dimensionality, and data acquisition needs expensive cameras with multiple sampling in visible and nearinfrared spectra. Despite mentioned challenges, hyperspectral images contain more independent and significant information obtained from different sub-bands than 2D images. Hence, hyperspectral images represented in 3D-cubes are by far more capable in classification processes, which is also ideal for spoofing attacks. In this thesis, we propose novel methods for feature extraction for facial hyperspectral image recognition. The main goal of the thesis is to improve the recognition accuracy of hyperspectral face images. In the first method, three different approaches are proposed employing 3D discrete wavelet transform (3D-DWT) to extract features from the subbands generated by discrete wavelet decomposition. Three approaches include 3D-subband energy (3D-SE), 3D-subband overlapping cube (3D-SOC) and 3D-global energy (3D-GE), which extract different feature vector for each approach containing the energy values calculated from different wavelet sub-bands at different levels of decomposition. Feature vectors generated by three different approaches go through a classifier to complete the face recognition task. In the second proposed method, fusion of spectral information into a single 2D image is achieved by band-specific signal to noise ratio (SNR) based weighting. The fusion method assigns weights based on the calculated band-specific SNR values, weighted sum of the bands generate a single 2D face image. Hence, each pixel along spectral axis is fused to a single pixel resulting a 2D output face image for each 3D hyperspectral face cube. In the third method, in order to fuse spectral bands in hyperspectral face cubes, we apply discrete wavelet transform (DWT) to each pixel along the spectral axis consecutively until the spectral vector for each pixel is decimated to a single pixel transforming the 3D input spectral face image cube into a 2D output image. 2D output images obtained by the second and third methods are processed using principal component analysis method and face recognition is performed with the help of a classifier. The experimental results reveal that recognition accuracy of all proposed methods by using standard hyperspectral databases outperform alternative hyperspectral face recognition of the state-of-the-art methods. Keywords: hyperspectral face image, face recognition, discrete wavelet transform, feature extraction, classification, signal to noise ratio. ÖZ: Yüz, kimlik tanıma için kullanılan en yaygın biyometrik yöntemlerden biridir. Bu bağlamda, yüz tanıma, son birkaç on yılda kimlik tanıma sistemlerine dayanan biyometrik uygulamalarda önemli bir rol oynamıştır. Yüz tanıma, kimlik tanımlama sırasında fiziksel bir etkileşim gerekmediğinden, dağıtımı ve uygulaması kolaydır. Yüz tanımada, bir yüz, veritabanındaki tüm yüzlerle karşılaştırılarak bilinen veya bilinmeyen olarak kategorize edilir. Doğasında farklı özellikler nedeniyle, insan yüzü analizi bireyleri tanımlamanın en etkili yöntemlerinden biridir. Günümüzde, yüz tanımada hiperspektral görüntülerin kullanılması, sadece mekansal boyutlarda (doku ve yapı) bilgi içeren 2D görüntülere kıyasla ek önemli spektral bilgi içerdiğinden, biyometride en önemli araştırma konularından biridir. Hiperspektral görüntü, iki uzamsal ve bir spektral boyut içeren bir veri küpüdür. Hiperspektral görüntü örnekleri, görünür spektrumda ve komşu kızılötesine yakın spektrumlarda birden çok dar bantta çalışan bir hiperspektral kamera tarafından yakalanır. Hiperspektral görüntüleme, hem boşlukta hem de spektral eksenlerde bilgi içerdiğinden yüz tanıma doğruluğunu iyileştirmek için yeni beklentiler sağlar. Bu nedenle, her insan için farklı dalga boylarında yansıtılan, emilen ve salınan elektromanyetik enerjiye dayanan cilt hakkında önemli bilgiler elde edilebilir. Geleneksel gri / renkli yüz görüntülerine gömülmeyen ek spektral bilgiler, tanıma doğruluğunu artırma fırsatı sunar. Hiperspektral görüntüleme aynı anda mekansal ve spektral ilişkiyi kullanır, bu da ilgili uygulamalarda segmentasyonu ve sınıflandırmayı geliştirir. Yönlendirme, aydınlatma veya ifadelerdeki varyans gibi görünür ışık tabanlı yüz tanıma sistemlerinde karşılaşılan zorluklar, hiperspektral görüntüleme kullanılarak en aza indirilebilir. Bu fırsatların yanı sıra, hiperspektral görüntüler düşük sinyal-gürültü oranları, yüksek boyutluluk ve veri toplama gibi bazı zorluklar ortaya çıkarır ve görünür ve kızılötesine yakın spektrumlarda çoklu örnekleme ile pahalı kameralar gerektirir. Bahsedilen zorluklara rağmen, hiperspektral görüntüler 2D görüntülerden farklı alt bantlardan elde edilen daha bağımsız ve anlamlı bilgiler içerir. Bu nedenle, 3D küplerde temsil edilen hiperspektral görüntüler, kimlik sahtekarlığı saldırıları için de ideal olan sınıflandırma işlemlerinde çok daha yeteneklidir. Bu tezde, yüz hiperspektral görüntü tanıma için öznitelik çıkarımı için yeni yöntemler öneriyoruz. Tezin temel amacı, hiperspektral yüz görüntülerinin tanınma doğruluğunu arttırmaktır. İlk yöntemde, ayrık dalgacık ayrışımı ile oluşturulan alt bantlardan öznitelikler elde etmek için 3B ayrık dalgacık dönüşümü (3D-DWT) kullanılarak üç farklı yaklaşım önerilmektedir. Önerilen üç yaklaşım, 3B alt bant enerjisi (3D-SE), 3B alt bant örtüşen küp (3D-SOC) ve 3B küresel enerji (3D-GE) olup, farklı dalgacık alt bantlarından hesaplanan enerji değerlerini içeren her yaklaşım için farklı öznitelik vektörü çıkarılır. Üç farklı öznitelik vector sınıflandırıcıdan geçirilerek yüz tanıma işlemi tamamlanmaktadır. Önerilen ikinci yöntemde, spektral bilginin tek bir 2B görüntüye füzyonu, banda özgü sinyal/gürültü oranı (SNR) tabanlı ağırlıklandırma ile elde edilir. Füzyon yöntemi, hesaplanan banda özgü SNR değerlerine dayalı olarak ağırlıklar atanarak, bantların ağırlıklı toplamı tek bir 2B yüz görüntüsü oluşturulur. Üçüncü yöntemde, hiperspektral yüz küplerindeki spektral bantları birleştirmek için, her piksel için spektral vektör 3B giriş spektralini tek bir piksele dönüştürülene kadar spektral eksen boyunca her piksele ayrı dalgacık dönüşümü (DWT) uygulayarak, yüz küpü bir 2B çıktı görüntüsüne dönüştürülür. İkinci ve üçüncü yöntemlerle elde edilen 2B çıktı görüntüleri Ana bileşenler analizi yöntemi ile dönüştürülerek bir sınıflandırıcı yardımı ile yüz tanıma işlemi gerçekleştirilmektedir. Deneysel sonuçlar, standart hiperspektral veri tabanları kullanılarak önerilen tüm yöntemlerin doğruluğunun, modern yöntemlerin alternatif hiperspektral yüz tanıma işleminden daha iyi performans gösterdiğini ortaya koymaktadır. Anahtar Kelimeler: hiperspektral yüz görüntüsü, yüz tanıma, ayrık dalgacık dönüşümü, özellik çıkarımı, sınıflandırma, sinyal-gürültü oranı. Doctor of Philosophy in Electrical and Electronic Engineering. Thesis (Ph.D.)--Eastern Mediterranean University, Faculty of Engineering, Dept. of Electrical and Electronic Engineering, 2020. Supervisor: Prof. Dr. Hasan Demirel

This article contains data related to the research article entitle 'Multi-modal Deep Learning for Fruit Detection Using RGB-D Cameras and their Radiometric Capabilities' [1]. The development of reliable fruit detection and localization systems is essential for future sustainable agronomic management of high-value crops. RGB-D sensors have shown potential for fruit detection and localization since they provide 3D information with color data. However, the lack of substantial datasets is a barrier for exploiting the use of these sensors. This article presents the KFuji RGBDS database which is composed by 967 multi-modal images of Fuji apples on trees captured using Microsoft Kinect v2 (Microsoft, Redmond, WA, USA). Each image contains information from 3 different modalities: color (RGB), depth (D) and range corrected IR intensity (S). Ground truth fruit locations were manually annotated, labeling a total of 12,839 apples in all the dataset. The current dataset is publicly available at http://www.grap.udl.cat/publicacions/datasets.html. This work was partly funded by the Secretaria d’Universitats i Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya, the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund (ERDF) under Grants 2017 SGR 646, AGL2013-48297-C2-2-R and MALEGRA, TEC2016-75976-R. The Spanish Ministry of Education is thanked for Mr. J. Gené’s pre-doctoral fellowships (FPU15/03355). We would also like to thank Nufri and Vicens Maquinària Agrícola S.A. for their support during data acquisition.

El presente trabajo se ha implementado un sistema que permite obtener la señal de tally de un mezclador de vídeo ATEM de Blackmagic Design y mostrarla en tiempo real en forma de indicador luminoso. Dicha señal va a permitir discernir cuál es la fuente que se encuentra en anticipo y cuál en programa, pudiendo indicar al operador de cámara si la señal que está captando está preparada para emitirse en directo o se está emitiendo. Esta información es crítica durante una realización audiovisual y puede gestionarse también mediante dispositivos de intercomunicación de audio. Sin embargo, estos últimos se utilizan para la coordinación entre todos los equipos técnicos (sonido, iluminación, regidor, cámaras, etc.), por lo que no es conveniente congestionar este canal de comunicación. Existen múltiples sistemas en el mercado con dicho propósito que se analizan en el Estado del Arte (véase página 3). Todos ellos constan de un dispositivo que se conecta al mezclador mediante un cable Ethernet, extraen la información de tally del bus de datos y en base a esta, actúan sobre los indicadores luminosos de los dispositivos receptores ubicados en las cámaras. Tanto el software como el hardware del sistema realizado están basados en código abierto, es funcional y utiliza tecnología inalámbrica que facilita el montaje y la movilidad. Además el sistema es adaptativo, ya que selecciona el canal Wi-Fi con menos interferencia, permite adaptar la luminosidad de los indicadores en función de las condiciones lumínicas del entorno, así como asociar los dispositivos receptores con un número de cámara sin necesidad de modificar el código. Para este propósito, se han estudiado los comandos del protocolo de red utilizado por el mezclador ATEM y las librerías Arduino de código abierto que proveen de las funciones para conectar y controlar los mezcladores de video ATEM, obtenidos por la empresa SKAARHOJ a través de ingeniería inversa. En base dichos recursos, se ha utilizado el software Arduino IDE (Integrated Development Environment), un entorno de desarrollo integrado multiplataforma cuyo lenguaje está basado en wiring, que a su vez lo está en Processing y cuyas librerías están escritas en C/C++. En cuanto a hardware, se han utilizado placas de desarrollo con un SoC (System on chip) Wi-Fi ESP8266 de Espressif Systems y un módulo ESP-12. Se han realizado los scripts necesarios para obtener la señal de tally del mezclador y distribuirla a otros dispositivos mediante el estándar 802.11 b/g/n de redes Wi-Fi que opera en la banda de 2.4 GHz. Así mismo, se ha diseñado una placa de circuito impreso mediante el software de diseño electrónico KiCad para integrar y conexionar los componentes que conforman el receptor final de la señal de tally. Este cuenta con dos indicadores led, un potenciómetro para regular la intensidad luminosa de estos, un interruptor eléctrico DIP (del inglés: Dual In Line Package) para determinar a qué número de cámara corresponde y un duplicador USB que se utiliza para proporcionar la alimentación necesaria al receptor. Además, el dispositivo dispone de un adaptador para acoplarse a cualquier cámara con zapata para flash.

We propose a technique for coherently co-clustering uncalibrated views of a scene with a contour-based representation. Our work extends the previous framework, an iterative algorithm for segmenting sequences with small variations, where the partition solution space is too restrictive for scenarios where consecutive images present larger variations. To deal with a more flexible scenario, we present three main contributions. First, motion information has been considered both for region adjacency and region similarity. Second, a two-step iterative architecture is proposed to increase the partition solution space. Third, a feasible global optimization that allows to jointly process all the views has been implemented. In addition to the previous contributions, which are based on low-level features, we have also considered introducing higher level features as semantic information in the co-clustering algorithm. We evaluate these techniques on multiview and temporal datasets, showing that they outperform state-of-the-art approaches.

El objetivo de este trabajo es optimizar un programa de correlación digital de imagen. La idea consiste en hacer una primera fotografía de un objeto sin deformar, por ejemplo, de una probeta de hormigón, y a continuación, hacer una segunda fotografía después de aplicar una carga al objeto. A partir de la imagen de referencia, la imagen actual y una serie de parámetros, el programa busca la deformación que mejor correlacione las dos imágenes. Este trabajo consiste en la continuación de dos trabajos anteriores: “Development and analysis of a Numpy-based DIC application for strain calculation” por Marc Guillem Zamora Agustí y “Contribució al desenvolupament del programari de correlació digital d’imatges (DIC)” por Antonio Carreras Luque. En el primer trabajo se desarrolla la primera versión del software en Python; el segundo proyecto hace un primer intento de optimizar el programa mediante el lenguaje C++. El objetivo de este trabajo es continuar la optimización en C++, y además combinar el trabajo hecho en los dos proyectos para tener una versión del algoritmo completa y optimizada. Después del proceso de optimización descrito en esta memoria, se consigue disminuir el tiempo de ejecución en un 88,1% respecto el proyecto de Marc Guillem Zamora y un 97,3% respecto el proyecto de Antonio Carreras. Además, también se reduce el consumo de RAM en un 27,4% respecto al original. También se prepara la estructura para una futura implementación de una interfaz de usuario construida en Python.

We propose a patch sampling strategy based on a sequential Monte-Carlo method for high resolution image classification in the context of Multiple Instance Learning. When compared with grid sampling and uniform sampling techniques, it achieves higher generalization performance. We validate the strategy on two artificial datasets and two histological datasets for breast cancer and sun exposure classification., Comment: accepted at 4th International Workshop on Deep Learning for Medical Image Analysis (DLMIA), MICCAI 2018, Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer International Publishing, 2018

We introduce deep neural networks for scanpath and saliency prediction trained on 360-degree images. The scanpath prediction model called SaltiNet is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation using a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. We also show how a similar architecture achieves state-of-the-art performance for the related task of saliency map prediction. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017.

Digital Image Correlation (DIC) is a full-field, non-contact optical technique that tracks 2-D or 3-D images in order to establish a one-to-one correspondence to obtain accurate deformation data. This technique is being used more and more due to the technological improvements in computation and the rapid development of high resolution cameras. This project aims to explain, first of all, an efficient algorithm to perform a DIC analysis, mostly based on the compilation of procedures done for Ncorr, a DIC application made by Justin Blaber as a completion of his master’s degree. The algorithm uses the inverse compositional Gauss-Newton method (IC-GN) and interpolates the images by means of B-splines in a sophisticated way. An attempt has been made to describe the mathematical framework deeply enough to serve as the basis for developing an application. A parallel objective of the project is, therefore, developing the application. The software is written in Python, but using mainly the scientific library NumPy, along with other C-based packages (SciPy, Cython, Matplotlib and PyInstaller) which allow, among other features, to manipulate image files and generate plots. Moreover, the vectorization technique, which has been necessary to speed up the code to a reasonable level, is outlined. Finally, a brief set of tests are carried out to verify the accuracy and the reliability of the developed application, comparing the results with those obtained using the commercial package GOM Correlate. In the light of these results, it is clear that the application works appropriately, although further modifications need to be applied to improve the effectiveness of showing the strain data. Furthermore, another purpose is that the project goes ahead, since the application is quite basic, but there are still some programming techniques and other DIC improvements that could be tested and implemented. For this reason, a set of next steps have been listed at the end of this report.

This thesis consists on developing an interactive lifelog search engine for the LSC 2018 search challenge at ACM ICMR 2018. This search engine is created in order to browse for images from a given lifelog dataset and display them along with some written information related to them and four other images providing contextualization about the searched one. First of all, the work makes an introduction to the relevance of this project. It introduces the reader to the main social problems affronted and the aim of our project to deal with them. Thus, go ahead with the scope of the project introducing to the main objectives fixed. Also, the work is gone by the actual state of the same kind of prototypes that already exist to let the reader see the differences that our project presents. After the project approach is done, it begins a travel trough the methodology and creation process, going deep in the main aspects and the explanation of every election and decision, also remarking the limits of the current prototype. Additionally, the project concludes with a result section where the system is tested with six users. They are asked to find three specific images using the search engine. This test is divided in two sections: first, a qualitative section where the user is asked to test the system and fill out a survey to see how comfortable it is for him. And a second section, more quantitative, where they value the speed of our system. Finally, the project concludes going through the actual and future ethics of lifelogging in general and with a final conclusion further investigation and future improvement

In this work, we present our system architecture developed to manage and pre- and post-process TIR (thermal infrared) imagery for wildfire applications. It is mainly composed of an acquisition module, a preprocessing unit and a GUI graphical interface for post-processing analysis. Specific data-acquisition software was developed to overcome the limitations of the applications provided by camera manufacturers. This data acquisition module was complemented with a pre-processing unit to deal with those videos already recorded using manufacturers’ software. This pre-processing unit converts stored IR raw files (which are unreadable) into a standard format that allows an overall thermal analysis. Our system can handle video formats used by two of the most important IR camera providers, i.e. Optris and FLIR. Finally, a graphical interface application provides the user with a user-friendly menu to detect fire edge position at desired time steps (i.e. automated perimeter tracking), to calculate rate of spread maps and interact with the data-driven forecasting module. This system makes two important contributions. First, it constitutes a common framework that ensures coherence in the treatment of IR data, no matter how it was obtained. Additionally, it incorporates the image processing algorithms in which we have been working for the past years, making them easy-to-use and facilitating the analysis of their result.

In this paper we propose a novel action tube extractor for RGB-D action recognition in trimmed videos. The action tube extractor takes as input a video and outputs an action tube. The method consists of two parts: spatial tube extraction and temporal sampling. The first part is built upon MobileNet-SSD and its role is to define the spatial region where the action takes place. The second part is based on the structural similarity index (SSIM) and is designed to remove frames without obvious motion from the primary action tube. The final extracted action tube has two benefits: 1) a higher ratio of ROI (subjects of action) to background; 2) most frames contain obvious motion change. We propose to use a two-stream (RGB and Depth) I3D architecture as our 3D-CNN model. Our approach outperforms the state-of-the-art methods on the OA and NTU RGB-D datasets. © 2018 IEEE.

We introduce SaltiNet, a deep neural network for scanpath prediction trained on 360-degree images. The model is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017., Winner of the Best Scan-path Award at the Salient360!: Visual attention modeling for 360 degrees Images Grand Challenge of ICME 2017. Presented at the ICCV 2017 Workshop on Egocentric Perception, Interaction and Computing (EPIC)

In this work we propose an adversarial learning approach to generate high resolution MRI scans from low resolution images. The architecture, based on the SRGAN model, adopts 3D convolutions to exploit volumetric information. For the discriminator, the adversarial loss uses least squares in order to stabilize the training. For the generator, the loss function is a combination of a least squares adversarial loss and a content term based on mean square error and image gradients in order to improve the quality of the generated images. We explore different solutions for the up sampling phase. We present promising results that improve classical interpolation, showing the potential of the approach for 3D medical imaging super-resolution.

We introduce PathGAN, a deep neural network for visual scanpath prediction trained on adversarial examples. A visual scanpath is defined as the sequence of fixation points over an image defined by a human observer with its gaze. PathGAN is composed of two parts, the generator and the discriminator. Both parts extract features from images using off-the-shelf networks, and train recurrent layers to generate or discriminate scanpaths accordingly. In scanpath prediction, the stochastic nature of the data makes it very difficult to generate realistic predictions using supervised learning strategies, but we adopt adversarial training as a suitable alternative. Our experiments prove how PathGAN improves the state of the art of visual scanpath prediction on the iSUN and Salient360! datasets. Source code and models are available at https://imatge-upc.github.io/pathgan/, Comment: ECCV 2018 Workshop on Egocentric Perception, Interaction and Computing (EPIC). This work obtained the 2nd award in Prediction of Head-gaze Scan-paths for Images, and the 2nd award in Prediction of Eye-gaze Scan-paths for Images at the IEEE ICME 2018 Salient360! Challenge

With the recent advances of networking, sensors, and storage technologies, many mul- tidimentional datasets are being generated in various fields. These datasets are often incomplete or contaminated during the acquisition process. Recovering the missing or noise-free data from degraded observations thus becomes crucial to obtaining precise information to refer to. The aim of this thesis is towards restoration of multidimen- sional (tensor) data. Specifically, we consider three problems: 1) tensor inpainting, 2) magnetic resonance image denoising and, 3) hyperspectral image denoising. The theory of tensors has become popular in dealing with multidimensional data, due to the capability of tensors in exploiting additional structure in comparison with matrix based alternatives. The most commonly used decomposition of multidimensional data to date is higher order singular value decomposition (HOSVD). The HOSVD is an ef- ficient way for eliciting intrinsic structure of multidimensional data. It offers a simple, adaptive and natural way to exploit sparsity among all dimensions of multidimensional data. The HOSVD decomposes a particular tensor data into the product of a sparse ten- sor and a few orthogonal matrices, each of which captures the subspace information corresponding to one dimension. In this work, we solve the restoration problems by employing the HOSVD transform and by exploiting the sparsity of the multidimen- sional signals. We enforce the sparsity using iterative regularization technique, which is shown to be very effective for our problems. The first contribution of this work is employing the iterative regularization scheme for tensor inpainting. The rationale of this approach is based on an enhanced sparse rep- resentation in HOSVD domain and it uses the iterative regularization procedure for inpainting. Improved performances of this algorithm are demonstrated in our exper- iments on three dimensional tensors, taken from multi-channel (color) images, video sequences, and magnetic resonance images. The evaluation is made quantitatively in terms of peak signal-to-noise ratio and structural similarity index, and qualitatively via visual comparisons. Despite the success of magnetic resonance imaging techniques in many applications, acquisition noise is still a limiting factor for the quality and hence the usefulness of the techniques. Our second contribution is improving the application of the iterative higher order singular value decomposition framework to denoising the magnetic res- onance images. The proposed algorithm forms a single tensor from the noisy data. This tensor undergoes an HOSVD, where its sparse representation coefficients are cal- culated with respect to a set of directional orthogonal basis matrices. Denoising is achieved by iteratively applying soft thresholding on the calculated sparse representa- tion coefficients. The proposed algorithm is further enhanced with a post-process of Wiener filtering. The performance of the proposed method is evaluated using synthetic and real magnetic resonance images. Validation results and quantitative comparisons with the state-of-the-art in magnetic resonance image denoising clearly demonstrate the advantages of the proposed method. The hyperspectral data cube is considered as a three-order tensor that is able to jointly treat both the spatial and spectral dimensions. Noise in hyperspectral image can de- grade the visual quality and limit the applicability of computerized analysis processes. Hence, toward the third contribution we consider the denoising of the hyperspectral images to improve the performance of the subsequent applications. In this work, not only we use the proposed iterative higher order singular value decomposition, but also we go one step further and propose a new iterative denoising method which utilizes the advantages of the patch-based HOSVD sparse model and the iterative regularization technique. The experiments with both synthetic noisy data and real hyperspectral data reveal that the proposed iterative algorithm improves the hyperspectral data quality in terms of both quality metrics and visual inspection. The subsequent classification re- sults further validate the effectiveness of the proposed hyperspectral noise reduction algorithm. In conclusion, extensive experiments on synthetic and real world datasets have shown the competitive performance of the proposed algorithms for inpainting, magnetic reso- nance image denoising, and hyperspectral image denoising over existing state-of-the- art ones. Keywords: Denoising, higher order singular value decomposition, hyperspectral, iter- ative regularization, MR images, patch-based, soft thresholding, sparse representation, Tucker decomposition. ÖZ: Son zamanlarda ağ oluşturma, sensörler, ve depolama teknolojileri alanında yaşanan ilerleme ve gelişmeler ile birlikte farklı alanlarda birçok çok-boyutlu veri seti oluşturulmuştur. Bu veri setleri çoğunlukla veri toplama süreci boyunca eksik kalmakta veya bozulmaktadır. Bu neden ile parazit içermeyen verilerin eksilen gözlemleme verilerinden geriye alınması, atıfta bulunmak üzere dakik bilgilerin elde edilmesi açısından oldukça önemlidir. Bu tez çalışmasının amacı çok-boyutlu (tensör) verilerin restorasyonu ve yeniden yapılandırılmasından ibarettir. Bu tez çalışmasında 3 problem özellikle dikkate alınmaktadır: 1) Tensör içboyaması, 2) manyetik yankılama görüntülerinin parazit temizlemesi, 3) aşırı spektral görüntülerin parazit temizlemesi. Tensör teorisi, tensörlerin matris bazlı alternatifler ile karşılaştırıldığında ek yapıları işleme kapasitesi nedeniyle çok-boyutlu verilerin işlenmesi için popüler hale gelmiştir. Günümüzde en yaygın olarak kullanılan çok-boyutlu veri ayrıştırma yöntemi üst seviye tekil değer ayrıştırmasıdır (ÜSTDA). ÜSTDA, çok-boyutlu verilerin içsel yapısının ortaya çıkarılması için etkin bir yöntemdir. Bu yöntem, çokboyutlu verilerin tüm boyutları arasında seyrekliğin işletilmesi için sade, uyarlanabilir ve doğal bir yöntem sunmaktadır. ÜSTDA belirli bir tensör verisini seyrek bir tensör ile her biri tek bir boyuta ait altuzay bilgilerini yansıtan birkaç ortogonal matrisin çarpımı halinde ayrıştırmaktadır. Bu çalışmada ÜSTDA dönüşümünden yararlanılarak ve çok-boyutlu sinyallerin seyrekliği işletilerek çözünürlük problemleri çözülmektedir. Seyreklik, problemlerimiz açısından çok etkili olduğu gösterilen tekrarlayan düzenleme tekniği kullanılarak uygulanmaktadır. Bu çalışmanın temel katkısı tensör içboyaması için tekrarlayan düzenleme planının kullanımından ibarettir. Bu yöntemin mantığı, ÜSTDA etkinlik alanında gelişmiş bir seyrek temsile dayalı olup içboyama için tekrarlayan düzenleme yöntemini kullanmaktadır. Bu algoritmanın geliştirilmiş performansı, çok kanallı (renkli) görüntüler, video dizinleri, ve manyetik yankılama görüntülerinden alınan üç boyutlu tensörler üzerindeki deneylerimizde gösterilmiştir. Değerlendirmeler nicel olarak en yüksek sinyal gürültü oranı ve yapısal benzerlik indeksi açısından ve nitel olarak ise görsel kıyaslama aracılığı ile gerçekleştirilmiştir. Manyetik yankılama görüntülerinin birçok uygulamadaki başarısına rağmen, veri toplama parazitleri kalite açısından halen sınırlandırıcı bir faktör olup dolayısıyla bu tekniklerin yararlı olmamasına neden olmaktadır. Bu tez çalışmasının ikinci katkısı ise manyetik yankılama görüntülerinin parazitlerden arındırılmaları amacıyla tekrarlayan üst seviye tekil değer ayrıştırma çerçevesinin uygulamasının geliştirilmesinden ibarettir. Önerilen algoritma parazitli veriden tek bir tensör oluşturmaktadır. Bu tensör, seyrek temsil katsayılarının bir takım yönlü ortogonal taban matrisi bakımından hesaplandığı bir ÜSTDA’ya maruz kalmaktadır. Parazit temizleme işlemi, hesaplanan seyrek temsil katsayıları üzerinde yumuşak eşiklemenin tekrarlanarak uygulanması yolu ile elde edilmektedir. Önerilen algoritma daha sonra Wiener filtreleme ile sağlanan ileri bir işlem ile geliştirilmektedir. Önerilen yöntemin performansı sentetik ve gerçek manyetik yankılama görüntüleri kullanılarak değerlendirilmektedir. Onaylama sonuçları ve manyetik yankılama görüntülerinin parazit temizleme alanındaki en son gelişmeler ile yapılan nicel karşılaştırmalar önerilen yöntemin avantajlarını açıkça ortaya çıkarmaktadır. Aşırı spektral veri küpü, hem uzaysal hem de spektral boyutları ortaklaşa bir şekilde işleme kabiliyeti olan üç dereceli bir tensör olarak dikkate alınmaktadır. Aşırı spektral bir görüntüdeki parazit, görsel kaliteyi düşürüp bilgisayarlı analiz işlemlerinin uygulanabilirliğini sınırlandırabilmektedir. Dolayısıyla üçüncü katkı olarak bu tez çalışmasında müteakip uygulamaların performansının geliştirilmesi amacıyla aşırı spektral görüntülerin parazitten arındırılması dikkate alınmıştır. Bu tez çalışmasında yalnızca önerilen tekrarlayan üst seviye tekil değer ayrıştırması dikkate alınmamış olup aynı zamanda bir adım ileriye giderek yama bazlı ÜSTDA seyrek model ile tekrarlayan düzenleme tekniğinin avantajlarını kullanan yeni bir tekrarlayan parazit temizleme yöntemi önerilmiştir. Hem sentetik parazitli veriler hem de gerçek aşırı spektral veriler üzerinde gerçekleştirilen deneylerin sonuçları önerilen tekrarlayan algoritmanın aşırı spektral veri kalitesini hem kalite ölçüleri hem de görsel inceleme açısından geliştirdiğini ortaya çıkarmaktadır. Müteakip sınıflandırma sonuçları daha sonra önerilen aşırı spektral parazit azaltma algoritmasının etkinliğini onaylamaktadır. Sonuç olarak sentetik ve gerçek veri setleri üzerinde gerçekleştirilen kapsamlı deneyler, içboyama, manyetik yankılama görüntülerinin parazit temizlemesi ve aşırı spektral görüntülerin parazit temizlemesi için önerilen algoritmaların ilgili alanlarda mevcut olan en son gelişmeler ile karşılaştırıldığında rekabet edebilir performansa sahip olduğunu göstermiştir. Anahtar Kelimeler: Parazit temizleme, üst seviye tekil değer ayrıştırması, aşırı spektral, tekrarlayan düzenleme, MR görüntüleri, yama bazlı, yumuşak eşikleme, seyrek temsil, Tucker ayrıştırması. Doctor of Philosophy in Electrical and Electronic Engineering. Thesis (Ph.D.)--Eastern Mediterranean University, Faculty of Engineering, Dept. of Electrical and Electronic Engineering, 2018. Co-Supervisor: Prof. Dr. Hasan Demirel, Supervisor: Prof. Dr. Runyi Yu.

In this paper we propose a patch sampling strategy based on sequential Monte-Carlo methods for Whole Slide Image classification in the context of Multiple Instance Learning and show its capability to achieve high generalization performance on the differentiation between sun exposed and not sun exposed pieces of skin tissue.

Evaluating image retrieval systems in a quantitative way, for example by computing measures like mean average precision, allows for objective comparisons with a ground-truth. However, in cases where ground-truth is not available, the only alternative is to collect feedback from a user. Thus, qualitative assessments become important to better understand how the system works. Visualizing the results could be, in some scenarios, the only way to evaluate the results obtained and also the only opportunity to identify that a system is failing. This necessitates developing a User Interface (UI) for a Content Based Image Retrieval (CBIR) system that allows visualization of results and improvement via capturing user relevance feedback. A well-designed UI facilitates understanding of the performance of the system, both in cases where it works well and perhaps more importantly those which highlight the need for improvement. Our open-source system implements three components to facilitate researchers to quickly develop these capabilities for their retrieval engine. We present: a web-based user interface to visualize retrieval results and collect user annotations; a server that simplifies connection with any underlying CBIR system; and a server that manages the search engine data. The software itself is described in a separate submission to the ACM MM Open Source Software Competition.

Metric Access Methods (MAMs) are indexing techniques which al- low working in generic metric spaces. Therefore, MAMs are specially useful for Content-Based Image Retrieval systems based on features which use non Lp norms as similarity measures. MAMs naturally allow the design of image browsers due to their inherent hierarchical structure. The Hierarchical Cellular Tree (HCT), a MAM-based indexing technique, provides the starting point of our work. In this paper, we describe some limitations detected in the original formulation of the HCT and propose some modi cations to both the index building and the search algorithm. First, the covering radius, which is de ned as the distance from the representative to the furthest element in a node, may not cover all the elements belonging to the node's subtree. Therefore, we pro- pose to rede ne the covering radius as the distance from the representative to the furthest element in the node's subtree. This new de nition is essen- tial to guarantee a correct construction of the HCT. Second, the proposed Progressive Query retrieval scheme can be redesigned to perform the nearest neighbor operation in a more e cient way. We propose a new retrieval scheme which takes advantage of the bene ts of the search algorithm used in the index building. Furthermore, while the evaluation of the HCT in the original work was only subjective, we propose an objective evaluation based on two aspects which are crucial in any approximate search algorithm: the retrieval time and the retrieval accuracy. Finally, we illustrate the usefulness of the proposal by presenting some actual applications.

Image retrieval in realistic scenarios targets large dynamic datasets of unlabeled images. In these cases, training or fine-tuning a model every time new images are added to the database is neither efficient nor scalable. Convolutional neural networks trained for image classification over large datasets have been proven effective feature extractors for image retrieval. The most successful approaches are based on encoding the activations of convolutional layers, as they convey the image spatial information. In this paper, we go beyond this spatial information and propose a local-aware encoding of convolutional features based on semantic information predicted in the target image. To this end, we obtain the most discriminative regions of an image using Class Activation Maps (CAMs). CAMs are based on the knowledge contained in the network and therefore, our approach, has the additional advantage of not requiring external information. In addition, we use CAMs to generate object proposals during an unsupervised re-ranking stage after a first fast search. Our experiments on two public available datasets for instance retrieval, Oxford5k and Paris6k, demonstrate the competitiveness of our approach outperforming the current state-of-the-art when using off-the-shelf models trained on ImageNet. The source code and model used in this paper are publicly available at http://imatge-upc.github.io/retrieval-2017-cam/., Comment: To appear in the British Machine Vision Conference (BMVC), September 2017

The increasing availability of affect-rich multimedia resources has bolstered interest in understanding sentiment and emotions in and from visual content. Adjective-noun pairs (ANP) are a popular mid-level semantic construct for capturing affect via visually detectable concepts such as "cute dog" or "beautiful landscape". Current state-of-the-art methods approach ANP prediction by considering each of these compound concepts as individual tokens, ignoring the underlying relationships in ANPs. This work aims at disentangling the contributions of the `adjectives' and `nouns' in the visual prediction of ANPs. Two specialised classifiers, one trained for detecting adjectives and another for nouns, are fused to predict 553 different ANPs. The resulting ANP prediction model is more interpretable as it allows us to study contributions of the adjective and noun components. Source code and models are available at https://imatge-upc.github.io/affective-2017-musa2/ ., Oral paper at ACM Multimedia 2017 Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes (MUSA2)

With the rapid increase of users of wearable cameras in recent years and of the amount of data they produce, there is a strong need for automatic retrieval and summarization techniques. This work addresses the problem of automatically summarizing egocentric photo streams captured through a wearable camera by taking an image retrieval perspective. After removing non-informative images by a new CNN-based filter, images are ranked by relevance to ensure semantic diversity and finally re-ranked by a novelty criterion to reduce redundancy. To assess the results, a new evaluation metric is proposed which takes into account the non-uniqueness of the solution. Experimental results applied on a database of 7,110 images from 6 different subjects and evaluated by experts gave 95.74% of experts satisfaction and a Mean Opinion Score of 4.57 out of 5.0. Source code is available at https://github.com/imatge-upc/egocentric-2017-lta, Comment: Oral paper at the ACM Multimedia 2017 Workshop on Lifelogging Tools and Applications (LTA), Mountain View, California USA. http://lta2017.computing.dcu.ie/

La detecció de prominència d’un espai visual és una tasca que tradicionalment ha estat abordada mitjançant l’anàlisi de l’espai amb tècniques de processat d’imatge, aquestes tècniques es basen a estudiar característiques com el color, la intensitat, la forma, l’orientació entre d’altres per acabar generant un mapa d’intensitats on cada punt representa un nivell d’interès. De forma alternativa, en aquest treball es pretén analitzar el moviment de persones dins un espai per tal de detectar la prominència generada per un grup d’objectes. La primera fase del treball ha consistit a captar vídeos de persones durant una exposició a un espai amb objectes d’interès i extreure informació dels mateixos obtenint diferents conjunts de dades. La segona part, ha estat basada en el desenvolupament i avaluació de mètodes que utilitzen els conjunts de dades anteriors per tal d’estimar el nivell d’interès aportat per cada objecte. Els resultats obtinguts mostren com un dels mètodes s’acosta força a unes estadístiques realitzades en la primera fase. La detección de prominencia de un espacio visual es una tasca que tradicionalmente ha sido abordada mediante el análisis del espacio con técnicas de procesado de imagen,estas técnicas están basadas en estudiar características como el color, la intensidad, la forma, la orientación entre otras para finalmente generar un mapa de intensidades donde cada punto representa un nivel de interés. De forma alternativa, en este trabajo se pretende analizar el movimiento de persones dentro de un espacio para poder detectar la prominencia generada por un grupo de objetos. La primera fase del trabajo ha consistido en captar vídeos de personas durante una exposición a un espacio con objetos de interés i extraer información de los mismos obteniendo diferentes conjuntos de datos. La segunda parte, ha sido basada en el desarrollo i avaluación de métodos que utilizan los conjuntos de datos anteriores con el fin de detectar el nivel de interés aportado por cada objeto. Los resultados obtenidos muestran como uno de los métodos se acerca bastante a unas estadísticas realizadas en la primera fase. The salience detection of a visual space is a task that traditionally has been solved by the analysis of the space with image processing techniques, these techniques are based on the study of specifications like color, intensity, orientation and so on to finally end with the generation of an intensity map where every point represents a level of interest. Alternatively, this project aims to analyze the movement of people in a room with the purpose of detecting the salience generated by a set of objects.The first phase of the project has consisted of recording people during an exposition in a space with objects of interest and extract information from those in order to get different data sets. The second phase, has been based on the development and evaluation of methods that use the previous data sets to estimate the interest level contributed by each object. The results show that one of the methods is similar to the statistics made in the first phase.

Aquest treball consisteix en l’estudi de subsidència amb el processat d’imatges amb el mètode DInSAR de les zones de Sallent i Súria on es du a terme una activitat minera subterrània. Les dades de partida que es faran servir són les imatges proporcionades pel satèl·lit SENTINEL-1 i les coordenades cartesianes del GPS proporcionades per la Càtedra ICL en Mineria Sostenible de l’Escola Politècnica Superior d’Enginyeria de Manresa. Aquests punts són fruits de mesures realitzades durant els anys 2014-2016 i els que es realitzen durant l’any 2017, les dades s’han realitzat en campanyes topogràfiques amb mesures de GPS per obtenir les coordenades X, Y i Z dels punts situats en zones que presenten principalment deformacions del tipus vertical. Per l’estudi de la subsidència de la zona es farà servir el processat amb el programa SUBSIDENCEGUI a partir d’imatges del satèl·lit SENTINEL-1 en els dos sentit d’adquisició, l’origen del software està a càrrec del Departament de Teoria i Senyal de la UPC i millorat per donar solucions en l’àmbit de mineria per l’empresa DARES TECHNOLOGY. El procediment a tractar serà el següent, primer el processat de les imatges ascendent i descendent per separat, després calcular l’estimació lineal de la velocitat de deformació i finalment calcular la velocitat no lineal per així obtenir la deformació total. El primer objectiu d’aquest treball consisteix a validar els resultats de la velocitat deformació InSAR amb els valors calculats amb les campanyes de GPS, fent servir els dos mètodes podem millorar en l’eficiència i control de les campanyes del GPS i fer-ho cada any. Un aspecte important a considerar és la presència de vegetació i camps de conreu que són els principals obstacles quan apliquem aquesta tècnica i també l’inconvenient per no tenir la superposició perfecta en temps exacte entres les dades calculades amb el GPS i l’adquisició d’imatges del satèl·lit per una de les campanyes de GPS. El segon objectiu consisteix a calcular l’angle límit i així poder representar la cubeta d’enfonsament a partir de les dades del límit de les labors de l’explotació amb les corbes de nivell de subsidència obtingudes amb el GPS. El tercer objectiu consisteix a trobar la relació entre el mètode d’explotació de cambres i pilars amb la subsidència que presenta la superfície. Finalment, s’explica en les conclusions els objectius assolits i els inconvenients que s’ha trobat per realitzar aquest treball, també s’explicarà els aspectes per millorar i projectes d’estudi de cara al futur.

Optical Character recognition is used widely as a tool in intelligent transportation systems for recognition of the car license plate from a still image or video. The accuracy of Optical Character Recognition partially depends on the quality of the input image. In this study, a set of simple and efficient methods are proposed to improve the quality of the car license plate image extracted from video clips to reduce the error rate for the license plate OCR even at low resolutions. Mean, median, and maximum filters are commonly used algorithms to filter noise and enhance an image. The proposed technique by Dr. Bodur extends them to time domain by including the pixels of the consequent images of the video clip in filtering algorithm. The OCR error rate is tested on fifty road and street video clips by decreasing the resolution of the images and filtering them with common and proposed filtering methods. The test results indicate that all proposed methods, improve the accuracy of OCR, and the highest reduction of error is obtained by the proposed temporal maximum filtering method. Keywords: License Plate Recognition, temporal image enhancement, Vehicle Plate OCR. ÖZ: Optik Karakter tanıma (OCR), akıllı ulaşım sistemlerinde hareketsiz bir görüntüdeki veya videodaki araç plakasını tanımak için yaygın olarak kullanılan bir araçtır. OCR'nın doğruluğu kısmen girilen görüntünün kalitesine bağlıdır. Bu çalışmada, düşük çözünürlüklerde bile plaka OCR’ındaki hata oranını düşürmek için video kliplerden araç plakası görüntüsü oluştururken görüntü kalitesini iyileştirmek için basit ve verimli bir dizi yöntem önerildi. Ortalama, medyan ve maksimum görüntü filtreleri, gürültüyü filtrelemek ve görüntüyü düzeltmek için yaygın olarak kullanılan algoritmalardır. Dr. Bodur tarafından önerilen teknikler video kliplerinin sonuç görüntülerinin piksellerini filtreleme algoritmasına dahil ederek uzaysal filtreleri zamana genişletmektedir. OCR hata oranı, çözünürlüğü dokuz seviyede azaltan elli yol ve sokak videosundan alınan görüntüler üzerinde yaygın kullanılan fıltreler ile önerilen filtrelerin OCR hatalarını karşılaştırarak suretiyle test edildi. Test sonuçları, önerilen tüm yöntemlerin OCR doğruluğunu iyileştirdiğini ve hata azalmasının en fazla önerilen zamansal maksimum filtreleme yöntemiyle elde edildiğini göstermektedir. Anahtar Kelimeler: Taşıt Plakası Tanıma, Zamansal görüntü iyileştirme, Araç Plakası Tanıma. Master of Science in Computer Engineering. Thesis (M.S.)--Eastern Mediterranean University, Faculty of Engineering, Dept. of Computer Engineering, 2017. Supervisor: Assoc. Prof. Dr. Mehmet Bodur.

In this thesis, we introduce a method to synthesize and recognize buildings using a set of at least two 2D images taken from different views. Based on a coarse set of affine invariant salient feature points (corner points) on the images, a 3D high-resolution building model is obtained in accordance with the observed images. Corresponding salient points are found using the ratio of triangle areas formed from a set of four consecutive ordered salient corresponding points that form two triangles. The order is obtained by finding the vertices of the convex hull of the salient points. The salient points are tessellated to form a high-resolution triangular mesh with the appearance of a triangular patch in the image imported onto the personalized 3D model. With multiple images, all coordinates and appearances are reconstructed in accordance with the observed images. The 3D model reconstruction method allows for a 3D classification of a test building to one of many possible buildings stored in the database. The classification is based on a geometric 3D point cloud error. For buildings with very close 3D point cloud errors, a further classification is achieved based on the mean squared error (MSE) on the appearance of corresponding points on the test and base models. Our method can also be used in localization when preloaded location information of each model in the database is stored, hence helping an observer navigate without a GPS system.

The use of Structural Health Monitoring (SHM) techniques is paramount to the safety and longevity of the structures. Many fields use this approach to monitor the performance of a system through time to determine the proper time and funds associated with repair and replacement. The monitoring of these systems includes nondestructive testing techniques (NDT), sensors permanently installed on the structure, and can also rely heavily on visual inspection. Visual inspection is widely used due to the level of trust owners have in the inspection personnel, however it is time consuming, expensive, and relies heavily on the experience of the inspectors. It is for these reasons that rapid data acquisition platforms must be developed using remote sensing systems to collect, process, and display data to decision makers quickly to make well informed decisions based on quantitative data or provide information for further inspection with a contact technique for targeted inspection. The proposed multirotor Unmanned Aerial System (UAS) platform carries a multispectral imaging payload to collect data and serve as another tool in the SHM cycle. Several demonstrations were performed in a laboratory setting using UAS acquired imagery for identification and measurement of structures. Outdoor validation was completed using a simulated bridge deck and ground based setups on in service structures. Finally, static laboratory measurements were obtained using multispectral patterns to obtain multiscale deformation measurements that will be required for use on a UAS. The novel multiscale, multispectral image analysis using UAS acquired imagery demonstrates the value of the remote sensing system as a nondestructive testing platform and tool for SHM.

We present a method for performing hierarchical object detection in images guided by a deep reinforcement learning agent. The key idea is to focus on those parts of the image that contain richer information and zoom on them. We train an intelligent agent that, given an image window, is capable of deciding where to focus the attention among five different predefined region candidates (smaller windows). This procedure is iterated providing a hierarchical image analysis.We compare two different candidate proposal strategies to guide the object search: with and without overlap. Moreover, our work compares two different strategies to extract features from a convolutional neural network for each region proposal: a first one that computes new feature maps for each region proposal, and a second one that computes the feature maps for the whole image to later generate crops for each region proposal. Experiments indicate better results for the overlapping candidate proposal strategy and a loss of performance for the cropped image features due to the loss of spatial resolution. We argue that, while this loss seems unavoidable when working with large amounts of object candidates, the much more reduced amount of region proposals generated by our reinforcement learning agent allows considering to extract features for each location without sharing convolutional computation among regions., Deep Reinforcement Learning Workshop (NIPS 2016). Project page at https://imatge-upc.github.io/detection-2016-nipsws/

Visual multimedia have become an inseparable part of our digital social lives, and they often capture moments tied with deep affections. Automated visual sentiment analysis tools can provide a means of extracting the rich feelings and latent dispositions embedded in these media. In this work, we explore how Convolutional Neural Networks (CNNs), a now de facto computational machine learning tool particularly in the area of Computer Vision, can be specifically applied to the task of visual sentiment prediction. We accomplish this through fine-tuning experiments using a state-of-the-art CNN and via rigorous architecture analysis, we present several modifications that lead to accuracy improvements over prior art on a dataset of images from a popular social media platform. We additionally present visualizations of local patterns that the network learned to associate with image sentiment for insight into how visual positivity (or negativity) is perceived by the model., Accepted for publication in Image and Vision Computing. Models and source code available at https://github.com/imatge-upc/sentiment-2016

This work presents a retrieval pipeline and evaluation scheme for the problem of finding the last appearance of personal objects in a large dataset of images captured from a wearable camera. Each personal object is modelled by a small set of images that define a query for a visual search engine.The retrieved results are reranked considering the temporal timestamps of the images to increase the relevance of the later detections. Finally, a temporal interleaving of the results is introduced for robustness against false detections. The Mean Reciprocal Rank is proposed as a metric to evaluate this problem. This application could help into developing personal assistants capable of helping users when they do not remember where they left their personal belongings., Comment: Lifelogging Tools and Applications Workshop (LTA'16) at ACM Multimedia 2016

Semantic image retrieval from large amounts of egocentric visual data requires to leverage powerful techniques for filling in the semantic gap. This paper introduces LEMoRe, a Lifelog Engine for Moments Retrieval, developed in the context of the Lifelog Semantic Access Task (LSAT) of the the NTCIR-12 challenge and discusses its performance variation on different trials. LEMoRe integrates classical image descriptors with high-level semantic concepts extracted by Convolutional Neural Networks (CNN), powered by a graphic user interface that uses natural language processing. Although this is just a first attempt towards interactive image retrieval from large egocentric datasets and there is a large room for improvement of the system components and the user interface, the structure of the system itself and the way the single components cooperate are very promising.

In this thesis we present two applications where 3D surface modeling and registration play a role of paramount importance. The first application is in archaeology, where our proposed method assists the archaeologist in the tedious process of reconstructing ceramic vessels from the excavated fragments (or shards). The method takes advantage of the surface intrinsic features coupled with a library of generic models constructed by the archaeologists to produce a virtual reconstruction of what the original vessel may have looked like. Generic models are generated based on the expert historical knowledge of the period, provenance of the artifact, and location of the digging site. The generic models need not to be identical to the original vessel, but must be within a geometric transformation of it in most of its parts. The methods are suited for ceramic vessels with distinctive color patterns or reliefs (i.e., surface with molding, carving, or stamping). We introduce a novel curvature B-spline fitting method that makes the extraction of robust features possible. The alignment of the shards against the generic model uses a novel set of 3D weighted curve moments and absolute affine invariants derived from them. The transformation is computed from corresponding parabolic contours on the shard and the generic model. When the shards have no color or relief on them, they are virtually mended to abutting shards using intrinsic differential anchor points computed on the shards break curves and a set of absolute invariants. For axially symmetric vessels, a global constraint induced by the surface of revolution is applied to guarantee global mending consistency. In our experiment involving around 300 shards, more than 85% of them are successfully mended. The mended shards consist of over 90% surface areas of the ceramic vessels. The second application is in 3D human face synthesis and recognition, where our proposed method assists the national security agencies in constructing a 3D face database to be used for spotting missing personnel or identify possible suspects when close up images of that person are flagged up. A gender/race generic 3D face model is morphed into a personalized unique 3D face of a given person from using a limited number of 2D images of the given person taken from different viewpoints. The 2D images need not to be of the same resolution, or taken under similar brightness or same place, or even same time period. The method applies also to drawing sketches of the person in question. The reconstructed 3D personalized model is an accurate rendition of the actual human face, and the reconstruction errors are smaller than a 3D scanner resolution that is used to scan the person's face. Our method renders a difficult 2D face recognition problem into a much simpler 3D surface registration (alignment) problem. Given a set of images of a test person, a personalized 3D face model is constructed and "best" aligned with one of the 3D faces stored in the database. When the minimum alignment errors are below an accepted threshold (lower than the scanner resolution), the identity of the test person is matched with the one in the database. Experimental results are shown in controlled environment with high resolution images, under ideal illumination condition, as well as for images taken in uncontrolled environment with low resolution, arbitrary illumination. In both conditions, the recognition rate is over 95%. It is also proved that better face recognition rate is achieved by using a gender/race based generic 3D face model which allows for accurate pose estimation as well as the face identification in 3D.

This thesis project produced a Virtual Chinese Shadow Puppet System used in concert with Microsoft's Kinect V2. The Virtual Puppet System enables players to use their whole body motions to control virtual puppets instead of using rods or sticks in a traditional way. There are two modes of play in the system. Players can elect to play in a group with multiple puppets performing at the same time, or an individual player may choose to play alone. Twelve puppets were produced for the Virtual Puppet System based on classical characters from the famous Chinese fiction, "Journey to the West." The thesis includes a literature review of relevant works that influenced the project; it describes the workings of the interface and gesture control systems as well as the 2D virtual puppets produced. It also describes lessons learned from observations of a group of 7 - 15 year olds studying Chinese culture who played with the system, and with conversations held with a panel of experts who also observed the children using the system.

The organisation of personal data is receiving increasing research attention due to the challenges we face in gathering, enriching, searching, and visualising such data. Given the increasing ease with which personal data being gathered by individuals, the concept of a lifelog digital library of rich multimedia and sensory content for every individual is fast becoming a reality. The LTA 2016 workshop aims to bring together academics and practitioners to discuss approaches to lifelog data analytics and applications; and to debate the opportunities and challenges for researchers in this new and challenging area.

The future of wind energy industry passes through the use of larger and more flexible wind turbines in remote locations, which are increasingly offshore to benefit stronger and more uniform wind conditions. Cost of operation and maintenance of offshore wind turbines is among 15-35% of the total cost. From this, 80% comes from unplanned maintenance due to different faults in the wind turbine components. Thus, an auspicious way to contribute to the increasing demands and challenges is by applying low-cost advanced fault detection schemes. This work proposes a new method for fault detection of wind turbine actuators and sensors faults in variable-speed wind turbines. For this purpose, time domain signals acquired from the operating wind turbine are converted into two-dimensional matrices to obtain gray-scale digital images. Then, the image pattern recognition is processed getting texture features under a multichannel representation. In this work, four types of texture features are used: statistical, wavelet, granulometric and Gabor features. Then, the most significant features are selected with the conditional mutual criterion. Finally, the fault detection is performed using an automatic classification tool. In particular, a 10-fold cross validation is used to obtain a more generalized model and evaluate the classification performance. In this way, the healthy and faulty conditions of the wind turbine can be detected. Coupled non-linear aero-hydro-servo-elastic simulations of a 5MW offshore type wind turbine are carried out for several fault scenarios. The results show a promising methodology able to detect the most common wind turbine faults.

In 3D video, view synthesis is used to create new virtual views between encoded camera views. Errors in the coding of the depth maps introduce geometry inconsistencies in synthesized views. In this paper, a new 3D plane representation of the scene is presented which improves the performance of current standard video codecs in the view synthesis domain. Two image segmentation algorithms are proposed for generating a color and depth segmentation. Using both partitions, depth maps are segmented into regions without sharp discontinuities without having to explicitly signal all depth edges. The resulting regions are represented using a planar model in the 3D world scene. This 3D representation allows an efficient encoding while preserving the 3D characteristics of the scene. The 3D planes open up the possibility to code multiview images with a unique representation.

This paper extends our previous work on the potential of EEG-based brain computer interfaces to segment salient objects in images. The proposed system analyzes the Event Related Potentials (ERP) generated by the rapid serial visual presentation of windows on the image. The detection of the P300 signal allows estimating a saliency map of the image, which is used to seed a semi-supervised object segmentation algorithm. Thanks to the new contributions presented in this work, the average Jaccard index was improved from 0.47 to 0.66 when processed in our publicly available dataset of images, object masks and captured EEG signals. This work also studies alternative architectures to the original one, the impact of object occupation in each image window, and a more robust evaluation based on statistical analysis and a weighted F-score.

Building a visual summary from an egocentric photostream captured by a lifelogging wearable camera is of high interest for different applications (e.g. memory reinforcement). In this paper, we propose a new summarization method based on keyframes selection that uses visual features extracted by means of a convolutional neural network. Our method applies an unsupervised clustering for dividing the photostreams into events, and finally extracts the most relevant keyframe for each event. We assess the results by applying a blind-taste test on a group of 20 people who assessed the quality of the summaries., Paper accepted in the IEEE First International Workshop on Wearable and Ego-vision Systems for Augmented Experience (WEsAX). Turin, Italy. July 3, 2015

Spline based deformation methods are common in medical imaging due to their robustness and accuracy. These methods require a signi cant amount of computation time in order to obtain a sufficiently accurate spatial deformation between two or more image volumes. This work focuses on comparing the thin-plate spline registration technique against the bspline on a variety of hardware architectures. The architectures in question include the common i5/i7 architecture with single and multiple threads, nVidia's CUDA architecture on the GTX760, and Intel's Xeon Phi architecture.

The search of images on the internet has become a natural process for the internet surfer. Most of the search engines use complex algorithms to look up for images but their metadata is mostly ignored, in part because many image hosting sites remove metadata when the image is uploaded. The JPSearch standard has been developed to handle interoperability in metadata based searches, but it seems that the market is not interested on supporting it. The starting point of this project is a broker based federated search for image retrieval which was supposed to be the backbone of the nal software. However, due to out of date state of the previous project a new software has been developed. The objective of this approach is to support scalability on an image search system based on querying to the API of well known image hosting services and providing a tool to analyse images metadata for low level searches. A design is proposed to dodge the API rate limitations, the actual major problem regarding scalability, and a working software is developed to prove its viability. A set of tests are introduced to evaluate the performance of the approach and its results are interpreted. Finally a future work is suggested in order to improve the weaknesses.

This paper explores novel approaches for improving the spatial codification for the pooling of local descriptors to solve the semantic segmentation problem. We propose to partition the image into three regions for each object to be described: Figure, Border and Ground. This partition aims at minimizing the influence of the image context on the object description and vice versa by introducing an intermediate zone around the object contour. Furthermore, we also propose a richer visual descriptor of the object by applying a Spatial Pyramid over the Figure region. Two novel Spatial Pyramid configurations are explored: Cartesian-based and crown-based Spatial Pyramids. We test these approaches with state-of-the-art techniques and show that they improve the Figure-Ground based pooling in the Pascal VOC 2011 and 2012 semantic segmentation challenges., Comment: Paper accepted at the IEEE International Conference on Image Processing, ICIP 2015. Quebec City, 27-30 September. Project page: https://imatge.upc.edu/web/publications/improving-spatial-codification-semantic-segmentation