Search

In this work, we study the application of multimodal analogical reasoning to image retrieval. Multimodal analogy questions are given in a form of tuples of words and images, e.g., "cat":"dog"::[an image of a cat sitting on a bench]:?, to search for an image of a dog sitting on a bench. Retrieving desired images given these tuples can be seen as a task of finding images whose relation between the query image is close to that of query words. One way to achieve the task is building a common vector space that exhibits analogical regularities. To learn such an embedding, we propose a quadruple neural network called multimodal siamese network. The network consists of recurrent neural networks and convolutional neural networks based on the siamese architecture. We also introduce an effective procedure to generate analogy examples from an image-caption dataset for training of our network. In our experiments, we test our model on analogy-based image retrieval tasks. The results show that our method outperforms the previous work in qualitative evaluation. [ABSTRACT FROM AUTHOR]

The Human decision-making process works by recollecting past sequences of observations and using them to decide the best possible action in the present. These past sequences of observations are stored in a derived form which only includes important information the brain thinks might be useful in the future, while forgetting the rest. we propose an architecture that tries to mimic the human brain and improve the memory efficiency of transformers by using a modified TransformerXL architecture which uses Automatic Chunking which only attends to the relevant chunks in the transformer block. On top of this, we use ForgetSpan which is technique to remove memories that do not contribute to learning. We also theorize the technique of Similarity based forgetting to remove repetitive memories. We test our model in various tasks that test the abilities required to perform well in a human–robot collaboration scenario. [ABSTRACT FROM AUTHOR]

Artificial intelligence is nowadays used for cell detection and classification in optical microscopy during post‐acquisition analysis. The microscopes are now fully automated and next expected to be smart by making acquisition decisions based on the images. It calls for analysing them on the fly. Biology further imposes training on a reduced data set due to cost and time to prepare the samples and have the data sets annotated by experts. We propose a real‐time image processing compliant with these specifications by balancing accurate detection and execution performance. We characterised the images using a generic, high‐dimensional feature extractor. We then classified the images using machine learning to understand the contribution of each feature in decision and execution time. We found that the non‐linear‐classifier random forests outperformed Fisher's linear discriminant. More importantly, the most discriminant and time‐consuming features could be excluded without significant accuracy loss, offering a substantial gain in execution time. It suggests a feature‐group redundancy likely related to the biology of the observed cells. We offer a method to select fast and discriminant features. In our assay, a 79.6 ± 2.4% accurate classification of a cell took 68.7 ± 3.5 ms (mean ± SD, 5‐fold cross‐validation nested in 10 bootstrap repeats), corresponding to 14 cells per second, dispatched into eight phases of the cell cycle, using 12 feature groups and operating a consumer market ARM‐based embedded system. A simple neural network offered similar performances paving the way to faster training and classification, using parallel execution on a general‐purpose graphic processing unit. Finally, this strategy is also usable for deep neural networks paving the way to optimizing these algorithms for smart microscopy. [ABSTRACT FROM AUTHOR]

In the past few decades, many image quality indices have been developed. However, they stem from different theoretical frameworks, application scenarios and purposes. Thus, users and researchers are often faced with the time-consuming task of deciding which quality index to choose when they require a reliable image quality index that is capable of emulating the human visual system (HVS). In this work, general criteria for selecting the most appropriate index from a given set of quality indices according to application needs are established. These criteria are based on the statistical coefficients of correlation and concordance. It is discussed why Kendall's Tau and Spearman's rank correlation coefficients--which are widely used to compare quality index performance--are not sufficient for this purpose; moreover, additional nonparametric tests and methods of agreement are incorporated: the concordance coefficients (Kendall's w, Cohen's kappa, Scott's pi and Fleiss' kappa) not explored so far, to determine the best procedures to compare digital images. The combination of all these strategies led to a more complete comparison method, from which a ranking of quality indices could be generated from any set of them. As an application, the performance and suitability of a large number of quality indices for various real-world scenarios is compared. Our experiments reveal that the indices are sensitive to the type of distortions. This work expanded previous studies by incorporating directional indices, which perform well in the numerical experiments developed using real datasets. [ABSTRACT FROM AUTHOR]

In our constantly monitored world, surveillance cameras play a crucial role in curbing crime and violence in public spaces by serving as a deterrent. To enhance their effectiveness, there is a growing need for automated tools that can detect crimes in real time. In this paper, we propose a novel deep learning architecture that accurately and efficiently detects violent crimes in surveillance videos. We rely on what we believe are the most essential pieces of information to detect violence, namely: human bodies and their interaction. To this end, we employ human pose extractors and change detectors as the input of our proposal. Subsequently, we combine them using a novel method, which relies on additions instead of multiplications to guarantee the transmission of information even when one of the inputs provides a zero-valued signal; outperforming other combination alternatives of the literature. Finally, to account for both spatial and temporal information, we use a convolutional alternative of the standard LSTM, the ConvLSTM. The experiments performed on several benchmark datasets demonstrate the efficacy and efficiency of our proposal, achieving state-of-the-art results with much fewer trainable parameters. We release the code to replicate the proposed architecture at https://github.com/atmguille/Violence-Detection-With-Human-Skeletons. • Use of human skeletons and change detection to efficiently detect violence. • Novel proposal to combine pipelines that guarantees the transmission of information. • Use of a ConvLSTM to aggregate spatio-temporal information. • The proposal achieves SOTA results with 5x fewer parameters than public alternatives and runs in real time. [ABSTRACT FROM AUTHOR]

Existing computer vision systems can compete with humans in understanding the visible parts of objects, but still fall far short of humans when it comes to depicting the invisible parts of partially occluded objects. Image amodal completion aims to equip computers with human-like amodal completion functions to understand an intact object despite it being partially occluded. The main purpose of this survey is to provide an intuitive understanding of the research hotspots, key technologies and future trends in the field of image amodal completion. Firstly, we present a comprehensive review of the latest literature in this emerging field, exploring three key tasks in image amodal completion, including amodal shape completion, amodal appearance completion, and order perception. Then we examine popular datasets related to image amodal completion along with their common data collection methods and evaluation metrics. Finally, we discuss real-world applications and future research directions for image amodal completion, facilitating the reader's understanding of the challenges of existing technologies and upcoming research trends. • Systematically review amodal completion based on a single image. • Present a novel taxonomy to provide comprehensive insight. • Survey the latest research and public datasets and identify real-world applications. • Discuss current challenges and suggest possible avenues for future research. [ABSTRACT FROM AUTHOR]

Image-based approaches to plant phenotyping are gaining momentum providing fertile ground for several interesting vision tasks where fine-grained categorization is necessary, such as leaf segmentation among a variety of cultivars, and cultivar (or mutant) identification. However, benchmark data focusing on typical imaging situations and vision tasks are still lacking, making it difficult to compare existing methodologies. This paper describes a collection of benchmark datasets of raw and annotated top-view color images of rosette plants. We briefly describe plant material, imaging setup and procedures for different experiments: one with various cultivars of Arabidopsis and one with tobacco undergoing different treatments. We proceed to define a set of computer vision and classification tasks and provide accompanying datasets and annotations based on our raw data. We describe the annotation process performed by experts and discuss appropriate evaluation criteria. We also offer exemplary use cases and results on some tasks obtained with parts of these data. We hope with the release of this rigorous dataset collection to invigorate the development of algorithms in the context of plant phenotyping but also provide new interesting datasets for the general computer vision community to experiment on. Data are publicly available at http://www.plant-phenotyping.org/datasets . [ABSTRACT FROM AUTHOR]

Abstract: Online learning has shown to be successful in tracking-by-detection of previously unknown objects. However, most approaches are limited to a bounding-box representation with fixed aspect ratio and cannot handle highly non-rigid and articulated objects. Moreover, they provide only a limited foreground/background separation, which in turn, increases the amount of noise introduced during online self-training. To overcome the limitations of a rigid bounding box, we present a novel tracking-by-detection approach based on the generalized Hough-transform. We extend the idea of Hough Forests to the online domain and couple the voting-based detection and back-projection with a rough GrabCut segmentation. Because of the increased granularity of the object description the amount of noisy training samples during online learning is reduced significantly which prevents drifting of the tracker. To show the benefits of our approach, we demonstrate it for a variety of previously unknown objects even under heavy non-rigid transformations, partial occlusions, scale changes, and rotations. Moreover, we compare our tracker to state-of-the-art methods (bounding-box-based as well as part-based) and show robust and accurate tracking results on various challenging sequences. [Copyright &y& Elsevier]

Abstract: Live Surface allows users to segment and render complex surfaces from 3D image volumes at interactive (sub-second) rates using a novel, cascading graph cut (CGC). Live Surface consists of two phases: (1) preprocessing for generation of a complete 3D hierarchy of tobogganed regions followed by tracking of all region surfaces; (2) user interaction in which, with each mouse movement, the volume is segmented and the 3D object is rendered at interactive rates. Interactive segmentation is accomplished by cascading through the 3D hierarchy from the top, applying graph cut successively, at each level, only to regions bordering the segmented surface from the previous level. CGC allows the entire image volume to be segmented an order of magnitude faster than existing techniques that make use of graph cut. OpenGL rendering provides for display and update of the segmented surface at interactive rates. The user selects objects by tagging voxels with either foreground (object) or background seeds. Seeds can be placed on image cross-sections or directly on the 3D rendered surface. Interaction with the rendered surface improves the user''s ability to steer the segmentation, augmenting or subtracting from the current selection. Segmentation and rendering, combined, is accomplished in about 0.35s, allowing 3D surfaces to be displayed and updated dynamically as each additional seed is deposited. The immediate feedback of Live Surface allows the segmentation of 3D image volumes using an interaction paradigm similar to the Live Wire (Intelligent Scissors) tool used in 2D images. [Copyright &y& Elsevier]

The goal of image-based 3D head pose estimation is try to estimate the facial direction with 2D images. It is an important attribute widely used in many applications related to faces. However, accurate estimation is hard due to complicated part and pose absence in facial images. Recently, some improvement has been obtained with methods based on neural networks, but most of them ignore the geometric information of facial parts. In this paper, we try to tackle this issue and propose a novel geometry-aware representation. It is based on Stacked Capsule Graph Autoencoders (SCGAE). Different from current methods, we apply Stacked Capsule Autoencoders (SCAE) to encode the parts and poses of facial images. These parts and poses are used to train templates and reconstruct the original faces in decoders. In addition, we improve SCAE with locality loss, in which the inner relationships of similar samples are utilized. To achieve it, graph regularization is applied. In this way, an improved geometry-aware representation can be computed. It is compatible with existing regression methods and experimental results on commonly-used datasets about head pose estimation validate the effectiveness of SCGAE. [Display omitted] • An unsupervised method is proposed to obtain geometry-aware representations of faces. • The representation is improved by capturing the locality information among features. • An end-to-end regression is implemented to achieve 3D head pose estimation. The experimental results on two datasets verify the effectiveness. [ABSTRACT FROM AUTHOR]

Abstract: The work in this paper explores the discriminatory power of target outline description features in conjunction with Support Vector Machine (SVM) based classification committees, when attempting to recognize a variety of targets from Synthetic Aperture Radar (SAR) images. In specific, approximate target outlines are first determined from SAR images via a simple mathematical morphology-based segmentation approach that discriminates target from radar shadow and ground clutter. Next, the obtained outlines are expressed as truncated Elliptical Fourier Series (EFS) expansions, whose coefficients are utilized as discriminatory features and processed by an ensemble of SVM classifiers. In order to experimentally illustrate the merit of the proposed scheme, this work reports classification results on a 3-class target recognition problem using SAR intensity imagery from the well-known Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset. The novel approach was compared to selected methods mentioned in the literature in terms of classification accuracy. The results illustrate that only a small amount of EFS coefficients is necessary to achieve recognition rates that rival other established methods and, thus, target outline information can be a powerful discriminatory feature for automatic target recognition applications relevant to SAR imagery. [Copyright &y& Elsevier]