Search

This paper introduces a novel method that enables robots to identify objects based on user gaze, tracked via eye-tracking glasses. This is achieved without prior knowledge of the objects' categories or their locations and without external markers. The method integrates a two-part system: a category-agnostic object shape and pose estimator using superquadrics and Siamese networks. The superquadrics-based component estimates the shapes and poses of all objects, while the Siamese network matches the object targeted by the user's gaze with the robot's viewpoint. Both components are effectively designed to function in scenarios with partial occlusions. A key feature of the system is the user's ability to move freely around the scenario, allowing dynamic object selection via gaze from any position. The system is capable of handling significant viewpoint differences between the user and the robot and adapts easily to new objects. In tests under partial occlusion conditions, the Siamese networks demonstrated an 85.2% accuracy in aligning the user-selected object with the robot's viewpoint. This gaze-based Human–Robot Interaction approach demonstrates its practicality and adaptability in real-world scenarios. [ABSTRACT FROM AUTHOR]

Abstract: Latent topic models have become a popular paradigm in many computer vision applications due to their capability to unsupervisely discover semantics in visual content. Relying on the Bag-of-Words representation, they consider images as mixtures of latent topics that generate visual words according to some specific distributions. However, the performance of these methods is still limited by the way in which they take into account the spatial distribution of visual words and, what is even more important, the currently used appearance distributions. In this paper, we propose a novel region-centered latent topic model that introduces two main contributions: first, an improved spatial context model that allows for considering inter-topic inter-region influences; and second, an advanced region-based appearance distribution built on the Kernel Logistic Regressor. It is worth highlighting that the proposed contributions have been seamlessly integrated in the model, so that all the parameters are concurrently estimated using a unified inference process. Furthermore, the proposed model has been extended to work in both unsupervised and supervised modes. Our results for unsupervised mode improve 30% those of previous latent topic models. For supervised mode, where discriminative approaches are preponderant, our results are quite close to those of discriminative state-of-the-art methods. [Copyright &y& Elsevier]

Abstract: The use of feature enhancement techniques to obtain estimates of the clean parameters is a common approach for robust automatic speech recognition (ASR). However, the decoding algorithm typically ignores how accurate these estimates are. Uncertainty decoding methods incorporate this type of information. In this paper, we develop a formulation of the uncertainty decoding paradigm for Frequency Filtered (FF) parameters using spectral subtraction as a feature enhancement method. Additionally, we show that the uncertainty decoding method for FF parameters admits a simple interpretation as a spectral weighting method that assigns more importance to the most reliable spectral components. Furthermore, we suggest combining this method with SSBD-HMM (Spectral Subtraction and Bounded Distance HMM), one recently proposed technique that is able to compensate for the effects of features that are highly contaminated (outliers). This combination pursues two objectives: to improve the results achieved by uncertainty decoding methods and to determine which part of the improvements is due to compensating for the effects of outliers and which part is due to compensating for other less deteriorated features. [Copyright &y& Elsevier]

Abstract: Additive noise generates important losses in automatic speech recognition systems. In this paper, we show that one of the causes contributing to these losses is the fact that conventional recognisers take into consideration feature values that are outliers. The method that we call bounded-distance HMM is a suitable method to avoid that outliers contribute to the recogniser decision. However, this method just deals with outliers, leaving the remaining features unaltered. In contrast, spectral subtraction is able to correct all the features at the expense of introducing some artifacts that, as shown in the paper, cause a larger number of outliers. As a result, we find that bounded-distance HMM and spectral subtraction complement each other well. A comprehensive experimental evaluation was conducted, considering several well-known ASR tasks (of different complexities) and numerous noise types and SNRs. The achieved results show that the suggested combination generally outperforms both the bounded-distance HMM and spectral subtraction individually. Furthermore, the obtained improvements, especially for low and medium SNRs, are larger than the sum of the improvements individually obtained by bounded-distance HMM and spectral subtraction. [Copyright &y& Elsevier]

In most video coding standards, motion estimation be- comes the most time-consuming subsystem. Consequently, in the last few years, a great deal of effort has been devoted to the re- search of novel algorithms capable of saving computations with minimal effects on the coding quality. Adaptive algorithms and particularly multipattern solutions, have evolved as the most robust general-purpose solutions owing to two main reasons: 1) real video sequences usually exhibit a wide-range of motion content, from uniform to random, and 2) a vast amount of coding applications have appeared demanding different degrees of coding quality. In this study, we propose an adaptive algorithm, called motion classification-based search (MCS), which makes use of an especially tailored classifier that detects some motion cues and chooses the search pattern that best fits them. The MCS has been experimentally assessed for a comprehensive set of selected video sequences and qualities. Our experimental results show that MCS notably reduces the computational cost up to 55% and 84% in search points, with respect to two state-of-the-art methods, while maintaining the quality. [ABSTRACT FROM AUTHOR]

The latest High Efficiency Video Coding (HEVC) standard relies on a large number of coding tools from which the encoder should choose for every coding unit. This optimization process is based on the minimization of a Lagrangian cost function that evaluates the distortion produced and the bit-rate needed to encode each coding unit. The value of the Lagrangian parameter λ , which balances the weight of the rate and distortion terms, is related to the quantization parameter through a model that has been implemented in the HEVC reference software. Nevertheless, in this paper we show that this model can be refined, especially for static background sequences, so that the coding performance of HEVC can be improved by adaptively modifying the relation between λ and the quantization parameter. Specifically, the proposed method (i) determines whether the background of a sequence is static or not by means of a simple classifier; and (ii) when static, it evaluates an exponential regression function to estimate a proper value of the λ parameter. In so doing, the proposed method becomes content-aware, being able to dynamically act on the λ parameter. Experiments conducted over a large set of static and dynamic background video sequences prove that the proposed method achieves an average bit-rate saving of − 6.72 % ( − 11.07 % for static background video sequences) compared with the reference HM16.0 software, notably outperforming the results of a state-of-the-art method. [ABSTRACT FROM AUTHOR]

The latest High Efficiency Video Coding standard (HEVC) provides a set of new coding tools to achieve a significantly higher coding efficiency than previous standards. In this standard, the pixels are first grouped into Coding Units (CU), then Prediction Units (PU), and finally Transform Units (TU). All these coding levels are organized into a quadtree-shaped arrangement that allows highly flexible data representation; however, they involve a very high computational complexity. In this paper, we propose an effective early CU depth decision algorithm to reduce the encoder complexity. Our proposal is based on a hierarchical approach, in which a hypothesis test is designed to make a decision at every CU depth, where the algorithm either produces an early termination or decides to evaluate the subsequent depth level. Moreover, the proposed method is able to adaptively estimate the parameters that define each hypothesis test, so that it adapts its behavior to the variable contents of the video sequences. The proposed method has been extensively tested, and the experimental results show that our proposal outperforms several state-of-the-art methods, achieving a significant reduction of the computational complexity (36.5% and 38.2% average reductions in coding time for two different encoder configurations) in exchange for very slight losses in coding performance (1.7% and 0.8% average bit rate increments). [ABSTRACT FROM AUTHOR]

Recently, convolutional neural networks have greatly outperformed previous systems based on handcrafted features once the size of public databases has increased. However, these algorithms learn feature representations that are difficult to interpret and analyse. On the other hand, experts require automatic systems to explain their decisions according to clinical criteria which, in the field of melanoma diagnosis, are related to the analysis of dermoscopic features found in the lesions. In recent years, the interpretability of deep networks has been explored using methods that obtain visual features highlighted by neurones or analyse activations to extract more useful information. Following the latter approach, this study proposes a system for melanoma diagnosis that explicitly incorporates dermoscopic feature segmentations into a diagnosis network through a channel modulation scheme. Modulation weights control the influence of the detected visual patterns based on the lesion content. As shown in the experimental section, our design not only improves the system performance on the ISIC 2016 (average AUC of 86.6% vs. 85.8%) and 2017 (average AUC of 94.0% vs. 93.8%) datasets, but also notably enhances the interpretability of the diagnosis, providing useful and intuitive cues to clinicians. [ABSTRACT FROM AUTHOR]

• A noise-robust loss based on Multiple Instance Learning (MIL) is used to train CNNs for retrieval under noisy datasets. • The Bag Exponential (BE) presented is flexible enough to be used for other purposes than dealing with noise, such as online hard positive mining. • Our method allows to use noisy generated training sets, which are easy and quick to create, to adapt CNNs for image retrieval on any new object. Although the CNNs are a very powerful tool for image retrieval, the need of training datasets properly adapted to the application at hand hinders the usefulness of such networks, specially since the datasets need to be free of noise to avoid spoiling the learning process. An ad hoc preprocessing of the dataset to mitigate the noise is a possible solution, but it is usually non-trivial and requires significant human intervention. In this paper, we pave the road for training CNNs for image retrieval with noisy datasets. In particular, we propose a novel Bag Exponential Loss function that, inspired by the Multiple Instance Learning framework, works with bags of matching images instead of single pairs, and allows a dynamical weighting of the relevance of each sample as the training progresses. The formulation of the proposed model is general enough and may serve to other purposes than dealing with noise if parameters are chosen appropriately. Extensive experimental results show the superior performance of the proposed loss with respect to the current state-of-the-art as well as its ability to cope with noisy training sets. Pytorch code available in https://github.com/tmcortes/BELoss [ABSTRACT FROM AUTHOR]

Vanishing Point (VP) detection is a computer vision task that can be useful in many different fields of application. In this work, we present a VP detection algorithm for natural landscape images based on an multi-threshold edge extraction process that combines several representations of an image, and on novel clustering and cluster refinement procedures. Our algorithm identifies a VP candidate in images with single-point perspective and improves detection results on two datasets that have already been tested for this task. Furthermore, we study how VP detection results have been reported in literature, pointing out the main drawbacks of previous approaches. To overcome these drawbacks, we present a novel error measure that is based on a probabilistic consistency measure between edges and VP hypothesis, and that can be tuned to vary the strictness on the results. Our reasoning on how our measure is more correct is supported by an intuitive analysis, simulations and an experimental validation. • Novel methodology for improved vanishing point detection in landscape images. • Improved edge extraction process by combining different representations of an image. • New cluster refinement method for discarding weak VP hypothesis. • New probabilistic error measure for more robust vanishing point detection. • Qualitative and quantitative analysis validating the reliability of the new measure. [ABSTRACT FROM AUTHOR]

Highlights • Automatic tools for touristic applications are highly valuable for the user • Single-scale clustering methods are insufficient to solve real clustering problems • Multi-scale (hierarchical) density-based clustering improves landmark detection • The separation of inhabited population cores facilitates the clustering approach • Increasing the dimensionality improves the results within crowded sample spaces. Abstract The process of determining relevant landmarks within a certain region is a challenging task, mainly due to its subjective nature. Many of the current lines of work include the use of density-based clustering algorithms as the base tool for such a task, as they permit the generation of clusters of different shapes and sizes. However, there are still important challenges, such as the variability in scale and density. In this paper, we present two novel density-based clustering algorithms that can be applied to solve this: K-DBSCAN, a clustering algorithm based on Gaussian Kernels used to detect individual inhabited cores within regions; and V-DBSCAN, a hierarchical algorithm suitable for sample spaces with variable density, which is used to attempt the discovery of relevant landmarks in cities or regions. The obtained results are outstanding, since the system properly identifies most of the main touristic attractions within a certain region under analysis. A comparison with respect to the state-of-the-art show that the presented method clearly outperforms the current methods devoted to solve this problem. [ABSTRACT FROM AUTHOR]

The electrodermal activity (EDA) is a psychophysiological indicator which can be considered a somatic marker of the emotional and attentional reaction of subjects towards stimuli. EDA measurements are not biased by the cognitive process of giving an opinion or a score to characterize the subjective perception, and group-level EDA recordings integrate the reaction of the whole audience, thus reducing the signal noise. This paper contributes to the field of affective video content analysis, extending previous novel work on the use of EDA as ground truth for prediction algorithms. Here, we label short video clips according to the audience's emotion (high vs. low) and attention (increasing vs. decreasing), derived from EDA records. Then, we propose a set of low-level audiovisual descriptors and train binary classifiers that predict the emotion and attention with 75% and 80% accuracy, respectively. These results, along with those of previous works, reinforce the usefulness of such low-level audiovisual descriptors to model video in terms of the induced affective response. [ABSTRACT FROM AUTHOR]

A two-level variable bit rate (VBR) control algorithm for hierarchical video coding, specifically tailored for the new High Efficiency Video Coding (HEVC) standard, is presented here. A long-term level monitors the current bit count along a sliding window of a few seconds, comprising several intra-periods (IPs) and shifted on an IP basis. This long-term view allows the accommodation of the naturally occurring rate variations at a slow pace, avoiding the annoying sharp quality changes commonly appearing when non-sliding window approaches are used. The bit excesses or defects observed at this level are evenly delivered to a short-term level mechanism that establishes target bit budgets for a narrower sliding window covering a single IP and shifting on a frame basis. At this level, an adequate quantization parameter is estimated to comply with the designated target bit rate. Recommended test conditions as well as two few minutes long video sequences with scene cuts have been used for the assessment of the proposed VBR controller. Comparisons with a state-of-the-art rate control algorithm have produced good results in terms of quality consistency, in exchange for moderate rate-distortion performance losses. [ABSTRACT FROM AUTHOR]

• End-to-end fully-automated system for cell migration analysis. • Increasing throughput, reducing human effort and removing human biases. • Performing a comprehensive shape- and motion- based feature extraction. • Enhancing behavioral cell profiling through an automatic explainability method. • Results show the applicability of our approach in a demanding scenario. [Display omitted] Cell detection and tracking applied to in vivo fluorescence microscopy has become an essential tool in biomedicine to characterize 4D (3D space plus time) biological processes at the cellular level. Traditional approaches to cell motion analysis by microscopy imaging, although based on automatic frameworks, still require manual supervision at some points of the system. Hence, when dealing with a large amount of data, the analysis becomes incredibly time-consuming and typically yields poor biological information. In this paper, we propose a fully-automated system for segmentation, tracking and feature extraction of migrating cells within blood vessels in 4D microscopy imaging. Our system consists of a robust 3D convolutional neural network (CNN) for joint blood vessel and cell segmentation, a 3D tracking module with collision handling, and a novel method for feature extraction, which takes into account the particular geometry in the cell-vessel arrangement. Experiments on a large 4D intravital microscopy dataset show that the proposed system achieves a significantly better performance than the state-of-the-art tools for cell segmentation and tracking. Furthermore, we have designed an analytical method of cell behaviors based on the automatically extracted features, which supports the hypotheses related to leukocyte migration posed by expert biologists. This is the first time that such a comprehensive automatic analysis of immune cell migration has been performed, where the total population under study reaches hundreds of neutrophils and thousands of time instances. [ABSTRACT FROM AUTHOR]

In this paper we propose a temporal segmentation and a keyframe selection method for User-Generated Video (UGV). Since UGV is rarely structured in shots and usually user’s interest are revealed through camera movements, a UGV temporal segmentation system has been proposed that generates a video partition based on a camera motion classification. Motion-related mid-level features have been suggested to feed a Hierarchical Hidden Markov Model (HHMM) that produces a user-meaningful UGV temporal segmentation. Moreover, a keyframe selection method has been proposed that picks a keyframe for fixed-content camera motion patterns such as zoom , still , or shake and a set of keyframes for varying-content translation patterns. The proposed video segmentation approach has been compared to a state-of-the-art algorithm, achieving 8% performance improvement in a segmentation-based evaluation. Furthermore, a complete search-based UGV annotation system has been developed to assess the influence of the proposed algorithms on an end-user task. To that purpose, two UGV datasets have been developed and made available online. Specifically, the relevance of the considered camera motion types has been analyzed for these two datasets, and some guidelines are given to achieve the desired performance-complexity tradeoff. The keyframe selection algorithm for varying-content translation patterns has also been assessed, revealing a notable contribution to the performance of the global UGV annotation system. Finally, it has been shown that the UGV segmentation algorithm also produces improved annotation results with respect to a fixed-rate keyframe selection baseline or a traditional method relying on frame-level visual features. [ABSTRACT FROM AUTHOR]

Abstract: In this article, an improved version of one of the most cited intra mode decision algorithms in H.264/AVC video coding is proposed with the aim to improve its efficiency and performance. The reference algorithm determines the interpolation/extrapolation spatial direction (mode) for achieving the best intra prediction using the Sobel gradient calculation. The proposed algorithm suggests reducing the number of gradients calculated and changing the Sobel operator to that of Roberts. In addition to this, it also proposes including the previously encoded neighboring block modes among the evaluated modes. Experimental results show that the proposed algorithm reduces the computational load of the original algorithm and substantially improves its rate-distortion performance. [ABSTRACT FROM AUTHOR]

The rate control problem has been extensively studied in parallel to the development of the different video coding standards. The bit allocation via Cauchy-density-based rate-distortion modeling of the discrete cosine transform coefficients has proved to be one of the most accurate solutions at picture level. Nevertheless, in some specific applications operating in real-time low-delay environments, a basic unit (BU) layer is recommended in order to provide a good tradeoff between picture quality and delay control. In this letter, a novel BU bit allocation for H.264/advanced video coding is proposed based on a simplified Cauchy probability density function source modeling. The experimental results are twofold: 1) the proposed rate control algorithm (RCA) achieves an average peak signal-to-noise ratio improvement of 0.28 dB respect to a well-known BU layer RCA, while maintaining a similar buffer occupancy evolution, and 2) it achieves to notably reduce the buffer occupancy fluctuations respect to a well-known picture layer RCA, while maintaining similar quality levels. [ABSTRACT FROM PUBLISHER]

Abstract: Automatic speech recognition (ASR) is called to play a relevant role in the provision of spoken interfaces for IP-based applications. However, as a consequence of the transit of the speech signal over these particular networks, ASR systems need to face two new challenges: the impoverishment of the speech quality due to the compression needed to fit the channel capacity and the inevitable occurrence of packet losses. In this framework, bitstream-based approaches that obtain the ASR feature vectors directly from the coded bitstream, avoiding the speech decoding process, have been proposed ([S.H. Choi, H.K. Kim, H.S. Lee, Speech recognition using quantized LSP parameters and their transformations in digital communications, Speech Commun. 30 (4) (2000) 223–233. A. Gallardo-Antolín, C. Pelàez-Moreno, F. Díaz-de-María, Recognizing GSM digital speech, IEEE Trans. Speech Audio Process., to appear. H.K. Kim, R.V. Cox, R.C. Rose, Performance improvement of a bitstream-based front-end for wireless speech recognition in adverse environments, IEEE Trans. Speech Audio Process. 10 (8) (2002) 591–604. C. Peláez-Moreno, A. Gallardo-Antolín, F. Díaz-de-María, Recognizing voice over IP networks: a robust front-end for speech recognition on the WWW, IEEE Trans. Multimedia 3(2) (2001) 209–218], among others) to improve the robustness of ASR systems. LSP (Line Spectral Pairs) are the preferred set of parameters for the description of the speech spectral envelope in most of the modern speech coders. Nevertheless, LSP have proved to be unsuitable for ASR, and they must be transformed into cepstrum-type parameters. In this paper we comparatively evaluate the robustness of the most significant LSP to cepstrum transformations in a simulated VoIP (voice over IP) environment which includes two of the most popular codecs used in that network (G.723.1 and G.729) and several network conditions. In particular, we compare ‘pseudocepstrum’ [H.K. Kim, S.H. Choi, H.S. Lee, On approximating Line Spectral Frequencies to LPC cepstral coefficients, IEEE Trans. Speech Audio Process. 8 (2) (2000) 195–199], an approximated but straightforward transformation of LSP into LP cepstral coefficients, with a more computationally demanding but exact one. Our results show that pseudocepstrum is preferable when network conditions are good or computational resources low, while the exact procedure is recommended when network conditions become more adverse. [Copyright &y& Elsevier]