Search

Feature aggregation is a crucial step in many methods of image classification, like the Bag-of-Words (BoW) model or the Convolutional Neural Networks (CNN). In this aggregation step, usually known as spatial pooling, the descriptors of neighbouring elements within a region of the image are combined into a local or a global feature vector. The combined vector must contain relevant information, while removing irrelevant and confusing details. Maximum and average are the most common aggregation functions used in the pooling step. To improve the aggregation of relevant information without degrading their discriminative power for classification in this work we propose the use of Ordered Weighted operators. We provide an extensive evaluation that shows that the final result of the classification using OWA aggregation is always better than average pooling and better than maximum pooling when dealing with small dictionary sizes.

In this paper, we address the challenge of identifying music suitable to accompany typical daily activities. We first derive a list of common activities by analyzing social media data. Then, an automatic approach is proposed to find music for these activities. Our approach is inspired by our experimentally acquired findings (a) that genre and instrument information, i.e., as appearing in the textual metadata, are not sufficient to distinguish music appropriate for different types of activities, and (b) that existing content-based approaches in the music information retrieval community do not overcome this insufficiency. The main contributions of our work are (a) our analysis of the properties of activity-related music that inspire our use of novel high-level features, e.g., drop-like events, and (b) our approach's novel method of extracting and combining low-level features, and, in particular, the joint optimization of the time window for feature aggregation and the number of features to be used. The effectiveness of the approach method is demonstrated in a comprehensive experimental study including failure analysis.

Video summarization is a powerful tool for video understanding and browsing and is considered as an enabler for many video analysis tasks. While the effect of video summarization models has been largely studied in video retrieval and indexing applications over the last decade, its impact has not been well investigated in content-based video recommendation systems (RSs) based on low-level visual features, where the goal is to recommend items/videos to users based on visual content of videos. This work reveals specific problems related to video summarization and their impact on video recommendation. We present preliminary results of an analysis involving applying different video summarization models for the problem of video recommendation on a real-world RS dataset (MovieLens-10M) and show how temporal feature aggregation and video segmentation granularity can significantly influence/improve the quality of recommendation.

3D object detection has recently become a research hotspot in the field of autonomous driving. Although great progress has been made, it still needs to be further improved. Therefore, this paper presents FDCA, a feature deep continuous aggregation network using multi-sensors for 3D vehicle detection. The proposed network adopts a two-stage structure with the bird&rsquo, s-eye view (BEV) map and the RGB image as an input. In the first stage, two feature extractors were used to generate feature maps with the high-resolution and representational ability for each input view. These feature maps were then fused and fed to a 3D proposal generator to obtain the reliable 3D vehicle proposals. In the second stage, the refinement network aggregated the features of the proposal regions further and performs classifications, a 3D bounding boxes regression, and orientation estimations to predict the location and heading of vehicles in 3D space. The FDCA network proposed was trained and evaluated on the KITTI 3D object detection benchmark. The experimental results of the validation set illustrated that compared with other fusion-based methods, the 3D average precision (AP) could achieve 76.82% on a moderate setting while having real-time capability, which was higher than that of the second-best performing method by 2.38%. Meanwhile, the results of ablation experiments show that the convergence rate of FDCA was much faster and the stability was also much better, making it a candidate for application in autonomous driving.

The proposed method has 30 streams, i.e., 15 spatial streams and 15 temporal streams. Each spatial stream corresponds to each temporal stream. Therefore, this work correlates with the symmetry concept. It is a difficult task to classify video-based facial expression owing to the gap between the visual descriptors and the emotions. In order to bridge the gap, a new video descriptor for facial expression recognition is presented to aggregate spatial and temporal convolutional features across the entire extent of a video. The designed framework integrates a state-of-the-art 30 stream and has a trainable spatial&ndash, temporal feature aggregation layer. This framework is end-to-end trainable for video-based facial expression recognition. Thus, this framework can effectively avoid overfitting to the limited emotional video datasets, and the trainable strategy can learn to better represent an entire video. The different schemas for pooling spatial&ndash, temporal features are investigated, and the spatial and temporal streams are best aggregated by utilizing the proposed method. The extensive experiments on two public databases, BAUM-1s and eNTERFACE05, show that this framework has promising performance and outperforms the state-of-the-art strategies.

The problem of Near-Duplicate Video Retrieval (NDVR) has attracted increasing interest due to the huge growth of video content on the Web, which is characterized by high degree of near duplicity. This calls for efficient NDVR approaches. Motivated by the outstanding performance of Convolutional Neural Networks (CNNs) over a wide variety of computer vision problems, we leverage intermediate CNN features in a novel global video representation by means of a layer-based feature aggregation scheme. We perform extensive experiments on the widely used CC_WEB_VIDEO dataset, evaluating three popular deep architectures (AlexNet, VGGNet, GoogLeNet) and demonstrating that the proposed approach exhibits superior performance over the state-of-the-art, achieving a mean Average Precision (mAP) score of 0.976.

Aggregating a set of local features has been used widely to realize recognition or retrieval of multimedia data including 2D images and 3D models. A number of feature aggregation algorithms (e.g., Bag-of-Features, Locality-constrained Linear coding, or Fisher Vector coding) have been proposed. They first learn a codebook, or a set of codewords, by clustering the local features and then encode these local features by using the learned codebook. Despite the great success of these feature aggregation algorithms, we argue that they are not necessarily optimal in terms of accuracy since their codebook learning and feature encoding are computed separately. In this paper, we propose two novel feature aggregation algorithms based on k-Sparse Autoencoder (kSA) that realize more accurate local feature aggregation. Our proposed algorithms, called Database-adaptive kSA (DkSA) aggregation and Per-data-adaptive kSA (PkSA) aggregation, jointly optimize codebook learning and feature encoding. In addition, the kSA-based feature encoding enhances saliency of local features due to k-sparseness constraints and non-negativity constraints. Of the two proposed algorithms, the PkSA aggregation exploits reconstruction error of a local feature derived from the kSA for more accurate aggregated feature. Experimental evaluation using a shape-based 3D model retrieval scenario showed that the retrieval accuracy of our proposed algorithms are superior to the existing feature aggregation algorithms we have compared against.

An effective and widespread approach for shape-based 3D model retrieval (3DMR) is to use a feature vector per 3D model obtained by aggregating, or pooling, a set of local features extracted from the 3D model. State-of-the-art feature aggregation algorithms, such as Fisher Vector (FV) coding [7] or Super Vector (SV) coding [22], used in the approach is not spatially efficient, however. The FV or SV, for example, typically encodes a local feature into a very high-dimensional (e.g., 300k-dimensional) vector. For a database containing a large number of 3D models, the spatial cost of storing all the aggregated feature vectors for the database becomes very high. In this paper, we propose a novel, spatially efficient yet accurate feature aggregation algorithm called Sum of Sparse Binary codes (SSB) aggregation. The SSB first encodes a local feature into a highly sparse binary code. Then, a set of sparse binary codes are aggregated efficiently by simple summing into a compact feature vector. We also propose fast SSB (fSSB) aggregation, which is a computationally efficient approximation of the SSB. Experiments using a 3DMR scenario show that the proposed algorithms are significantly more efficient than the state-of-the-art feature aggregation algorithms we have compared against. At the same time, retrieval accuracies of the proposed algorithms are equal or better than the state-of-the-art aggregation algorithms.

We propose a novel query-dependent feature aggregation (QDFA) method for medical image retrieval. The QDFA method can learn an optimal feature aggregation function for a multi-example query, which takes into account multiple features and multiple examples with different importance. The experiments demonstrate that the QDFA method outperforms three other feature aggregation methods.

A method for aggregating binary features to represent an image in a single vector is proposed. Binary features are decomposed into several sub-features to permit the individual aggregation of each sub-feature into a histogram. The method is completely quantisation-free, enabling exceptionally fast image representation. Experimental results demonstrate that the method is a competitive alternative for image representation when runtime efficiency is crucial.

This paper considers a family of metrics to compare images based on their local descriptors. It encompasses the vector or locally aggregated descriptors descriptor and matching techniques such as hamming embedding. Making the bridge between these approaches leads us to propose a match kernel that takes the best of existing techniques by combining an aggregation procedure with a selective match kernel. The representation underpinning this kernel is approximated, providing a large scale image search both precise and scalable, as shown by our experiments on several benchmarks. We show that the same aggregation procedure, originally applied per image, can effectively operate on groups of similar features found across multiple images. This method implicitly performs feature set augmentation, while enjoying savings in memory requirements at the same time. Finally, the proposed method is shown effective for place recognition, outperforming state of the art methods on a large scale landmark recognition benchmark.

A line generalization solution is presented based on the operations known as water-lining and medial-axis transformation. Although the solution is of general application, this report focuses on sho...

Raised intracranial pressure (ICP) causes serious problem on traumatic brain injury patient. Automated and non-intrusive ICP level prediction saves cost and enhances efficiency. An automated ICP level prediction model based on machine learning method is proposed in this paper. Multiple features, including midline shift, intracranial air cavities, ventricle size, texture patterns, and blood amount, are selected, extracted and aggregated using different methods. Some demographic information, such as age and injury severity score, is also considered as candidate features. After the feature aggregation, the most important features are selected by a feature selection scheme applied on 10 fold nested cross validation. The final support vector machine classification result using RapidMiner shows the effectiveness of the proposed method in ICP level prediction.

The analysis of voluminous patterns is often problematic due to the confounding effect of features that are not relevant to the problem at hand. For instance, the classification of biomedical spectra is often best achieved through the identification of a subset of highly discriminatory features while ignoring the non-relevant ones. With respect to pattern classification, we present a feature reduction strategy, which begins with the instantiation of many classifiers operating on different subsets of features, employing a feature sampling method to identify discriminatory feature subsets. These subsets are further aggregated to improve the overall performance of the underlying classifiers. We empirically demonstrate, using a voluminous biomedical dataset, that this strategy produces superior classification accuracies compared against a set of benchmarks.

This paper presents a study on merging confidence measures using fuzzy logic. Instead of the previous approaches using the notion of probability, we propose to observe the uncertainty of the recognition hypotheses and the notion of possibility thanks to fuzzy reasoning. Four different confidence measures are developed, coming from different parts of a speech recognizer. Various merging methods are studied to improve the performance of the confidence measures. The methods are evaluated in terms of Confidence Error Rate (CER) and in terms of their Detection Error Tradeoff (DET) curves on a French broadcast news corpus. They are compared to some fuzzy logic aggregation techniques among which the technique based on the Choquet Integral yields to a significant improvement in terms of CER.

Feature aggregation is a critical technique in content- based image retrieval systems that employ multiple visual features to characterize image content. In this paper, the p-norm is introduced to feature aggregation that provides a framework to unify various previous feature aggregation schemes such as linear combination, Euclidean distance, Boolean logic and decision fusion schemes in which previous schemes are instances. Some insights of the mechanism of how various aggregation schemes work are discussed through the effects of model parameters in the unified framework. Experiments show that performances vary over feature aggregation schemes that necessitates an unified framework in order to optimize the retrieval performance according to individual queries and user query concept. Revealing experimental results conducted with IAPR TC-12 ImageCLEF2006 benchmark collection that contains over 20,000 photographic images are presented and discussed.

In standard radiomics studies the features extracted from clinical images are mostly quantified with simple statistics such as the average or variance per Region of Interest (ROI). Such approaches may smooth out any intra-region heterogeneity and thus hide some tumor aggressiveness that may hamper predictions. In this paper we study the importance of feature aggregation within the standard radiomics workflow, which allows to take into account intra-region variations. Feature aggregation methods transform a collection of voxel values from feature response maps (over a ROI) into one or several scalar values that are usable for statistical or machine learning algorithms. This important step has been little investigated within the radiomics workflows, so far. In this paper, we compare several aggregation methods with standard radiomics approaches in order to assess the improvements in prediction capabilities. We evaluate the performance using an aggregation function based on Bags of Visual Words (BoVW), which allows for the preservation of piece-wise homogeneous information within heterogeneous regions and compared with standard methods. The different models are compared on a cohort of 214 head and neck cancer patients coming from 4 medical centers. Radiomics features were extracted from manually delineated tumors in clinical PET-FDG and CT images were analyzed. We compared the performance of standard radiomics models, the volume of the ROI alone and the BoVW model for survival analysis. The average concordance index was estimated with a five fold cross-validation. The performance was significantly better using the BoVW model 0.627 (95% CI: 0.616–0.637) as compared to standard radiomics0.505 (95% CI: 0.499–0.511), mean-var. 0.543 (95% CI: 0.536–0.549), mean0.547 (95% CI: 0.541–0.554), var.0.530 (95% CI: 0.524–0.536) or volume 0.577 (95% CI: 0.571–0.582). We conclude that classical aggregation methods are not optimal in case of heterogeneous tumors. We also showed that the BoVW model is a better alternative to extract consistent features in the presence of lesions composed of heterogeneous tissue.