Search

PurposeSemantic retrieval and discovery of datasets published as open data remains a challenging task. The datasets inherently originate in the globally distributed web jungle, lacking the luxury of centralized database administration, database schemes, shared attributes, vocabulary, structure and semantics. The existing dataset catalogs provide basic search functionality relying on keyword search in brief, incomplete or misleading textual metadata attached to the datasets. The search results are thus often insufficient. However, there exist many ways of improving the dataset discovery by employing content-based retrieval, machine learning tools, third-party (external) knowledge bases, countless feature extraction methods and description models and so forth.Design/methodology/approachIn this paper, the authors propose a modular framework for rapid experimentation with methods for similarity-based dataset discovery. The framework consists of an extensible catalog of components prepared to form custom pipelines for dataset representation and discovery.FindingsThe study proposes several proof-of-concept pipelines including experimental evaluation, which showcase the usage of the framework.Originality/valueTo the best of authors’ knowledge, there is no similar formal framework for experimentation with various similarity methods in the context of dataset discovery. The framework has the ambition to establish a platform for reproducible and comparable research in the area of dataset discovery. The prototype implementation of the framework is available on GitHub.

We present Videolytics, a web-based system for advanced analytics over recorded video streams. Video cameras have become widely used for indoor and outdoor surveillance. Covering even more public space in cities, the cameras serve various purposes ranging from security to traffic monitoring, urban life, and marketing. The goal is to obtain effective and efficient models to process the video data automatically and produce the desired features for data analytics. Videolytics combines the best of deep learning and hand-designed analytical models to create a solution applicable in real-life situations. The architecture of the Videolytics framework is centered around a database of video features and detected objects, where new higher-level objects result from fusion of (lower-level) objects and features already stored in the database. The system provides a number of visualization options, an SQL-based analytics module as well as a real-time surveillance mode.

Similarity queries play the crucial role in content-based retrieval. The similarity function itself is regarded as the function of relevance between a query object and objects from database; the most similar objects are understood as the most relevant. However, such an automatic adoption of similarity as relevance leads to limited applicability of similarity search in domains like entity discovery, where relevant objects are not supposed to be similar in the traditional meaning. In this paper, we propose the meta-model of data-transitive similarity operating on top of a particular similarity model and a database. This meta-model enables to treat directly non-similar objects \(\mathbf{x} \), \(\mathbf{y} \) as similar if there exists a chain of objects \(\mathbf{x} \), \(i_1\), ..., \(i_n\), \(\mathbf{y} \) having the neighboring members similar enough. Hence, this approach places the similarity in the role of relevance, where objects do not need to be directly similar but still remain relevant to each other (transitively similar). The data-transitive similarity concept allows to use standard similarity-search methods (queries, joins, rankings, analytics) in more complex tasks, like the entity discovery, where relevant results are often complementary or orthogonal to the query, rather than directly similar. Moreover, we show the data-transitive similarity is inherently self-explainable and non-metric. We discuss the approach in the domain of open dataset discovery.

Video cameras have become widely used for indoor and outdoor surveillance. Covering more and more public space in cities, the cameras serve various purposes ranging from security to traffic monitoring, urban life, and marketing. However, with the increasing quantity of utilized cameras and recorded streams, manual video monitoring and analysis becomes too laborious. The goal is to obtain effective and efficient artificial intelligence models to process the video data automatically and produce the desired features for data analytics. To this end, we propose a framework for real-time video feature extraction that fuses both learned and hand-designed analytical models and is applicable in real-life situations. Nowadays, state-of-the-art models for various computer vision tasks are implemented by deep learning. However, the exhaustive gathering of labeled training data and the computational complexity of resulting models can often render them impractical. We need to consider the benefits and limitations of each technique and find the synergy between both deep learning and analytical models. Deep learning methods are more suited for simpler tasks on large volumes of dense data while analytical modeling can be sufficient for processing of sparse data with complex structures. Our framework follows those principles by taking advantage of multiple levels of abstraction. In a use case, we show how the framework can be set for an advanced video analysis of urban life.

Many institutions publish datasets as Open Data in catalogs, however, their retrieval remains problematic issue due to the absence of dataset search benchmarking. We propose a framework for evaluating findability of datasets, regardless of retrieval models used. As task-agnostic labeling of datasets by ground truth turns out to be infeasible in the general domain of open data datasets, the proposed framework is based on evaluation of entire retrieval scenarios that mimic complex retrieval tasks. In addition to the framework we present a proof of concept specification and evaluation on several similarity-based retrieval models and several dataset discovery scenarios within a catalog, using our experimental evaluation tool. Instead of traditional matching of query with metadata of all the datasets, in similarity-based retrieval the query is formulated using a set of datasets (query by example) and the most similar datasets to the query set are retrieved from the catalog as a result.

Many institutions choose to make their datasets available as Open Data. Open Data datasets are described by publisher-provided metadata and are registered in catalogs such as the European Data Portal. In spite of that, findability still remain a major issue. One of the main reasons is that metadata is captured in different contexts and with different background knowledge, so that keyword-based search provided by the catalogs is insufficient. A solution is to use an enriched querying that employs a dataset similarity model built on a shared context represented by a knowledge graph. However, the “black-box” dataset similarity may not fit well the user needs. If an explainable similarity model is used, then the issue can be tackled by providing users with a visualisation of the dataset similarity. This paper introduces a web-based tool for dataset similarity visualisation called ODIN (Open Dataset INspector). ODIN visualises knowledge graph-based dataset similarity, offering thus an explanation to the user. To understand the similarity, users can discover additional datasets that match their needs or reformulate the query to better reflect the knowledge graph. Last but not least, the user can analyze and/or design the similarity model itself.

The TriGen algorithm is a general approach to transform distance spaces in order to provide both exact and approximate similarity search in metric and non-metric spaces. This paper focuses on the reduction of intrinsic dimensionality using TriGen. Besides the well-known intrinsic dimensionality based on distance distribution, we inspect properties of triangles used in metric indexing (the triangularity) as well as properties of quadrilaterals used in ptolemaic indexing (the ptolemaicity). We also show how LAESA with triangle and ptolemaic filtering behaves on several datasets with respect to the proposed indicators.

There is a vast amount of datasets available as Open Data on the Web. However, it is challenging for consumers to find datasets relevant to their goals. This is because the available metadata in catalogs is not descriptive enough. Nevertheless, datasets exist in various types of contexts not expressed in the metadata. These may include information about the data publisher, the legislation related to dataset publication, etc. In this paper we describe an idea of a data model that enables consumers to better understand the data. We propose to define a formal model for representation of the datasets and their contexts, and we propose to apply existing similarity techniques, adjust them to fit each identified dataset context type and combine them together to measure similarity of datasets in new ways, improving their findability.

Since the boom in new proposals on techniques for efficient querying of XML data is now over and the research world has shifted its attention toward new types of data formats, we believe that it is crucial to review what has been done in the area to help users choose an appropriate strategy and scientists exploit the contributions in new areas of data processing. The aim of this work is to provide a comprehensive study of the state-of-the-art of approaches for the structural querying of XML data. In particular, we start with a description of labeling schemas to capture the structure of the data and the respective storage strategies. Then we deal with the key part of every XML query processing: a twig query join, XML query algebras, optimizations of query plans, and selectivity estimation of XML queries. To the best of our knowledge, this is the first work that provides such a detailed description of XML query processing techniques that are related to structural aspects and that contains information about their theoretical and practical features as well as about their mutual compatibility and general usability.

The behavior, environment, and characteristics of clients form a crucial source of information for various businesses. There exists a number of supervised as well as unsupervised data mining or other approaches that allow analyzing the respective data. In our ongoing project, focusing primarily on the financial sector, we suggest an innovative strategy that will overcome persisting shortcomings of the state-of-the-art methods using an analysis of a social network of clients. In addition, we do not assume the existence of such a network, but from a given set of client financial activities, we are able to infer a social network representing their relationships and behavior. Using real-world data and selected use cases from our domain, we show (a part of) the process of construction of an inferred social network, i.e., what kind of "hidden" information can, for example, be found and exploited.

Virtual reality devices nowadays can effectively utilise other senses besides vision, too. The most often used secondary sense is hearing with binaural audio as VR engine. Currently, practical usage of binaural audio as the source of VR is impossible because of the inaccuracy of a general model. On the contrary, measuring the personalised parameters can be time-consuming. Our task was to prove the possibility of reconstruction of the binaural audio parameters in domestic conditions. We have focused on the design of the user interface that can be used independently on the platform. Our proposed browser-based application uses collaborative filtering as a recommender system. We have proven that sound-based navigation in axial plane is possible with 6.6° inaccuracy. The gamification and browser-based implementation make it easier for all people to find the best possible parameters. The resulting profile can be used both with fully VR environment and with semi-VR games.

The success of many businesses is based on a thorough knowledge of their clients. There exists a number of supervised as well as unsupervised data mining or other approaches that allow to analyze data about clients, their behavior or environment. In our ongoing project focusing primarily on bank clients, we propose an innovative strategy that will overcome shortcomings of the existing methods. From a given set of user activities, we infer their social network in order to analyze user relationships and behavior. For this purpose, not just the traditional direct facts are incorporated, but also relationships inferred using similarity measures and statistical approaches, with both possibly limited measures of reliability and validity in time. Such networks would enable analyses of client characteristics from a new perspective and could provide otherwise impossible insights. However, there are several research and technical challenges making the outlined pursuit novel, complex and challenging as we outline in this vision paper.

The metric space model is a popular and extensible model for indexing data for fast similarity search. However, there is often need for broader concepts of similarities (beyond the metric space model) while these cannot directly benefit from metric indexing. This paper focuses on approximate search in semi-metric spaces using a genetic variant of the TriGen algorithm. The original TriGen algorithm generates metric modifications of semi-metric distance functions, thus allowing metric indexes to index non-metric models. However, “analytic” modifications provided by TriGen are not stable in predicting the retrieval error. In our approach, the genetic variant of TriGen – the TriGenGA – uses genetically learned semi-metric modifiers (piecewise linear functions) that lead to better estimates of the retrieval error. Additionally, the TriGenGA modifiers result in better overall performance than original TriGen modifiers.

There is a large quantity of datasets available as Open Data on the Web. However, it is challenging for users to find datasets relevant to their needs, even though the datasets are registered in catalogs such as the European Data Portal. This is because the available metadata such as keywords or textual description is not descriptive enough. At the same time, datasets exist in various types of contexts not expressed in the metadata. These may include information about the dataset publisher, the legislation related to dataset publication, language and cultural specifics, etc. In this paper we introduce a similarity model for matching datasets. The model assumes an ontology/knowledge graph, such as Wikidata.org, that serves as a graph-based context to which individual datasets are mapped based on their metadata. A similarity of the datasets is then computed as an aggregation over paths among nodes in the graph. The proposed similarity aims at addressing the problem of explainability of similarity, i.e., providing the user a structured explanation of the match which, in a broader sense, is nowadays a hot topic in the field of artificial intelligence.

We present a scalable and unsupervised approach for content-based retrieval on 3D model collections. Our goal is to represent a 3D shape as a set of discriminative local features, which is important to maintain robustness against deformations such as non-rigid transformations and partial data. However, this representation brings up the problem on how to compare two 3D models represented by feature sets. For solving this problem, we apply the signature quadratic form distance (SQFD), which is suitable for comparing feature sets. Using SQFD, the matching between two 3D objects involves only their representations, so it is easy to add new models to the collection. A key characteristic of the feature signatures, required by the SQFD, is that the final object representation can be easily obtained in a unsupervised manner. Additionally, as the SQFD is an expensive distance function, to make the system scalable we present a novel technique to reduce the amount of features by detecting clusters of key points on a 3D model. Thus, with smaller feature sets, the distance calculation is more efficient. Our experiments on a large-scale dataset show that our proposed matching algorithm not only performs efficiently, but also its effectiveness is better than state-of-the-art matching algorithms for 3D models.

In this paper, we present a prototype web application of a product search engine of a fashion e-shop. Today, e-shop product metadata consist of text description, simple attributes (price, size, color, fabric, etc.) and visual information (product photo). Search engines used in e-shops mostly provide text and attribute/category interface for product filtering. In our model, we focus on the visual information applied in an interactive query-by-example scenario. The global visual descriptors may be often ambiguous and may not correspond well with the intended mental query of the user. Therefore, we proposed and evaluated model and GUI allowing user to guide the query process by selecting image regions (patches) of interest within the query. In the demo evaluation, we show that allowing user to specify relevant image patches led to a significant improvement of the results’ relevance in the vast majority of tested queries.

Collecting various types of data about users/clients in order to improve the services and competitiveness of companies has a long history. However, these approaches are often based on classical statistical methods and an assumption of limited computational power. In this paper we introduce the vision of our applied research project targeting to the financial sector. Our main goal is to develop an automated software solution for similarity modeling over big and semi-structured graph data representing behavior of bank clients. The main aim of similarity models is to improve the decision process in risk management, marketing, security and related areas.

Similarity search and content-based retrieval have become widely used in multimedia database systems that often manage huge data collections. Unfortunately, many effective content-based similarity models cannot be fully utilized for larger datasets, as they are computationally demanding and require massive parallel processing for both feature extraction and query evaluation tasks. In this work, we address the performance issues of effective similarity models based on feature signatures, where we focus on fast feature extraction from image thumbnails using affordable hardware. More specifically, we propose a multi-GPU implementation that increases the extraction speed by two orders of magnitude with respect to a single-threaded CPU implementation. Since the extraction algorithm is not directly parallelizable, we propose a modification of the algorithm embracing the SIMT execution model. We have experimentally verified that our GPU extractor can be successfully used to index large image datasets comprising millions of images. In order to obtain optimal extraction parameters, we employed the GPU extractor in an extensive empirical investigation of the parameter space. The experimental results are discussed from the perspectives of both performance and similarity precision.

In this demo paper, we present a prototype web application of a product search engine of a fashion e-shop. Although e-shop products consist of full-text description, relational attributes (e.g., price, type, size, color, etc.) as well as visual information (product photo), traditional search engines in e-shops only provide full-text and relational attributes for product filtering. In our retrieval model, we incorporate also the visual information into the search by extracting visual-semantic features using deep convolutional neural networks. Furthermore, visual exploration of the product space using the visual-semantic features (multi-example queries) is used to dynamically discover latent visual attributes that could enhance the original relational schema by fuzzy attributes (e.g., a floral pattern in product). In the demo, we show how these latent attributes could be used to recommend the user preferred products and even outfits (e.g., shoes, bag, jacket) that fit a certain visual style.

This paper presents, to our knowledge, the first study on analyzing mathematical expressions to detect academic plagiarism. We make the following contributions. First, we investigate confirmed cases of plagiarism to categorize the similarities of mathematical content commonly found in plagiarized publications. From this investigation, we derive possible feature selection and feature comparison strategies for developing math-based detection approaches and a ground truth for our experiments. Second, we create a test collection by embedding confirmed cases of plagiarism into the NTCIR-11 MathIR Task dataset, which contains approx. 60 million mathematical expressions in 105,120 documents from arXiv.org. Third, we develop a first math-based detection approach by implementing and evaluating different feature comparison approaches using an open source parallel data processing pipeline built using the Apache Flink framework. The best performing approach identifies all but two of our real-world test cases at the top rank and achieves a mean reciprocal rank of 0.86. The results show that mathematical expressions are promising text-independent features to identify academic plagiarism in large collections. To facilitate future research on math-based plagiarism detection, we make our source code and data available. published

We present a demo of behaviour-based similarity retrieval in network traffic data. The underlying framework is intended to support domain experts searching for network nodes (computers) infected by malicious software, especially in cases when single client-server communication does not have to be sufficient to reliably identify the infection. The focus is on interactive browsing enabling dynamic changes of the retrieval model, which is based on a recently proposed statistical description (fingerprint) of a communication between two network hosts and the bag of features approach. The demo/framework provides unique insight into the data and enables annotation of the data and model modifications during the search for more effective identification of infected hosts.

After two decades of research, the techniques for efficient similarity search in metric spaces have combined virtually all the available tricks resulting in many structural index designs. As the representative state-of-the-art metric access methods (also called metric indexes) that vary in the usage of filtering rules and in structural designs, we could mention the M-tree, the M-Index and the List of Clusters, to name a few. In this paper, we present the concept of cut-regions that could heavily improve the performance of metric indexes that were originally designed to employ simple ball-regions. We show that the shape of cut-regions is far more compact than that of ball-regions, yet preserving simple and concise representation. We present three re-designed metric indexes originating from the above-mentioned ones but utilizing cut-regions instead of ball-regions. We show that cut-regions can be fully utilized in the index structure, positively affecting not only query processing but also the index construction. In the experiments we show that the re-designed metric indexes significantly outperform their original versions. HighlightsThe new cut-region formalism that is suitable for simplified description of compact metric regions.New cheap dynamic construction techniques for the PM-tree that can compete with expensive strategies of the original PM-tree (e.g., multi-way leaf selection).Adaptation of M-Index and List of Clusters to operate with cut-regions.Thorough experimental evaluation also including comparison with the state-of-the-art MAMs.

Metric indexing is the state of the art in general distance-based retrieval. Relying on the triangular inequality, metric indexes achieve significant online speed-up beyond a linear scan. Recently, the idea of Ptolemaic indexing was introduced, which substitutes Ptolemy's inequality for the triangular one, potentially yielding higher efficiency for the distances where it applies. In this paper we have adapted several metric indexes to support Ptolemaic indexing, thus establishing a class of Ptolemaic access methods (PtoAM). In particular, we include Ptolemaic Pivot tables, Ptolemaic PM-Trees and the Ptolemaic M-Index. We also show that the most important and promising family of distances suitable for Ptolemaic indexing is the signature quadratic form distance , an adaptive similarity measure which can cope with flexible content representations of multimedia data, among other things. While this distance has shown remarkable qualities regarding the search effectiveness, its high computational complexity underscores the need for efficient search methods. We show that these distances are Ptolemaic metrics and present a study where we apply Ptolemaic indexing methods on real-world image databases, resolving exact queries nearly four times as fast as the state-of-the-art metric solution, and up to three orders of magnitude times as fast as sequential scan.

The popularity of similarity search expanded with the increased interest in multimedia databases, bioinformatics, or social networks, and with the growing number of users trying to find information in huge collections of unstructured data. During the exploration, the users handle database objects in different ways based on the utilized similarity models, ranging from simple to complex models. Efficient indexing techniques for similarity search are required especially for growing databases. In this paper, we study implementation possibilities of the recently announced theoretical framework SIMDEX, the task of which is to algorithmically explore a given similarity space and find possibilities for efficient indexing. Instead of a fixed set of indexing properties, such as metric space axioms, SIMDEX aims to seek for alternative properties that are valid in a particular similarity model (database) and, at the same time, provide efficient indexing. In particular, we propose to implement the fundamental parts of SIMDEX by means of the genetic programming (GP) which we expect will provide highquality resulting set of expressions (axioms) useful for indexing.

In this paper, we present detection of malware in HTTPS traffic using k-NN classification. We focus on the metric space approach for approximate k-NN searches over dataset of sparse high-dimensional descriptors of network traffic. We show the classification based on approximate k-NN search using metric index exhibits false positive rate reduced by an order of magnitude when compared to the state of the art method, while keeping the classification fast enough.

This paper presents a tool for interactive filtering and browsing of up to hundreds of hours of video content. In particular, we address the known-item search, i.e., searching for a short video clip known visually or by textual description. Video content is filtered with simple user-defined sketches of the searched scenes consisting of its distinct color regions and significant edges. Furthermore, the filtered content might be browsed with the query-by-example paradigm utilizing either visual or semantic similarity.

The Signature Quadratic Form Distance on feature signatures represents a flexible distance-based similarity model for effective content-based multimedia retrieval. Although metric indexing approaches are able to speed up query processing by two orders of magnitude, their applicability to large-scale multimedia databases containing billions of images is still a challenging issue. In this paper, we propose a parallel approach that balances the utilization of CPU and many-core GPUs for efficient similarity search with the Signature Quadratic Form Distance. In particular, we show how to process multiple distance computations and other parts of the search procedure in parallel, achieving maximal performance of the combined CPU/GPU system. The experimental evaluation demonstrates that our approach implemented on a common workstation with 2 GPU cards outperforms traditional parallel implementation on a high-end 48-core NUMA server in terms of efficiency almost by an order of magnitude. If we consider also the price of the high-end server that is ten times higher than that of the GPU workstation then, based on price/performance ratio, the GPU-based similarity search beats the CPU-based solution by almost two orders of magnitude. Although proposed for the SQFD, our approach of fast GPU-based similarity search is applicable for any distance function that is efficiently parallelizable in the SIMT execution model.

In biological applications, the tandem mass spectrometry is a widely used method for determining protein and peptide sequences from an ''in vitro'' sample. The sequences are not determined directly, but they must be interpreted from the mass spectra, which is the output of the mass spectrometer. This work is focused on a similarity-search approach to mass spectra interpretation, where the parameterized Hausdorff distance (d"H"P) is used as the similarity. In order to provide an efficient similarity search under d"H"P, the metric access methods and the TriGen algorithm (controlling the metricity of d"H"P) are employed. Moreover, the search model based on the d"H"P supports posttranslational modifications (PTMs) in the query mass spectra, what is typically a problem when an indexing approach is used. Our approach can be utilized as a coarse filter by any other database approach for mass spectra interpretation.

The metric space model has represented a reasonable trade-off concerning the efficiency and effectiveness problem in similarity search. However, complex similarity models that do not satisfy the metric properties have been used in a wide variety of research domains like multimedia information retrieval, digital libraries, biological and chemical databases, time series analysis, and biometry [2]. All these domains require the management of very large data collections, but the algorithms and data structures for searching in metric spaces cannot be used directly, as they require to use nonmetric similarity measures. As the term nonmetric simply means that a similarity function does not satisfy some (or all) the properties of a metric, we restrict its definition to nonmetric similarity functions that are "context-free and static", that is, the similarity between two objects is constant regardless of the context (time, user, query, other objects in the collection, etc.).

Since its introduction in 1997, the M-tree became a respected metric access method (MAM), while remaining, together with its descendants, still the only database-friendly MAM, that is, a dynamic structure persistent in paged index. Although there have been many other MAMs developed over the last decade, most of them require either static or expensive indexing. By contrast, the dynamic M-tree construction allows us to index very large databases in subquadratic time, and simultaneously the index can be maintained up-to-date (i.e., supports arbitrary insertions/deletions). In this article we propose two new techniques improving dynamic insertions in M-tree—the forced reinsertion strategies and so-called hybrid-way leaf selection. Both of the techniques preserve logarithmic asymptotic complexity of a single insertion, while they aim to produce more compact M-tree hierarchies (which leads to faster query processing). In particular, the former technique reuses the well-known principle of forced reinsertions, where the new insertion algorithm tries to re-insert the content of an M-tree leaf that is about to split in order to avoid that split. The latter technique constitutes an efficiency-scalable selection of suitable leaf node wherein a new object has to be inserted. In the experiments we show that the proposed techniques bring a clear improvement (speeding up both indexing and query processing) and also provide a tuning tool for indexing vs. querying efficiency trade-off. Moreover, a combination of the new techniques exhibits a synergic effect resulting in the best strategy for dynamic M-tree construction proposed so far.

Understanding the saliency of keyframes in short casual/home-made videos containing redundant information is an important step towards the design of successful keyframe selection and summarization techniques for such videos. Therefore, we present an extensive user study focusing on saliency of keyframes in such short redundant videos. In our study, more than 200 users annotated 32 videos, altogether selecting more than 20.000 keyframes. We present the description of the user study, the utilized annotation tool and we discuss the results. We provide also a preliminary comparison of several popular keyframe selection techniques using the ground truth derived from the annotations.

The success of our Signature-Based Video Browser presented last year at Video Browser Showdown 2014 (now renamed to Video Search Showcase) was mainly based on effective filtering using position-color feature signatures, while browsing in the results comprising matched keyframes was based just on a simple sequential search approach. Since the results can consist of highly similar keyframes (e.g., news studio scenes) making the browsing more difficult, we have enhanced our tool with more advanced browsing techniques considering also homogeneous result sets obtained after filtering phase. Furthermore, we have utilized improved search models based on feature signatures to make the filtering phase more effective.

The traditional content-based retrieval approaches usually use flat querying, where whole multimedia database is searched for a result of some similarity query with a user specified query object. However, there are retrieval scenarios (e.g., multimedia exploration), where users may not have a clear search intents in their minds, they just want to inspect a content of the multimedia collection. In such scenarios, flat querying is not suitable for the first phases of browsing, because it retrieves the most similar objects and does not consider a view on part of a multimedia space from different perspectives. Therefore, we defined a new Multilayer Exploration Structure (MLES), that enables exploration of a multimedia collection in different levels of details. Using the MLES, we formally defined popular exploration operations (zoom-in/out, pan) to enable horizontal and vertical browsing in explored space and we discussed several problems related to the area of multimedia exploration.

Multimedia exploration is an entertaining approach for multimedia retrieval enabling users to interactively browse and navigate through multimedia collections in a content-based way. The multimedia exploration approach extends the traditional query-by-example retrieval scenario to be a more intuitive approach for obtaining a global overview over an explored collection. However, novel exploration scenarios require many user studies demonstrating their benefits. In this paper, we present results of an extensive user study focusing on the comparison of 3-layer Multilayer Exploration Structure MLES structure with standard flat k-NN browsing. The results of the user study show that principles of the MLES lead to better effectiveness of the exploration process, especially when searching for a first object of the searched concept in an unknown collection.

During last decades, there have emerged various similarity models suitable for specific similarity search tasks. In this paper, we present a web-based portal that combines two popular similarity models (based on feature signatures and SURF descriptors) in order to improve the recall of multimedia exploration. Comparing to single-model approach, we demonstrate in the game-like fashion that a multi-model approach could provide users with more diverse and still relevant results.

In multi-dimensional databases the essential tool for accessing data is the range query (or window query). In this paper we introduce a new algorithm of processing range query in universal B-tree (UB-tree), which is an index structure for searching in multi-dimensional databases. The new range query algorithm (called the DRU algorithm) works efficiently, even for processing high-dimensional databases. In particular, using the DRU algorithm many of the UB-tree inner nodes need not to be accessed. We explain the DRU algorithm using a simple geometric model, providing a clear insight into the problem. More specifically, the model exploits an interesting relation between the Z-curve and generalized quad-trees. We also present experimental results for the DRU algorithm implementation.

In this paper, we investigate the content-based multimedia exploration techniques benefiting from the metric space indexing approach. We present two orthogonal approaches for browsing multimedia collections and discuss their strong and weak points. We also provide an implementation of the two approaches in our publicly available demo application where users can try to find as much objects of a predefined class as possible, given a limited time and/or a number of clicks.

In this demo paper, we present a video retrieval and browsing tool inspired by the natural human ability to memorize visual stimuli of color regions in video frames. Our tool utilizes feature signatures that can be used to represent both significant color regions in the key-frames and simple query sketches. As recently shown at the video browser showdown, such simple representation enables both effective end efficient interactive retrieval and browsing in video.

In this paper, we present a new signature-based video browser tool relying on the natural human ability to perceive and memorize visual stimuli of color regions in video frames. The tool utilizes feature signatures based on color and position extracted from the key frames in the preprocessing phase. Such content representation facilitates users in drawing simple query sketches and enables also effective and efficient processing of the query sketches. Besides user drawn simple sketches of desired scenes, the tool supports also several additional automatic content-based analysis techniques enabling restrictions to various concepts like faces or shapes.

In this paper, we present an effective yet efficient approach for known-item search in video data. The approach employs feature signatures based on color distribution to represent video key-frames. At the same time, the feature signatures enable users to intuitively draw simple colored sketches of the desired scene. We describe in detail the video retrieval model and also discuss and carefully optimize its parameters. Furthermore, several indexing techniques suitable for the model are presented and their performance is empirically evaluated in the experiments. Apart from that, we also investigate a bounding-sphere pruning technique suitable for similarity search in vector spaces.

With the huge expansion of smart devices and mobile applications, the ordinary users are consistently changing the conventional similarity search model. The users want to explore the multimedia data, so the typical query-by-example principle and the well-known keyword searching have become just a part of more complex retrieval processes. The emerging multimedia exploration systems with robust back-end retrieval system based on state of the art similarity search techniques provide a good solution. They enable interactive exploration process and implement exploration queries tightly connected with the user interface. However, they do not consider larger response times that might occur. To overcome this, we propose a scalable exploration system RTExp that allows evaluating the similarity queries in the near real time depending on user preferences (speed / precision). We describe building parts of the system and discuss various real-time characteristics for the exploration process. Also we provide results from the experimental evaluation of time-limited similarity queries and corresponding exploration operations.

The increasing amount of available unstructured content introduced a new concept of searching for information - the content-based retrieval. The principle behind is that the objects are compared based on their content which is far more complex than simple text or metadata based searching. Many indexing techniques arose to provide an efficient and effective similarity searching. However, these methods are restricted to a specific domain such as the metric space model. If this prerequisite is not fulfilled, indexing cannot be used, while each similarity search query degrades to sequential scanning which is unacceptable for large datasets. Inspired by previous successful results, we decided to apply the principles of genetic programming to the area of database indexing. We developed the GP-SIMDEX which is a universal framework that is capable of finding precise and efficient indexing methods for similarity searching for any given similarity data. For this purpose, we introduce the inequality symbolic regression principle and show how it helps the GP-SIMDEX Framework to find appropriate results that in most cases outperform the best-known indexing methods.

In this demo paper, we focus on the dynamic multimedia exploration techniques which are an intuitive, effective and entertaining way to present a pre-selected subset of a multimedia database to the users. More specifically, we present an exploration schema employing a similarity model based on SIFT descriptors that can be used to explore image database according to regions in the images. We also provide a simple mechanism to reduce the number of nonrelevant SIFT descriptors in the query image. The reduction of SIFTs in the query image improves the speed and fluency of the exploration process as demonstrated in our demo application.

The increasing diversity of unstructured databases leads to the development of advanced indexing techniques as the metric indexing model does not fit to the general similarity models. Once the most critical postulate, namely the triangle inequality, does not hold, the metric model produces notable errors during the query evaluation. To overcome this situation and to obtain more qualitative results, we want to discover better indexing models for databases using arbitrary similarity measures. However, each database is unique in a specific way, so we outline the automatic way of exploring the best indexing method. We introduce the exploration approach using parallel genetic programming principles in a multi-threaded environment built upon recently introduced SIMDEX Framework. Furthermore, we introduce smart pivot table which is an intelligent indexing method capable of incorporating obtained results. We supplement the theoretical background with experiments showing the achieved improvements in comparison to the single-threaded evaluations.

Recent popular applications like online video analysis or image exploration techniques utilizing content-based retrieval create a serious demand for fast and scalable feature extraction implementations. One of the promising content-based retrieval models is based on the feature signatures and the signature quadratic form distance. Although the model proved its competitiveness in terms of the effectiveness, the slow feature extraction comprising costly k-means clustering limits the model only for preprocessing steps. In this paper, we present a highly efficient multi-GPU implementation of the feature extraction process, reaching more than two orders of magnitude speedup with respect to classical CPU platform and the peak throughput that exceeds 8 thousand signatures per second. Such an implementation allows to extract requested batches of frames or images online without annoying delays. Moreover, besides online extraction tasks, our GPU implementation can be used also in a traditional preprocessing and training phase. For example, fast extraction allows indexing of huge databases or inspecting significantly larger parameter space when searching for an optimal similarity model configuration that is optimal according to both efficiency and effectiveness.

The signature quadratic form distance and feature signatures have become a respected similarity space for effective content-based retrieval. Furthermore, the similarity space is configurable by a parameter alpha affecting both retrieval precision and intrinsic dimensionality, and thus interesting trade-offs can be achieved when a metric index is used for exact search. In this paper we combine such configurable model with state of the art approximate search techniques developed for the M-Index. In the experiments, we show that employing a configuration resulting in the best effectiveness of the measure leads also to very competitive approximate search effectiveness when using the M-Index, regardless the high intrinsic dimensionality of the corresponding similarity space.