Search

To cope with ambiguous and/or underspecified queries, search result diversification (SRD) is a key technique that has attracted a lot of attention. This paper focuses on implicit SRD, where the possible subtopics underlying a query are unknown beforehand. We formulate implicit SRD as a process of selecting and ranking k exemplar documents that utilizes integer linear programming (ILP). Unlike the common practice of relying on approximate methods, this formulation enables us to obtain the optimal solution of the objective function. Based on four benchmark collections, our extensive empirical experiments reveal that: (1) The factors, such as different initial runs, the number of input documents, query types and the ways of computing document similarity significantly affect the performance of diversification models. Careful examinations of these factors are highly recommended in the development of implicit SRD methods. (2) The proposed method can achieve substantially improved performance over the state-of-the-art unsupervised methods for implicit SRD.

The second Asian Summer School in Information Access (ASSIA 2015) was held between 24th and 27th August, 2015 in Taipei City, Taiwan, R.O.C. The summer school offered 10 lectures in Information Retrieval, Social Media Search, and related topics, along with one panel discussion, a poster session, and a session from commercial sectors and young scholars. This reports a successful international summer school in Asia attracting a total of 45 participants from the range of countries in Asia and Europe.

iSchools have their roots in the collection, storage, analysis, and dissemination of recorded materials of human activities. We foresee that sensing data via lifelogging devices (or Internet of Things at large) is taking over its significant part in the coming years. This session for interaction and engagement (SIE) is to create a living lab environment where participants can experience various lifelogging devices such as wearable video recorders, wearable cameras, or audio recorders. The intended audience includes information behavioural researchers (both qualitative and quantitative), multimedia and/or UI developers, students who want to improve their work/life experience, and educators who explores the ways to develop reflective learning programs using lifelogging data. In addition to paper presentations, we will have a round-table session to identify some of the core research directions regarding the development and use of lifelog devices in Information Research and Learning.

The purpose of this half-day workshop is to bring together academic and industry interactive information retrieval (IIR) researchers with an interest in evaluation methodologies. The workshop articulates contemporary challenges in the investigation of IIR and invites user- and system-oriented researchers to work collaboratively to address these challenges by combining user- and system-centered methodologies in meaningful ways. We anticipate that this workshop will initiate productive knowledge exchange and partnerships that can respond to the increasing user, task, system, and contextual complexity of the IIR field.

The problem of content-based video retrieval continues to pose a challenge to the research community, the performance of video retrieval systems being low due to the semantic gap. In this paper we consider whether taking advantage of context can aid the video retrieval process by making the prediction of relevance easier, i.e. if it is easier for a classification system to predict the relevance of a video shot under a given context, then that context has potential in also improving retrieval, since the underlying features better differentiate relevant from non-relevant video shots. We use an operational definition of context, where datasets can be split into disjoint sub-collections which reflect a particular context. Contexts considered include task difficulty and user expertise, among others. In the classification process, four main types of features are used to represent video-shots: conventional low-level visual features representing physical properties of the video shots, behavioral features which are based on user interaction with the video shots, and two different bag-of-words features obtained from the Automatic Speech Recognition from the audio of the video. So, we measure how well each kind of video representation performs and, for each of these representations, our datasets are then split into different contexts in order to discover which contexts affect the performance of a number of trained classifiers. Thus, we aim to discover contexts which improve the classifier's performance and if this improvement is consistent regardless the kind of representation. Experimental results show which kind of the tested document representations works best for the different features; following on from this, we then identify the contexts which result in the classifiers performing better.

This paper presents an approach to detect personal highlights in videos based on the analysis of facial activities of the viewer. Our facial activity analysis was based on the motion vectors tracked on twelve key points in the human face. In our approach, the magnitude of the motion vectors represented a degree of a viewer's affective reaction to video contents. We examined 80 facial activity videos recorded for ten participants, each watching eight video clips in various genres. The experimental results suggest that useful motion vectors to detect personal highlights varied significantly across viewers. However, it was suggested that the activity in the upper part of face tended to be more indicative of personal highlights than the activity in the lower part.

Place names are often used to describe and to enquire about geographical information. It is common for users to employ vernacular names that have vague spatial extent and which do not correspond to the official and administrative place name terminology recorded within typical gazetteers. There is a need therefore to enrich gazetteers with knowledge of such vague places and hence improve the quality of place name-based information retrieval. Here we describe a method for modelling vague places using knowledge harvested from Web pages. It is found that vague place names are frequently accompanied in text by the names of more precise co-located places that lie within the extent of the target vague place. Density surface modelling of the frequency of co-occurrence of such names provides an effective method of representing the inherent uncertainty of the extent of the vague place while also enabling approximate crisp boundaries to be derived from contours if required. The method is evaluated using both precise and vague places. The use of the resulting approximate boundaries is demonstrated using an experimental geographical search engine.

In this paper, we present a novel Integer Linear Programming formulation (termed ILP4ID) for implicit search result diversification (SRD). The advantage is that the exact solution can be achieved, which enables us to investigate to what extent using the greedy strategy affects the performance of implicit SRD. Specifically, a series of experiments are conducted to empirically compare the state-of-the-art methods with the proposed approach. The experimental results show that: (1) The factors, such as different initial runs and the number of input documents, greatly affect the performance of diversification models. (2) ILP4ID can achieve substantially improved performance over the state-of-the-art methods in terms of standard diversity metrics.

This paper describes several steps in the derivation of boundaries of imprecise regions using the Web as the information source. We discuss how to use the Web to obtain locations that are part of and locations that are not part of the region to be delineated, and then we propose methods to compute the region algorithmically. The methods introduced are evaluated to judge the potential of the approach.

Time is one of the key constructs of information quality. Following an upsurge of research in temporal aspects of information search, it has become clear that the community needs standardized evaluation benchmark for fostering research in Temporal Information Access. This paper introduces Temporalia (Temporal Information Access), a new pilot task run at NTCIR-11 to create re-usable datasets for those who are interested in temporal aspects of search technologies, and discusses its task design in detail.

Temporal aspects of web search have gained a great level of attention in the recent years. However, many of the research attempts either focused on technical development of various tools or behavioral analysis based on log data. This paper presents the results of user survey carried out to investigate the practice and experience of temporal web search. A total of 110 people was recruited and answered 18 questions regarding their recent experience of web search. Our results suggest that an interplay of seasonal interests, technicality of information needs, target time of information, re-finding behaviour, and freshness of information can be important factors for the application of temporal search. These findings should be complementary to log analyses for further development of temporally aware search engines.

Considering search process in the evaluation of interactive information retrieval (IIR) is a challenging issue. This paper explores tempo of search actions (query, click, and judgement) to measure people's search process and performance. When we analyse how people consume their search resource (i.e., a total number of search actions taken to complete a task) over the time, it was observed that there was a different pattern in successful sessions and unsuccessful sessions. Successful sessions tend to have a regular tempo in search actions while poor sessions tend to have uneven distribution of resource usage. The resource consumption graph also allows us to observe where in the search process was affected by experimental conditions. Therefore, this paper suggests that tempo of search actions can be exploited to model successful search sessions.

Advance of search engines has removed an end-user’s constraint on search such as cost (free), operation time (fast response), language (cross-lingual), media (multimedia), and more recently, time-lag from content generation (realtime search). Consequently, we do not usually perceive strong constraint on the use of search engines. The removal of constraint is often seen as an improvement of a system’s functionality which should help people improve their search performance. However, it has also been suggested that constraint can have positive effects on people’s task performance. Therefore, we decided to study effects of search constraint on people’s perception, behaviour, and performance. Three types of constraint were investigated: time, time + number of queries one can submit, and time + number of documents one can view. A user study with 24 participants performing background search tasks suggests that 1) query and viewing constraint can increase people’s perceptional attention on query formulation and search results browsing; 2) query constraint has an effect not only on query formulation behaviour but click-through behaviour; and finally, 3) affected behaviour has medium effect on task performance. These findings warrant further studies to investigate how to leverage various forms of constraint effectively to improve human search performance.

Developing methods for searching image databases is a challenging and ongoing area of research. A common approach is to use manual annotations, although generating annotations can be expensive in terms of time and money, and therefore may not be justified in many situations. Content-based search techniques which extract visual features from image data can be used, but users are typically forced to express their information need using example images, or through sketching interfaces. This can be difficult if no visual example of the information need is available, or when the information need cannot be easily drawn.In this paper, we consider an alternative approach which allows a user to search for images through an intermediate database. In this approach, a user can search using text in the intermediate database as a way of finding visual examples of their information need. The visual examples can then be used to search a database that lacks annotations. Three experiments are presented which investigate this process. The first experiment automatically selects the image queries from the intermediary database; the second instead uses images which have been hand-picked by users. A third experiment, an interactive study, is then presented this study compares the intermediary interface to text search, where we consider text as an upper bound of performance. For this last study, an interface which supports the intermediary search process is described. Results show that while performance does not match manual annotations, users are able to find relevant material without requiring collection annotations.

With the increasing importance of search systems on the web, there is a continuing push to design interfaces which are a better match with the kinds of real-world tasks in which users are engaged. In this paper, we consider how broad, complex search tasks may be supported via the search interface. In particular, we consider search tasks which may be composed of multiple aspects, or multiple related subtasks. For example, in decision making tasks the user may investigate multiple possible solutions before settling on a single, final solution, while other tasks, such as report writing, may involve searching on multiple interrelated topics. A search interface is presented which is designed to support such broad search tasks, allowing a user to create search aspects, each of which models an independent subtask of some larger task. The interface is built on the intuition that users should be able to structure their searching environment when engaged on complex search tasks, where the act of structuring and organization may aid the user in understanding his or her task. A user study was carried out which compared our aspectual interface to a standard web-search interface. The results suggest that an aspectual interface can aid users when engaged in broad search tasks where the search aspects must be identified during searching; for a task where search aspects were pre-defined, no advantage over the baseline was found. Results for a decision making task were less clear cut, but show some evidence for improved task performance.

Recommender systems have been systematically applied in industry and academia to help users cope with information uncertainty. However, given the multiplicity of the preferences and needs it has been shown that no approach is suitable for all users in all situations. Thus, it is believed that an effective recommender system should incorporate a variety of techniques and features to offer valuable recommendations and enhance the search experience. In this paper we propose a novel video search interface that employs a multimodal recommender system, which can predict topical relevance. The multimodal recommender accounts for interaction data, contextual information, as well as users' affective responses, and exploits these information channels to provide meaningful recommendations of unseen videos. Our experiment shows that the multimodal interaction feature is a promising way to improve the performance of recommendation.

This paper presents an approach to affective video summarisation based on the facial expressions (FX) of viewers. A facial expression recognition system was deployed to capture a viewer's face and his/her expressions. The user's facial expressions were analysed to infer personalised affective scenes from videos. We proposed two models, pronounced level and expression's change rate, to generate affective summaries using the FX data. Our result suggested that FX can be a promising source to exploit for affective video summaries that can be tailored to individual preferences.

Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.

This paper revisits some of the established Information Retrieval (IR) techniques to investigate effective collaborative search strategies. We devised eight search strategies that divided labour and shared knowledge in teams using relevance feedback and clustering. We evaluated the performance of strategies with a user simulation enhanced by a query-pooling method. Our results show that relevance feedback is successful at formulating effective collaborative strategies while further effort is needed for clustering. We also measured the extent to which additional members improved the performance and an effect of search progress on the improvement.

This paper presents a user study that evaluated the effectiveness of an aggregated search interface in the context of non-navigational search tasks. An experimental system was developed to present search results aggregated from multiple information sources, and compared to a conventional tabbed interface. Sixteen participants were recruited to evaluate the performance of the two interfaces. Our results suggest that the aggregated search interface is a promising way of supporting non-navigational search tasks. The quantity and diversity of the retrieved items which participants accessed to complete a task, increased in the aggregated interface. Participants also found the aggregated presentation easier to access to retrieved items and to find relevant information, compared to the conventional interface.

Search interfaces are mainly designed to support a single searcher at a time. We therefore have a limited understanding of how an interface can support search where more than one searcher concurrently pursues a shared information need. This paper investigated the performance and user behaviour of concurrent search. Based on a recall-oriented search task, a user study was carried out to compare an independent search condition to collaborative search conditions. The results show that the collaborative conditions helped searchers diversify search vocabulary while reducing redundant documents to be bookmarked within teams. However, these effects were found to be insufficient to improve the retrieval effectiveness. We discussed the implications for concurrent search support based on our findings.

Generating query-biased summaries can take up a large part of the response time of interactive information retrieval (IIR) systems. This paper proposes to use document titles as an alternative to queries in the generation of summaries. The use of document titles allows us to pre-generate summaries statically, and thus, improve the response speed of IIR systems. Our experiments suggest that title-biased summaries are a promising alternative to query-biased summaries.

Much of the information stored on the web contains geographical context, but current search engines treat such context in the same way as all other content. In this paper we describe the design, implementation and evaluation of a spatially aware search engine which is capable of handling queries in the form of the triplet of ⟨theme⟩⟨spatial relationship⟩⟨location⟩. The process of identifying geographic references in documents and assigning appropriate footprints to documents, to be stored together with document terms in an appropriate indexing structure allowing real‐time search, is described. Methods allowing users to query and explore results which have been relevance‐ranked in terms of both thematic and spatial relevance have been implanted and a usability study indicates that users are happy with the range of spatial relationships available and intuitively understand how to use such a search engine. Normalised precision for 38 queries, containing four types of spatial relationships, is significantly higher (p

Finding significant contextual features is a challenging task in the development of interactive information retrieval (IR) systems. This paper investigated a simple method to facilitate such a task by looking at aggregated relevance judgements of retrieved documents. Our study suggested that the agreement on relevance judgements can indicate the effectiveness of retrieved documents as the source of significant features. The effect of highly agreed documents gives us practical implication for the design of adaptive search models in interactive IR systems.

Geographic Information Retrieval (GIR) is concerned with the retrieval of documents based on both thematic and geographic content. An important issue in GIR, as for all IR, is relevance. In this paper we argue that spatial relevance should be considered independently from thematic relevance, and propose an initial scheme. A pilot study to assess this relevance scheme is presented, with initial results suggesting that users can distinguish between these two relevance dimensions, and that furthermore they have different properties. We suggest that spatial relevance requires greater assessor effort and more localised geographic knowledge than judging thematic relevance.

Presentation of search results in Web-based information retrieval (IR) systems has been dominated by a textual form of information such as the title, snippet, URL, and/or file type of retrieved documents. On the other hand, document's visual aspects such as the layout, colour scheme, or presence of images have been studied in a limited context with regard to their effectiveness of search result presentation. This paper presents a comparative evaluation of textual and visual forms of document summaries as the additional document surrogate in the search result presentation. In our study, a sentence-based summarisation technique was used to create a textual document summary, and the thumbnail image of web pages was used to represent a visual summary. The experimental results suggest that both have the cases where the additional elements contributed to a positive effect not only in users' relevance assessment but also in query re/formulation. The results also suggest that the two forms of document summary are likely to have different contexts to facilitate user's search experience. Therefore, our study calls for further research on adaptive models of IR systems to make use of their advantages in appropriate contexts.

Many web documents refer to specific geographic localities and many\ud people include geographic context in queries to web search engines. Standard\ud web search engines treat the geographical terms in the same way as other terms.\ud This can result in failure to find relevant documents that refer to the place of\ud interest using alternative related names, such as those of included or nearby\ud places. This can be overcome by associating text indexing with spatial indexing\ud methods that exploit geo-tagging procedures to categorise documents with\ud respect to geographic space. We describe three methods for spatio-textual\ud indexing based on multiple spatially indexed text indexes, attaching spatial\ud indexes to the document occurrences of a text index, and merging text index\ud access results with results of access to a spatial index of documents. These\ud schemes are compared experimentally with a conventional text index search\ud engine, using a collection of geo-tagged web documents, and are shown to be\ud able to compete in speed and storage performance with pure text indexing.

A large scale collection of web pages has been essential for research in information retrieval and related areas. This paper provides an overview of a large web collection used in the SPIRIT project for the design and testing of spatially-aware retrieval systems. Several statistics are derived and presented to show the characteristics of the collection.

Forming test collection relevance judgments from the pooled output of multiple retrieval systems has become the standard process for creating resources such as the TREC, CLEF, and NTCIR test collections. This paper presents a series of experiments examining three different ways of building test collections where no system pooling is used. First, a collection formation technique combining manual feedback and multiple systems is adapted to work with a single retrieval system. Second, an existing method based on pooling the output of multiple manual searches is re-examined: testing a wider range of searchers and retrieval systems than has been examined before. Third, a new approach is explored where the ranked output of a single automatic search on a single retrieval system is assessed for relevance: no pooling whatsoever. Using established techniques for evaluating the quality of relevance judgments, in all three cases, test collections are formed that are as good as TREC.

A medium-scale user study was carried out to investigate the usability\ud of a concept-based query expansion support tool. The tool was fully integrated\ud into the interface of an IR system, and designed to support the user by offering\ud automatically generated concept hierarchies. Two types of hierarchies were compared\ud with a baseline. Several observations were made as a result of the study:\ud 1) the hierarchy is often accessed after an examination of the first page of search\ud results; 2) accessing the hierarchies reduces the number of iterations and paging\ud actions; 3) accessing the hierarchies increases the chance of finding relevant items\ud more accurately than the baseline; 4) the hierarchical structure helps the users to\ud handle a large number of concepts; and finally, 5) subjects were not aware of the\ud difference between two types of hierarchies.

Our demonstration shows the hierarchy system working on a locally run search engine. Hierarchies are dynamically generated from the retrieved documents, and visualised on the menus. When a user selects a term from the hierarchy, the documents linked to the term are listed, and the term is then added to the initial query to rerun a search. Through the demonstration we illustrate how hierarchical presentation of expansion terms is achieved, and how our approach supports users to articulate their information needs using the hierarchy.

GeoCLEF was a new pilot track in CLEF 2005. GeoCLEF was to test and evaluate cross-language geographic information retrieval (GIR) of text. Geographic information retrieval is retrieval oriented toward the geographic specification in the description of the search topic and returns documents which satisfy this geographic information need. For GeoCLEF 2005, twenty-five search topics were defined for searching against the English and German ad-hoc document collections of CLEF. Topic languages were English, German, Portuguese and Spanish. Eleven groups submitted runs and about 25,000 documents (half English and half German) in the pooled runs were judged by the organizers. The groups used a variety of approaches, including geographic bounding boxes and external knowledge bases (geographic thesauri and ontologies and gazetteers). The results were encouraging but showed that additional work needs to be done to refine the task for GeoCLEF in 2006.