Search

Practical news feed platforms generate a hybrid list of news articles and advertising items (e.g., products, services, or information) and many platforms optimize the position of news articles and advertisements independently. However, they should be arranged with careful consideration of each other, as we show in this study, since user behaviors toward advertisements are significantly affected by the news articles. This paper investigates the effect of news articles on users' ad consumption and shows the dependency between news and ad effectiveness. We conducted a service log analysis and showed that sessions with high-quality news article exposure had more ad consumption than those with low-quality news article exposure. Based on this result, we hypothesized that exposure to high-quality articles will lead to a high ad consumption rate. Thus, we conducted million-scale A/B testing to investigate the effect of high-quality articles on ad consumption, in which we prioritized high-quality articles in the ranking for the treatment group. The A/B test showed that the treatment group's ad consumption, such as the number of clicks, conversions, and sales, increased significantly while the number of article clicks decreased. We also found that users who prefer a social or economic topic had more ad consumption by stratified analysis. These insights regarding news articles and advertisements will help optimize news and ad effectiveness in rankings considering their mutual influence., Comment: 30th ACM International Conference on Information and Knowledge Management (CIKM2021)

In this study, we propose an efficient method for comparing the post-click metric (e.g., dwell time and conversion rate) of multiple rankings in online experiments. The proposed method involves (1) the decomposition of the post-click metric measurement of a ranking into a click model estimation and a post-click metric measurement of each item in the ranking, and (2) interleaving of multiple rankings to produce a single ranking that preferentially exposes items possessing a high population variance. The decomposition of the post-click metric measurement enables the free layout of items in a ranking and focuses on the measurement of the post-click metric of each item in the multiple rankings. The interleaving of multiple rankings reduces the sample variance of the items possessing a high population variance by optimizing a ranking to be presented to the users so that those items received more samples of the post-click metric. In addition, we provide a proof that the proposed method leads to the minimization of the evaluation error in the ranking comparison and propose two practical techniques to stabilize the online experiment. We performed a comprehensive simulation experiment and a real service setting experiment. The experimental results revealed that (1) the proposed method outperformed existing methods in terms of efficiency and accuracy, and the performance was especially remarkable when the input rankings shared many items, and (2) the two stabilization techniques successfully improved the evaluation accuracy and efficiency., Comment: The 7th ACM SIGIR International Conference on the Theory of Information Retrieval (ICTIR2021)

This study presents a theoretical analysis on the efficiency of interleaving, an efficient online evaluation method for rankings. Although interleaving has already been applied to production systems, the source of its high efficiency has not been clarified in the literature. Therefore, this study presents a theoretical analysis on the efficiency of interleaving methods. We begin by designing a simple interleaving method similar to ordinary interleaving methods. Then, we explore a condition under which the interleaving method is more efficient than A/B testing and find that this is the case when users leave the ranking depending on the item's relevance, a typical assumption made in click models. Finally, we perform experiments based on numerical analysis and user simulation, demonstrating that the theoretical results are consistent with the empirical results., Comment: The 45th European Conference on Information Retrieval (ECIR2023)

This paper addresses the problem of generating table captions for scholarly documents, which often require additional information outside the table. To this end, we propose a method of retrieving relevant sentences from the paper body, and feeding the table content as well as the retrieved sentences into pre-trained language models (e.g. T5 and GPT-2) for generating table captions. The contributions of this paper are: (1) discussion on the challenges in table captioning for scholarly documents; (2) development of a dataset DocBank-TB, which is publicly available; and (3) comparison of caption generation methods for scholarly documents with different strategies to retrieve relevant sentences from the paper body. Our experimental results showed that T5 is the better generation model for this task, as it outperformed GPT-2 in BLEU and METEOR implying that the generated text are clearer and more precise. Moreover, inputting relevant sentences matching the row header or whole table is effective.

This panel will address the issues associated with the practice and service of open research data curation and discovery from a global perspective. The sub-fields of information science such as information retrieval, information curation, information practices and human-centered data science have approached the open research data initiatives from multiple lenses. The issues of data creation, capturing, curation, sharing, discovery and reuse of cut across the sub-fields. We will identify and discuss the emerging themes in open data curation and discovery drawing on active research projects, repository practices and research data capturing and reuse in a selection of disciplines from health domain to archaeology and cultural heritage.

This panel will address the issues associated with the practice and service of open research data curation and discovery from a global perspective. The sub-fields of information science such as information retrieval, information curation, information practices and human-centered data science have approached the open research data initiatives from multiple lenses. The issues of data creation, capturing, curation, sharing, discovery and reuse of cut across the sub-fields. We will identify and discuss the emerging themes in open data curation and discovery drawing on active research projects, repository practices and research data capturing and reuse in a selection of disciplines from health domain to archaeology and cultural heritage.

This panel will address the issues associated with the practice and service of open research data curation and discovery from a global perspective. The sub-fields of information science such as information retrieval, information curation, information practices and human-centered data science have approached the open research data initiatives from multiple lenses. The issues of data creation, capturing, curation, sharing, discovery and reuse of cut across the sub-fields. We will identify and discuss the emerging themes in open data curation and discovery drawing on active research projects, repository practices and research data capturing and reuse in a selection of disciplines from health domain to archaeology and cultural heritage.

This panel will address the issues associated with the practice and service of open research data curation and discovery from a global perspective. The sub-fields of information science such as information retrieval, information curation, information practices and human-centered data science have approached the open research data initiatives from multiple lenses. The issues of data creation, capturing, curation, sharing, discovery and reuse of cut across the sub-fields. We will identify and discuss the emerging themes in open data curation and discovery drawing on active research projects, repository practices and research data capturing and reuse in a selection of disciplines from health domain to archaeology and cultural heritage.

This panel will address the issues associated with the practice and service of open research data curation and discovery from a global perspective. The sub-fields of information science such as information retrieval, information curation, information practices and human-centered data science have approached the open research data initiatives from multiple lenses. The issues of data creation, capturing, curation, sharing, discovery and reuse of cut across the sub-fields. We will identify and discuss the emerging themes in open data curation and discovery drawing on active research projects, repository practices and research data capturing and reuse in a selection of disciplines from health domain to archaeology and cultural heritage.

Automatic generation of natural language from images has attracted extensive attention. In this paper, we take one step further to investigate generation of poetic language (with multiple lines) to an image for automatic poetry creation. This task involves multiple challenges, including discovering poetic clues from the image (e.g., hope from green), and generating poems to satisfy both relevance to the image and poeticness in language level. To solve the above challenges, we formulate the task of poem generation into two correlated sub-tasks by multi-adversarial training via policy gradient, through which the cross-modal relevance and poetic language style can be ensured. To extract poetic clues from images, we propose to learn a deep coupled visual-poetic embedding, in which the poetic representation from objects, sentiments and scenes in an image can be jointly learned. Two discriminative networks are further introduced to guide the poem generation, including a multi-modal discriminator and a poem-style discriminator. To facilitate the research, we have released two poem datasets by human annotators with two distinct properties: 1) the first human annotated image-to-poem pair dataset (with 8,292 pairs in total), and 2) to-date the largest public English poem corpus dataset (with 92,265 different poems in total). Extensive experiments are conducted with 8K images, among which 1.5K image are randomly picked for evaluation. Both objective and subjective evaluations show the superior performances against the state-of-the-art methods for poem generation from images. Turing test carried out with over 500 human subjects, among which 30 evaluators are poetry experts, demonstrates the effectiveness of our approach.

We describe DCU’s participation in the NTCIR-12 SpokenQuery&Doc (SQD-2) task. In the context of the slide-group retrieval sub-task, we experiment with a passage retrieval method that re-scores each passage according to the relevance score of the document from which the passage is taken. This is performed by linearly interpolating their relevance scores which are calculated using the Okapi BM25 model of probabilistic retrieval for passages and documents independently. In conjunction with this, we assess the benefits of using pseudo-relevance feedback for expanding the textual representation of the spoken queries with terms found in the top-ranked documents and passages, and experiment with a general multidimensional optimisation method to jointly tune the BM25 and query expansion parameters with queries and relevance data from the NTCIR-11 SQD-1 task. Retrieval experiments performed over the SQD-1 and SQD-2 queries confirm previous findings which affirm that integrating document information when ranking passages can lead to improved passage retrieval effectiveness. Furthermore, results indicate that no significant gains in retrieval effectiveness can be obtained by using query expansion in combination with our retrieval models over these two query sets.

This paper presents an overview of the Spoken Query and Spoken Document retrieval (SpokenQuery&Doc-2) task at the NTCIR-12 Workshop. This task included spoken query driven spoken content retrieval (SQ-SCR) and a spoken query driven spoken term detection (SQ-STD) as the two subtasks. The paper describes details of each sub-task, the data used, the creation of the speech recognition systems used to create the transcripts, the design of the retrieval test collections, the metrics used to evaluate the sub-tasks and a summary of the results of submissions by the task participants.

We describe DCU’s participation in the NTCIR-12 SpokenQuery&Doc (SQD-2) task. In the context of the slide-group retrieval sub-task, we experiment with a passage retrieval method that re-scores each passage according to the relevance score of the document from which the passage is taken. This is performed by linearly interpolating their relevance scores which are calculated using the Okapi BM25 model of probabilistic retrieval for passages and documents independently. In conjunction with this, we assess the benefits of using pseudo-relevance feedback for expanding the textual representation of the spoken queries with terms found in the top-ranked documents and passages, and experiment with a general multidimensional optimisation method to jointly tune the BM25 and query expansion parameters with queries and relevance data from the NTCIR-11 SQD-1 task. Retrieval experiments performed over the SQD-1 and SQD-2 queries confirm previous findings which affirm that integrating document information when ranking passages can lead to improved passage retrieval effectiveness. Furthermore, results indicate that no significant gains in retrieval effectiveness can be obtained by using query expansion in combination with our retrieval models over these two query sets.

This paper presents an overview of the Spoken Query and Spoken Document retrieval (SpokenQuery&Doc-2) task at the NTCIR-12 Workshop. This task included spoken query driven spoken content retrieval (SQ-SCR) and a spoken query driven spoken term detection (SQ-STD) as the two subtasks. The paper describes details of each sub-task, the data used, the creation of the speech recognition systems used to create the transcripts, the design of the retrieval test collections, the metrics used to evaluate the sub-tasks and a summary of the results of submissions by the task participants.

We propose a method of searching for geographic entities in an unknown place with a query by example in a known place. Geographic entity searches are installed in many Web sites such as those for shopping and restaurants. Most of the sites hold classic attribute-based or keyword-based interfaces for entity retrieval; however, specifying each attribute users want is time-consuming, and keywords are not effective for representing users’ complex intentions. The proposed query by example method in a map interface allows users to intuitively query by selecting entities in places they know well. The most similar entities to an input are returned based on the similarity varying with individuals. Our proposed method is robust for estimating the similarity using not only selected examples, but also implicit negative feedback, which is predicted by how the user selects examples as a query in the map interface. Experimental results proved the effectiveness of our method, and the performance exceeded that of a previously proposed method.

We propose a query-by-example geographic object search method for users that do not know well about the place they are in. Geographic objects, such as restaurants, are often retrieved using an attribute-based or keyword query. These methods, however, are dif- ficult to use for users that have little knowledge on the place where they want to search. The proposed query-by-example method allows users to query by selecting examples in familiar places for retrieving objects in unfamiliar places. One of the challenges is to predict an effective distance metric, which varies for individuals. Another challenge is to calculate the distance between objects in heterogeneous domains considering the feature gap between them, for example, restaurants in Japan and China. Our proposed method is used to robustly estimate the distance metric by amplifying the difference between selected and non-selected examples. By using the distance metric, each object in a familiar domain is evenly assigned to one in an unfamiliar domain to eliminate the difference between those domains. We developed a restaurant search using data obtained from a Japanese restaurant Web guide to evaluate our method.

We propose a query-by-example geographic object search method for users that do not know well about the place they are in. Geographic objects, such as restaurants, are often retrieved using an attribute-based or keyword query. These methods, however, are dif- ficult to use for users that have little knowledge on the place where they want to search. The proposed query-by-example method allows users to query by selecting examples in familiar places for retrieving objects in unfamiliar places. One of the challenges is to predict an effective distance metric, which varies for individuals. Another challenge is to calculate the distance between objects in heterogeneous domains considering the feature gap between them, for example, restaurants in Japan and China. Our proposed method is used to robustly estimate the distance metric by amplifying the difference between selected and non-selected examples. By using the distance metric, each object in a familiar domain is evenly assigned to one in an unfamiliar domain to eliminate the difference between those domains. We developed a restaurant search using data obtained from a Japanese restaurant Web guide to evaluate our method.

We propose a method of searching for geographic entities in an unknown place with a query by example in a known place. Geographic entity searches are installed in many Web sites such as those for shopping and restaurants. Most of the sites hold classic attribute-based or keyword-based interfaces for entity retrieval; however, specifying each attribute users want is time-consuming, and keywords are not effective for representing users’ complex intentions. The proposed query by example method in a map interface allows users to intuitively query by selecting entities in places they know well. The most similar entities to an input are returned based on the similarity varying with individuals. Our proposed method is robust for estimating the similarity using not only selected examples, but also implicit negative feedback, which is predicted by how the user selects examples as a query in the map interface. Experimental results proved the effectiveness of our method, and the performance exceeded that of a previously proposed method.

We propose a query-by-example geographic object search method for users that do not know well about the place they are in. Geographic objects, such as restaurants, are often retrieved using an attribute-based or keyword query. These methods, however, are dif- ficult to use for users that have little knowledge on the place where they want to search. The proposed query-by-example method allows users to query by selecting examples in familiar places for retrieving objects in unfamiliar places. One of the challenges is to predict an effective distance metric, which varies for individuals. Another challenge is to calculate the distance between objects in heterogeneous domains considering the feature gap between them, for example, restaurants in Japan and China. Our proposed method is used to robustly estimate the distance metric by amplifying the difference between selected and non-selected examples. By using the distance metric, each object in a familiar domain is evenly assigned to one in an unfamiliar domain to eliminate the difference between those domains. We developed a restaurant search using data obtained from a Japanese restaurant Web guide to evaluate our method.

We propose a method of searching for geographic entities in an unknown place with a query by example in a known place. Geographic entity searches are installed in many Web sites such as those for shopping and restaurants. Most of the sites hold classic attribute-based or keyword-based interfaces for entity retrieval; however, specifying each attribute users want is time-consuming, and keywords are not effective for representing users’ complex intentions. The proposed query by example method in a map interface allows users to intuitively query by selecting entities in places they know well. The most similar entities to an input are returned based on the similarity varying with individuals. Our proposed method is robust for estimating the similarity using not only selected examples, but also implicit negative feedback, which is predicted by how the user selects examples as a query in the map interface. Experimental results proved the effectiveness of our method, and the performance exceeded that of a previously proposed method.

We propose a query-by-example geographic object search method for users that do not know well about the place they are in. Geographic objects, such as restaurants, are often retrieved using an attribute-based or keyword query. These methods, however, are dif- ficult to use for users that have little knowledge on the place where they want to search. The proposed query-by-example method allows users to query by selecting examples in familiar places for retrieving objects in unfamiliar places. One of the challenges is to predict an effective distance metric, which varies for individuals. Another challenge is to calculate the distance between objects in heterogeneous domains considering the feature gap between them, for example, restaurants in Japan and China. Our proposed method is used to robustly estimate the distance metric by amplifying the difference between selected and non-selected examples. By using the distance metric, each object in a familiar domain is evenly assigned to one in an unfamiliar domain to eliminate the difference between those domains. We developed a restaurant search using data obtained from a Japanese restaurant Web guide to evaluate our method.