Search

In order to overcome the inefficiency and resource consumption of full-text search in relational databases, a light full-text search model with auxiliary cache is developed. Specially, we utilize the MySQL as the data storage layer and the Redis as the index cache layer. We first design a full-index cache mechanism by the Redis-based inverted indexes construction methods to augment the efficient memory processing capability of relational databases. In addition, an increment-index synchronization mechanism is implemented to fit the dynamic update of relation database. For hot data, an index update optimization mechanism is provided to guarantee the fast response and accuracy of full-text search. The proposed Redis-based auxiliary cache method has also been put into practical industrial applications and achieved promising results. Finally, we evaluate our method from index space occupation, time consumption and the accuracy of retrieval results. The experimental results show that the proposed model outperforms MySQL Full-Text method 2–3 times and surpasses ElasticSearch 12 times in space resource consumption. [ABSTRACT FROM AUTHOR]

Full-text search has a wide range of applications, including tracking systems, computer vision, and natural language processing. Standard methods usually implement a two-phase procedure: indexing and retrieving, with the retrieval performance entirely dependent on the index efficiency. In most cases, the more powerful the index algorithm, the more memory and processing time are required. The amount of time and memory required to index a collection of documents is proportional to its overall size. In this paper, we propose a full-text search hardware implementation without the indexing phase, thus removing the time and memory requirements for indexing. Additionally, we propose an efficient design to leverage the parallel architecture of High Bandwidth Memory (HBM). To our knowledge, few (if not zero) researchers have integrated their full-text search system with an effective data access control on HBM. The functionality of the proposed system is verified on the Xilinx Alveo U50 Field-Programmable Gate Array (FPGA). The experimental results show that our system achieved a throughput of 8 Gigabytes per second, about 6697× speed-up compared to other software-based approaches. [ABSTRACT FROM AUTHOR]

Parole-Simple-Clips (PSC) is a computational lexicon of the Italian language, developed from 1996 to 2003 by the Institute of Computational Linguistics of the Italian National Research Council (ILC-CNR) in the context of national and European projects. The PSC resource is strongly structured and rich of data, and may provide an edge over existing linguistic resources if used in the support of NLP and text retrieval related tasks, such as, as currently being experimented, for full-text search. However, the lexicon still appears incomplete and contains some redundant, erroneous and missing data. This paper documents the first steps undertaken for the creation of LexicO, an Italian computational lexicon built upon PSC starting from an in depth analysis of its four linguistic layers (semantic, syntactic, morphological and phonological) in which it is structured. As a result of this work, LexicO has been released and made freely available for download.

A chatbot is an application that can respond and reply to questions from the questioner, without any human being involved. In general, chatbots are used to simplify and improve public services, so that users can get information quickly and directly, without time constraints. This research objective is to create promotional media in the form of chatbots in the 3D Virtual Tour to answer questions and provide general information about an object while providing more attractive visuals for users. The method used to run the chatbot is Full-Text Search. Full-Text Search is an efficient and powerful search method that is built into MySQL by default, which works by matching words between keywords and patterns in the database. This study succeeded in applying the Full-Text Search method as a search technique for chatbot patterns in the 3D Virtual Tour application at Buddhi Dharma University as a promotional medium. Evaluation using the User Acceptance Test (UAT) method yielded a value of 83.45% in a very satisfying category for users.

In the era of big data, data has never been more important because it contains hidden insights. Additionally, it is necessary and challenging to extract usable information from enormous volumes of data. When attempting to perform data processing and analytics in a variety of domains, developers of data-intensive systems have consequently met several challenges. In addition, full-text search is one of the most significant components of big data processing and analytics for discovering fragments of required data among large volumes of data. Due to the importance of the subject, this article begins with an examination of the characteristics, capabilities, and technical comparisons of full-text search technologies, followed by a systematic comparison of Apache Solr and Elasticsearch in terms of indexing times and queries on three separate datasets. According to our findings, based on default configuration, Apache Solr has better performance when looking at indexing times measured on three machines with different hardware specifications. Likewise, Apache Solr outperforms Elasticsearch in seven out of ten search queries. Regarding our results, on computers with restricted hardware resources, we recommend utilizing Apache Solr instead of Elasticsearch. In addition, this study provides researchers and developers of data-intensive systems with a complete comparison and suggestions for choosing the most effective fulltext search engine for their task. [ABSTRACT FROM AUTHOR]

The newspaper emerged as a distinct cultural form in early 17th-century Europe. It is bound up with the early modern period of history. Historical newspapers are of utmost importance to nations and its people, and researchers from different disciplines rely on these papers to improve our understanding of the past. In pursuit of satisfying this need, Istanbul University Head Office of Library and Documentation provides access to a big database of scanned historical newspapers. To take it another step further and make the documents more accessible, we need to run optical character recognition (OCR) and named entity recognition (NER) tasks on the whole database and index the results to allow for full-text search mechanism. We design and implement a system encompassing the whole pipeline starting from scrapping the dataset from the original website to providing a graphical user interface to run search queries, and it manages to do that successfully. Proposed system provides to search people, culture and security-related keywords and to visualise them. [ABSTRACT FROM AUTHOR]

Let there be a set of objects, where each object is characterized by Boolean features. Let logical query or Boolean query denote a query to find objects characterized by a Boolean function by features, for example, “documents with all words ...and any words ...and without words ...”. The terms “object” and “document” are used interchangeably hereinafter. The features may have different semantics, i. e. some features may correspond to the words of the document, some to labels or categories, and some to per-bit time quantum of a document’s date. Although indices executing Boolean queries are well researched in the literature, the common technique of maintaining posting lists is not always acceptable. If the data volume can reach to the order of petabytes, a compact index structure becomes vital. The aim of the research is to suggest a method to build an efficient bitmap index in secondary memory that allows us to update or append data being indexed with high write throughput. We propose an efficient LSM-based index for bitmaps and feature design for practical applications. We also discuss aspects of building of joined indices in order to achieve good scalability. The paper describes the architecture and algorithms of a suggested index and includes results of our experiments that show sustainable performance of our solution.

Purpose/Thesis: The article aims to analyze the conditions and possibilities of full-text search in Polish digital libraries (DL), taking into account the access to full-text search in individual DLs, the file formats and the software used, as well as the visibility of the DL resources in the Google search engine. Approach/Methods: Forty of the largest Polish DLs, whose resources primarily comprise digitized traditional library resources, were selected for the study. The study examined the type of the software used, the availability and efficiency of the full-text search, and the extent to which the resources were indexed as PDF files in Google and Google Scholar. Finally, the study compared the results of full-text search in ten DLs with those obtained from Google. Results and conclusions: All DLs selected for the study allow for full-text search. There are significant differences between specific interfaces. Each has advantages and disadvantages that require further development. The Google search engine is not currently a viable alternative to accessing content provided in DLs. Originality/Value: The issue of full-text search in DLs is rarely addressed, even though users consider it one of the most important functions of DLs. The result of the study present a picture of DL's current capacity in this area. [ABSTRACT FROM AUTHOR]

B-Tree based text indexes used in MongoDB are slow compared to different structures such as inverted indexes. In this study, it has been shown that the full-text search speed can be increased significantly by indexing a structure in which each different word in the text is included only once. The Multi-Stream Word-Based Compression Algorithm (MWCA), developed in our previous work, stores word dictionaries and data in different streams. While adding the documents to a MongoDB collection, they were encoded with MWCA and separated into six different streams. Each stream was stored in a different field, and three of them containing unique words were used when creating a text index. In this way, the index could be created in a shorter time and took up less space. It was also seen that Snappy and Zlib block compression methods used by MongoDB reached higher compression ratios on data encoded with MWCA. Search tests on text indexes created on collections using different compression options shows that our method provides 19 to 146 times speed increase and 34% to 40% less memory usage. Tests on regex searches that do not use the text index also shows that the MWCA model provides 7 to 13 times speed increase and 29% to 34% less memory usage. [ABSTRACT FROM AUTHOR]

First, the author will give an overview of the "Flexible Right Limitation Provisions" newly established by the Copyright Act Amendment in 2018. Then, the author will introduce the National Diet Library's initiatives related to these provisions, such as the project to convert digitized materials (images) into text data, the full-text search services and the development of services to improve the convenience of browsing and searching functions of digitized materials using AI (machine learning). The impact of the 2018 revision of the Copyright Act will also be discussed. [ABSTRACT FROM AUTHOR]

Address matching is one of the key technologies in the investigation and analysis of origin-destination in hazardous materials transportation. In order to help solving the problem of low accuracy of complex non-standard address matching for hazardous materials road transportation, this paper adopted self-expanding knowledge graph of address data, proposed a self-expanding Chinese word segmentation method and an address matching mechanism based on full-text index. Besides, this paper incorporated a weighted Pinyin full-text search mechanism to improve the matching accuracy for misspelling addresses. The online interface of geographic resolving and semi-supervised matching mechanism made the proposed address matching system a complete one. The matching results of the address data of the electronic waybill for the hazardous materials transportation show a high accuracy and high precision. The accuracy is over 94.6% for random address dataset and 67.5% for difficult-to-parse address dataset, both are much higher than the general matching methods and geographic search methods in terms of the accuracy and precision of address matching. [ABSTRACT FROM AUTHOR]

Summary: Digital document collections have been created with the transfer of a large number of documents to digital media. These digital archives have provided many benefits to users. As the diversity and size of digital image collections have grown exponentially, it has become increasingly important and difficult to obtain the desired image from them. The images on the document might contain critical information about the subject of it. In this study, an architecture is developed that can work on large‐scale data by creating regular expressions together with full‐text search approaches. The performance of the system has been tested on different academic documents and Elasticsearch and Apache Solr insert times are compared. Compared to Elasticsearch, Apache Solr achieved faster and more successful results. [ABSTRACT FROM AUTHOR]

Das Münchener Digitalisierungszentrum (MDZ) der Bayerischen Staatsbibliothek führte in fast zwei Jahren Projektlaufzeit einen grundlegenden technologischen Relaunch seiner „Digitalen Sammlungen" durch. Das rundum erneuerte und weitestgehend barrierefreie Angebot bietet eine optimierte Volltextsuche. Ein Großteil der Digitalisate wird im innovativen IIIF-Viewer „Mirador 3" präsentiert, der u. a. auch eine Überblendfunktion der Volltexte über die zugehörigen Bilder ermöglicht. Die Inhalte werden frei im Internet zur Verfügung gestellt: komfortable Downloadfunktionen und öffentlich zugängliche Schnittstellen erlauben eine Nachnutzung für nicht-kommerzielle Zwecke. After almost two years of work, the Munich Digitization Center (MDZ) of the Bavarian State Library completed a fundamental technological relaunch of its "Digital Collections". The service was completely remodeled and is now to a large extent barrier-free. It features optimized full-text search, the majority of the digitized objects is presented with IIIF viewer "Mirador 3". Content is freely available: convenient download options and publicly accessible APIs allow re-use for non-commercial purposes. [ABSTRACT FROM AUTHOR]

HDFS is a popular open-source system for scalable and reliable file management, which is designed as a general-purpose solution for distributed file storage. While it works well for medium or large files, it will suffer heavy performance degradations in case of lots of small files. To overcome this drawback, we propose here a system to enhance HDFS with a distributed true full-text search system SAES of 100% recall and precision ratios. By indexing the meta data of each file, e.g., name, size, date and description, files can be quickly accessed by efficient searches over metadata. Moreover, by merging many small files into a large file to be stored with better space and I/O efficiencies, the negative performance impacts caused by directly storing each small file individually are avoided. An experimental study is conducted for function and performance tests on both realistic and artificial data. The experimental results show that the system works well for file operations such as uploading, downloading and deleting. Moreover, the RAM consumption for managing massive small files is dramatically reduced, which is critical for good system performance. The proposed system could be a potential storage solution for massive small files. [ABSTRACT FROM AUTHOR]

The research aims to create a smart user Support Assistant LAVA - a solution providing context sensitive and user experience aware support to a user of a software product. The solution is based on machine learning and search in a two-level distributed knowledge store. LAVA model focuses on providing user on a mouse click with a content relevant to a current context. LAVA uses context transferred from the supported product to instantly show a FAQ list of items used and/or positively rated by other users in the same context. Full-text search and a topic hierarchy are provided as well to cover as many of cases as possible given the content available currently in the Knowledge base. Main design ideas of the LAVA model are presented in the article including the calculation of ranges and creation of ranged context-sensitive FAQs, customizing the full-text search, organizing the topic hierarchy and integrating with the Service desk. The article covers as well the main considerations on how the LAVA model should be implemented in a successful solution, and a description of method to evaluate the model performance. [ABSTRACT FROM AUTHOR]

Previous studies have shown that digital humanities studies are mostly based on content exploration by analyzing text content, and lack of auxiliary reading tools from the perspective of user reading behavior. The annotation of ancient books can increase the interpretation and supplementary of the content, create a link between concepts, and share with other learners to form knowledge exchange, which is quite helpful for improving reading comprehension.This study explores the functions of two annotation systems developed in ancient book database. 30 participants recruited for the study were those with humanistic background. This study analyzes the effects of reading ancient books on automatic and manual annotation system for reading effectiveness, technology acceptance and reading experience. The results of the study show that: (1) There is no significant difference in the effectiveness of reading comprehension using different annotation systems, but the usefulness of manual annotation system is significantly higher than automatic system; (2) the subjects have higher satisfaction with the "add new words" and "query vocabulary" functions of automatic annotation system, as well as the "query vocabulary" and "new labeling" functions of manual annotation system; (3) The subjects believe that the problem of automatic annotation is that automatic labelling vocabularies are inaccurate, and the manual annotation is not easy to label. [ABSTRACT FROM AUTHOR]

To avoid the low efficiency in massive place names searching in the traditional relational database, this paper proposed a fast place name database retrieval method with the integration of PanGuAnalyzer and Lucene full-text search toolbox. Firstly, it designed a place name data structure, and compared the segmentation performances of several open source Chinese analyzers. Based on the results, it integrated the excellent PanguAnalyzer with a rich place dictionary into Lucene so as to improve the effect of Chinese place name segmentation. To improve the efficiency of creating inverted index, it adopted memory index and multi-thread parallel processing. It also optimized the query result ranking strategy based on similarity scoring according to the category and display priority attributes of place names. Finally, it developed a place name searching system, which integrated various functions including place name searching, visualization, and location service. More than 5 000 000 real place name records were used to test the performance of the new searching technique. By comparing with the searching results of fuzzy query method based on MySQL database, the average response time of the new method was less than one second, and it was nearly fifteen times faster than the database retrieval. The new proposed full-text search strategy demon-strates its advantage in terms of accuracy and rapid response, and it can provide efficient and flexible public place name search service. [ABSTRACT FROM AUTHOR]

Mục tiêu của bài viết là đề xuất mô hình quản lý và khai thác hữu hiệu các dữ liệu phong phú, đa dạng đang tồn tại dưới dạng các văn bản, bảng tính của một tổ chức. Bên cạnh đó, chúng tôi cũng đề xuất giải pháp công nghệ cụ thể dựa trên các nền tảng Big Data phổ biến, bao gồm: (1) HDFS (Hadoop Distributed File System) của Hadoop dùng trong quản lý tập tin, (2) Lucene để lập chỉ mục nghịch đảo (Inverted Index) cho văn bản và (3) Apache Solr hỗ trợ cơ chế quản lý chỉ mục nghịch đảo, tìm kiếm toàn văn và một số chức năng tìm kiếm nâng cao. Bài viết cũng trình bày kết quả thực nghiệm, tổng hợp kết quả và trình bày biểu đồ thống kê của việc áp dụng mô hình trong phân tích xu hướng nghiên cứu khoa học tại Trường Đại học Cần Thơ.

In recent years, mass digitization has opened up voluminous text corpora to human interpretation. Full-text search lets historians now find new sources that can change their understanding of thoroughly studied historical episodes. At the same time, it forces scholars to access historical sources in a new way: through specific words. This article analyses the consequences of this new way of accessing sources and investigates which search technologies are best suited for historical source selection in digital repositories. It argues that to seize the opportunities that digitization offers, historians must refine their search technologies so that they are based on words but are less dependent on exact phraseology. [ABSTRACT FROM PUBLISHER]

This research presents the results of evaluating multiple free, open-source engines on matching ICD-10 diagnostic codes via full-text searches. The study investigates what it takes to get an accurate match when searching for a specific diagnostic code. For each code the evaluation starts by extracting the words that make up its text and continues with building full-text search queries from the combinations of these words. The queries are then run against all the ICD-10 codes until a match indicates the code in question as a match with the highest relative score. This method identifies the minimum number of words that must be provided in order for the search engines choose the desired entry. The engines analyzed include a popular Java-based full-text search engine, a lightweight engine written in JavaScript which can even execute on the user’s browser, and two popular open-source relational database management systems. [ABSTRACT FROM AUTHOR]