Search

PurposeThis study aims to identify user perception of different qualities of optical character recognition (OCR) in texts. The purpose of this paper is to study the effect of different quality OCR on users' subjective perception through an interactive information retrieval task with a collection of one digitized historical Finnish newspaper.Design/methodology/approachThis study is based on the simulated work task model used in interactive information retrieval. Thirty-two users made searches to an article collection of Finnish newspaper Uusi Suometar 1869–1918 which consists of ca. 1.45 million autosegmented articles. The article search database had two versions of each article with different quality OCR. Each user performed six pre-formulated and six self-formulated short queries and evaluated subjectively the top 10 results using a graded relevance scale of 0–3. Users were not informed about the OCR quality differences of the otherwise identical articles.FindingsThe main result of the study is that improved OCR quality affects subjective user perception of historical newspaper articles positively: higher relevance scores are given to better-quality texts.Originality/valueTo the best of the authors’ knowledge, this simulated interactive work task experiment is the first one showing empirically that users' subjective relevance assessments are affected by a change in the quality of an optically read text.

In information science, the user-oriented research tradition focuses on information seeking in the context of human behavior, while the system-oriented tradition focuses more narrowly on the information retrieval. These traditions still form largely disparate research fields. As an attempt to bridge these traditions, we present a metasynthesis of studies on information interactions in the historical domain, which is related to an ongoing research project. We approach our research task by focusing on both the cognitive space of human actors with task-specific information needs requiring interpretative close-reading and reasoning, and the document space containing potentially relevant pieces of information for the task at hand. We first study information needs and interactions in real work tasks of historians to understand the desired conceptual access points into relevant information (cognitive space). Then we study historical sources from the point of view of task-specific needs at the level of text (document space). We utilize the task-based information interaction (TBII) model to conceptualize the different types of activities during interaction which are important to consider. We believe that this approach is required to learn to support complex task-based information needs extending beyond topicality. publishedVersion

Many patients with muscle-invasive bladder cancer (BC) are either ineligible for or do not benefit from cisplatin-based chemotherapy, and there is an unmet need to estimate individuals' drug sensitivities. We investigated the suitability of conditionally reprogrammed (CR) cells for the characterization of BC properties and their feasibility for personalized drug sensitivity screening. The CR cultures were established from six BC tumors with varying histology and stage. Four cultures were successfully propagated for genomic, transcriptomic, and protein expression profiling and compared to the parental tumors. Two out of four CR cultures (urothelial carcinoma and small cell neuroendocrine carcinoma [SmCC]) corresponded well to their parental tumors and underwent drug sensitivity screening to identify novel drugs for the respective tumors. Both cultures were sensitive to standard BC chemotherapy agents (eg cisplatin and gemcitabine) and to conventional drugs such as taxanes and inhibitors of topoisomerase and proteasome. The SmCC cells were also sensitive to statins (eg, atorvastatin and pitavastatin). In summary, after confirming their representativeness and origin, we conclude that CR cells are a feasible platform for personalized drug sensitivity testing and might thus add to the approaches used to personalize BC treatment strategies. PATIENT SUMMARY: We investigated the conditional reprogramming method for generating patient-derived bladder cancer cell cultures and studied their feasibility for planning personalized treatment strategies.

This study continues a work in progress for implementing a full-text lexical semantic tagger for Finnish, FiST. The tagger is based on a 46,226 lexeme semantic lexicon of Finnish that was published in 2016 [1]. Kettunen [2], [3] describes the basic working version of FiST. FiST is based on freely available components: the first implementation uses Omorfi and FinnPos for morphological analysis and disambiguation of Finnish words. The current paper describes work with compound splitting for semantic tagging and its effects on the lexical coverage of the tagger. We try out two different approaches to morphological analysis and disambiguation of words for an improved version of FiST, FiSTComp: FinnPos [4], and Turku Dependency Parser [5], [6], UD1. Both these tools disambiguate morphological interpretations of words and provide boundary markings for compounds, but details and granularity of constituent decomposition vary. Our results with two-, three and four-part compounds show that analysis of compounds through their constituents with UD1 may improve the lexical coverage of the tagger with about 6.6 % units at best. Although we are able to proceed in basic problems of compound splitting, the results are still initial and further work is needed as compounds are a complex phenomenon.

The current paper presents work that has been carried out in the National Library of Finland (NLF) to improve optical character recognition (OCR) quality of the historical Finnish newspaper collection 1771–1910. Evaluation results reported in the paper are based mainly on a 500 000 word sample of the Finnish language part of the whole collection. The sample has three different parallel parts: a manually corrected ground truth version, original OCR with ABBYY FineReader v. 7 or v. 8, and an ABBYY FineReader v. 11 re-OCRed version for comparison with Tesseract’s OCR. Using this sample and its page image originals we have developed a re-OCRing procedure using the open source software package Tesseract v. 3.04.01. Our method achieved initially 27.48% improvement vs. ABBYY FineReader 7 or 8 and 9.16% improvement vs. ABBYY FineReader 11 on document level. On word level our method achieved 36.25% improvement vs. ABBYY FineReader 7 or 8 and 20.14% improvement vs. ABBYY FineReader 11. Our final precision and recall results on word level show clear improvement in the quality: recall is 76.0 and precision 92.0 in comparison to GT OCR. Other measures, such as recognizability of words with a morphological analyzer and character accuracy rate, show also steady improvement after re-OCRing.

International audience; This paper describes first large scale article detection and extraction efforts on the Finnish Digi 1 newspaper material of the National Library of Finland (NLF) using data of one newspaper, Uusi Suometar 1869-1898. The historical digital newspaper archive environment of the NLF is based on commercial docWorks 2 software. The software is capable of article detection and extraction, but our material does not seem to behave well in the system in this respect. Therefore, we have been in search of an alternative article segmentation system and have now focused our efforts on the PIVAJ machine learning based platform developed at the LITIS laboratory of University of Rouen Normandy [11-13, 16, 17]. As training and evaluation data for PIVAJ we chose one newspaper, Uusi Suometar. We established a data set that contains 56 issues of the newspaper from years 1869-1898 with 4 pages each, i.e. 224 pages in total. Given the selected set of 56 issues, our first data annotation and experiment phase consisted of annotating a subset of 28 issues (112 pages) and conducting preliminary experiments. After the preliminary annotation and 1 https://digi.kansalliskirjasto.fi/etusivu?set_language=en 2 https://extranet.content-conversion.com experimentation resulting in a consistent practice, we fixed the annotation of the first 28 issues accordingly. Subsequently, we annotated the remaining 28 issues. We then divided the annotated set into training and evaluation sets of 168 and 56 pages. We trained PIVAJ successfully and evaluated the results using the layout evaluation software developed by PRImA research laboratory of University of Salford [6]. The results of our experiments show that PIVAJ achieves success rates of 67.9, 76.1, and 92.2 for the whole data set of 56 pages with three different evaluation scenarios introduced in [6]. On the whole, the results seem reasonable considering the varying layouts of the different issues of Uusi Suometar along the time scale of the data. CCS CONCEPTS • CCS → Applied computing → Document management and text processing → Document capture → Document analysis • CCS → Information systems → Information retrieval → Document representation → Document structure • CCS → Information systems → Information systems applications → Digital libraries and archives KEYWORDS Document layout analysis, article extraction, historical digitized Finnish newspaper archives, PIVAJ software ACM Reference format

This paper describes utilization of article detection and extraction on the Finnish Digi (https://digi.kansalliskirjasto.fi/etusivu?set_language=en) newspaper material of the National Library of Finland (NLF) using data of one newspaper, Uusi Suometar 1869–1918. We use PIVAJ software [1] for detection and marking of articles in our collection. Out of the separated articles we can produce automatic clippings for the user. The user can collect clippings for own use both as images and as OCRed text. Together these functionalities improve usability of the digitized journalistic collection by providing a structured access to the contents of a page.

The aim of the study was to test whether query expansion by approximate string matching methods is beneficial in retrieval from historical newspaper collections in a language rich with compounds and inflectional forms Finnish. First, approximate string matching methods were used to generate lists of index words most similar to contemporary query terms in a digitized newspaper collection from the 1800s. Top index word variants were categorized to estimate the appropriate query expansion ranges in the retrieval test. Second, the effectiveness of approximate string matching methods, automatically generated inflectional forms, and their combinations were measured in a Cranfield-style test. Finally, a detailed topic-level analysis of test results was conducted. In the index of historical newspaper collection the occurrences of a word typically spread to many linguistic and historical variants along with optical character recognition OCR errors. All query expansion methods improved the baseline results. Extensive expansion of around 30 variants for each query word was required to achieve the highest performance improvement. Query expansion based on approximate string matching was superior to using the inflectional forms of the query words, showing that coverage of the different types of variation is more important than precision in handling one type of variation.

Named Entity Recognition (NER), search, classification and tagging of names and name like frequent informational elements in texts, has become a standard information extraction procedure for textual data. NER has been applied to many types of texts and different types of entities: newspapers, fiction, historical records, persons, locations, chemical compounds, protein families, animals etc. In general a NER system's performance is genre and domain dependent and also used entity categories vary [16]. The most general set of named entities is usually some version of three partite categorization of locations, persons and organizations. In this paper we report evaluation result of NER with data out of a digitized Finnish historical newspaper collection Digi. Experiments, results and discussion of this research serve development of the Web collection of historical Finnish newspapers.Digi collection contains 1,960,921 pages of newspaper material from years 1771-1910 both in Finnish and Swedish. We use only material of Finnish documents in our evaluation. The OCRed newspaper collection has lots of OCR errors; its estimated word level correctness is about 70-75% [7]. Our baseline NER tagger is a rule-based tagger of Finnish, FiNER, provided by the FIN-CLARIN consortium. Three other available tools are also evaluated: a Finnish Semantic Tagger (FST), Connexor's NER tool and Polyglot's NER.

Type-token ratio (TTR), also known as vocabulary size divided by text length (V/N) is a simple measure of lexical diversity. It has been used in literary studies, studies in child language and even psychiatry. The basic problem of TTR is that it is affected by the length of the text sample. Several suggestions for improving this fault have been given, including standardizing the length of text samples, using logarithms in the basic formula, etc. We show in this paper that simple TTR and its more elaborate calculation MATTR can be used for approximation of morphological complexity of languages. This usage of TTR has been notified by Juola with analysis of six languages. We analyse text material with TTR and MATTR from two differing sources: firstly, text of the EU constitution with 21 languages and secondly with 16 of the same languages with available non-parallel random data from the Leipzig corpus. We compare the automatic analysis results to two independent linguistic measures of morphological c...

Fifth-generation (5G) systems will introduce several new requirements impacting the mobile transport network. Understanding the impact of 5G systems on trans­port networks, both wireless and fixed, is essential to the successful introduction of 5G. This paper provides an over­view of the ongoing work supporting the specification of open interfaces between the different 5G building blocks. It also presents several technologies that can facilitate ef­fective transport networks, describes the architectural challenges in designing flexible 5G transport networks, and provides dimensioning guidelines.

There has been a huge interest in digitization of both hand-written and printed historical material in the last 10–15 years and most probably this interest will only increase in the ongoing Digital Humanities era. As a result of the interest we have lots of digital historical document collections available and will have more of them in the future.

PurposeThis paper seeks to examine how an electronic records management system has been used in a Finnish government agency. In particular, it aims to study the relationship between functional classification scheme and the way users in different organisational units and at different organisational levels have employed the system. The goal is to examine whether electronic records management systems were easier to use if the system “knew” what functional classes the user (or other employees in the user's organisational unit) typically need in their work.Design/methodology/approachThe study is based on two sources. The first source is metadata in records that were captured in the electronic records management system of the agency. It reflects actual behaviour of users when they interact with the system and classification of records. The second source is distribution of functions to organisational units in the light of policy documents and a survey made in the organisation. The study compares the two sources to see how the users have employed the electronic records management system in their work and how this relates to organisational structure and supposed usage of the system.FindingsIn general, individual employees employ only a small part of the classification. However, this does not apply at a higher level in the organisational hierarchy: the higher the person's position in the hierarchy, the more classes he/she is likely to use in the work. Regardless of the position, the classes are generally those identified as belonging to the employee's unit.Research limitations/implicationsThe study is based on one agency with a functional organisational structure. The findings may not apply to organisations where job descriptions are fluid. They should also be tested in more complex organisational settings. One could develop new methods of automated classification which combine analysis of document content with contextual reasoning about the likely functional classes.Practical implicationsAccess to electronic records management systems could be facilitated by creating in systems user/unit profiles defining what functional classes the user is most likely to need in their work. It would also be useful if systems simply remembered what functional classes the user has needed in the past.Originality/valueThe study offers insight into how an electronic records management system is used in an organisation. This is valuable for companies developing records management software and persons trying to gain a deeper understanding of records management in organisations.

During the last 15 years, records managers and archivists have focused on defining requirements for functionality and metadata of electronic records management systems. So far little is known about metadata that are actually created in these systems. This study examines metadata in records of a Finnish government agency. The metadata comply with the national SAHKE records management metadata specification. The main findings are that a substantial number of metadata elements in SAHKE remained unused. Generated metadata were largely about event history. Optional metadata elements were used rarely. The metadata show clear patterns, which may suggest systematic recordkeeping processes and possibly minimal human intervention in metadata creation.

Many operational IR indexes are non-normalized, i.e. no lemmatization or stemming techniques, etc. have been employed in indexing. This poses a challenge for dictionary-based cross-language retrieval (CLIR), because translations are mostly lemmas. In this study, we face the challenge of dictionary-based CLIR in a non-normalized index. We test two optional approaches: FCG (Frequent Case Generation) and s-gramming. The idea of FCG is to automatically generate the most frequent inflected forms for a given lemma. FCG has been tested in monolingual retrieval and has been shown to be a good method for inflected retrieval, especially for highly inflected languages. S-gramming is an approximate string matching technique (an extension of n-gramming). The language pairs in our tests were English-Finnish, English-Swedish, Swedish-Finnish and Finnish-Swedish. Both our approaches performed quite well, but the results varied depending on the language pair. S-gramming and FCG performed quite equally in all the other language pairs except Finnish-Swedish, where s-gramming outperformed FCG.

PurposeThe purpose of this article is to discuss advantages and disadvantages of various means to manage morphological variation of keywords in monolingual information retrieval.Design/methodology/approachThe authors present a compilation of query results from 11 mostly European languages and a new general classification of the language dependent techniques for management of morphological variation. Variants of the different techniques are compared in some detail in terms of retrieval effectiveness and other criteria. The paper consists mainly of an overview of different management methods for keyword variation in information retrieval. Typical IR retrieval results of 11 languages and a new classification for keyword management methods are also presented.FindingsThe main results of the paper are an overall comparison of reductive and generative keyword management methods in terms of retrieval effectiveness and other broader criteria.Originality/valueThe paper is of value to anyone who wants to get an overall picture of keyword management techniques used in IR.

In this article, we are studying the differences between the European Union languages using statistical and unsupervised methods. The analysis is conducted in the different levels of language: the lexical, morphological and syntactic. Our premise is that the difficulty of the translation could be perceived as differences or similarities in different levels of language. The results are compared to linguistic groupings. Two approaches are selected for the analysis. A Kolmogorov complexity-based approach is used to compare the language structure in syntactic and morphological levels. A morpheme-level comparison is conducted based on an automated segmentation of the languages into morpheme-like units. The way the languages convey information in these levels is taken as a measure of similarity or dissimilarity between languages and the results are compared to classical linguistic classifications. The results have a significant impact on the design of (statistical) machine translation systems. If the s...

Word form normalization through lemmatization or stemming is a standard procedure in information retrieval because morphological variation needs to be accounted for and several languages are morphologically non-trivial. Lemmatization is effective but often requires expensive resources. Stemming is also effective in most contexts, generally almost as good as lemmatization and typically much less expensive; besides it also has a query expansion effect. However, in both approaches the idea is to turn many inflectional word forms to a single lemma or stem both in the database index and in queries. This means extra effort in creating database indexes. In this paper we take an opposite approach: we leave the database index un-normalized and enrich the queries to cover for surface form variation of keywords. A potential penalty of the approach would be long queries and slow processing. However, we show that it only matters to cover a negligible number of possible surface forms even in morphologically complex languages to arrive at a performance that is almost as good as that delivered by stemming or lemmatization. Moreover, we show that, at least for typical test collections, it only matters to cover nouns and adjectives in queries. Furthermore, we show that our findings are particularly good for short queries that resemble normal searches of web users. Our approach is called FCG (for Frequent Case (form) Generation). It can be relatively easily implemented for Latin/Greek/Cyrillic alphabet languages by examining their (typically very skewed) nominal form statistics in a small text sample and by creating surface form generators for the 3---9 most frequent forms. We demonstrate the potential of our FCG approach for several languages of varying morphological complexity: Swedish, German, Russian, and Finnish in well-known test collections. Applications include in particular Web IR in languages poor in morphological resources.

The paper presents a new method for handling of morphological variation of query terms in best-match IR. The method is based on enhanced inflectional stems. Use of inflectional stems has earlier been shown to be a good retrieval method in inflected indexes in a best-match environment for a highly inflected and compound-rich language, Finnish. In this paper the earlier stem method is elaborated upon by enhancing the stems with regular expressions. Contrary to our expectations the results show that the enhanced stem queries do not outperform basic inflectional stems, but neither are they considerably worse with long queries. With short web-like queries they perform relatively better than with long queries and outperform clearly stemming (Finnish stemmer of Snowball) and plain, unprocessed query words. The main benefits of the proposed method, besides fairly good precision and recall (P-R) performance, are shorter and more manageable queries, which is of practical importance, e.g. with large web indexes.

In this paper we propose a soft-output RAKE receiver for CDMA mobile terminals using low spreading factor and/or high-order modulation constellations for high data rates. Based on the outputs of the RAKE correlator fingers, the receiver produces symbol or bit metrics for soft decision decoding. The formulation of these metrics is derived analytically taking into account the multipath propagation, interfering users and channel estimation errors. Simulations show that a performance gain is achieved when low spreading factors or high-order modulation constellations are used.

This paper presents results of a generative method for the management of morphological variation of query keywords in Bengali, Gujarati and Marathi. The method is called Frequent Case Generation (FCG). It is based on the skewed distributions of word forms in natural languages and is suitable for languages that have either fair amount of morphological variation or are morphologically very rich. We participated in the ad hoc task at FIRE 2011 and applied the FCG method on monolingual Bengali, Gujarati and Marathi test collections. Our evaluation was carried out with title and description fields of test topics, and the Lemur search engine. We used plain unprocessed word index as the baseline, and n-gramming and stemming as competing methods. The evaluation results show 30%, 16% and 70% relative mean average precision improvements for Bengali, Gujarati and Marathi respectively when comparing the FCG method to plain words. The method shows competitive performance in comparison to n-gramming and stemming.

This article focuses on typographical spellchecking. Typographical spellchecking verifies the use of characters such as ? ! - ; : \sp $ @ and other special purpose characters in respect to spaces or null elements. The author claims that this kind of spellchecking has not been developed to any substantial degree, although it could be of considerable practical use as writer's aid. The article discusses the basic challenges of typographical spellchecking and shows that some of the difficulties are greater than might be expected at first sight. Rules for describing the behavior of typographical characters are proposed.

This paper discusses information retrieval of Finnish and keyword variation management by generating inflected variant keyword forms. Finnish is a highly inflectional language, and thus keyword variation management of queries and query indexes is of utter importance for successful Finnish full-text retrieval. In the paper we show that generation of a quite small number of variant keyword forms leads to good retrieval performance using a probabilistic best-match retrieval system (Lemur). Generation of almost the full paradigm of inflected nominal forms improves the results slightly. We have also interesting results with regards to different index types: our evaluation shows that generated inflected queries behave extremely well in a lemmatized index, which is supposedly not suitable for this query type. We also show that in a research environment even inexact generation that produces lots of incorrect inflected forms achieves high precision-recall performance without considerable loss in query throughput effectiveness. We use two different word form generators and their variants and compare the results to commonly used reductive word form variation management methods, stemming and lemmatization. The paper includes also a short discussion about usage of the variant keyword method with Web search engines.

This study describes first usage of a particular implementation of Normalized Compression Distance (NCD) as a machine translation quality evaluation tool. NCD has been introduced and tested for clustering and classification of different types of data and found a reliable and general tool. As far as we know NCD in its Complearn implementation has not been evaluated as a MT quality tool yet, and we wish to show that it can also be used for this purpose. We show that NCD scores given for MT outputs in different languages correlate highly with scores of a state-of-the-art MT evaluation metrics, METEOR 0.6. Our experiments are based on translations between one source and three target languages with a smallish sample that has available reference translations, UN's Universal Declaration of Human Rights. Secondly we shall also briefly describe and discuss results of a larger scale evaluation of NCD as an MT metric with WMT08 Shared Task Evaluation Data. These evaluations confirm further that NCD is a noteworthy MT metric both in itself and also enriched with basic language tools, stemming and Wordnet.

This paper describes usage of MT metrics in choosing the best candidates for MT-based query translation resources. Our main metrics is METEOR, but we also use NIST and BLEU. Language pair of our evaluation is English *** German, because MT metrics still do not offer very many language pairs for comparison. We evaluated translations of CLEF 2003 topics of four different MT programs with MT metrics and compare the metrics evaluation results to results of CLIR runs. Our results show, that for long topics the correlations between achieved MAPs and MT metrics is high (0.85-0.94), and for short topics lower but still clear (0.63-0.72). Overall it seems that MT metrics can easily distinguish the worst MT programs from the best ones, but smaller differences are not so clearly shown. Some of the intrinsic properties of MT metrics do not also suit for CLIR resource evaluation purposes, because some properties of translation metrics, especially evaluation of word order, are not significant in CLIR.

This paper presents implementations of generative management method for morphological variation of query keywords. The method is called FCG, Frequent Case Generation. It is based on the skewed distributions of word forms in natural languages and is suitable for languages that either have fair amount of morphological variation or are morphologically very rich. The paper reports implementation and evaluation of automatic procedures of variant query keyword form generation with short and long queries of CLEF collections for English, Finnish, German and Swedish. The evaluated languages show varying degrees of morphological complexity.

This paper presents a new management method for morphological variation of keywords. The method is called FCG, Frequent Case Generation. It is based on the skewed distributions of word forms in natural languages and is suitable for languages that have either fair amount of morphological variation or are morphologically very rich. The proposed method has been evaluated so far with four languages, Finnish, Swedish, German and Russian, which show varying degrees of morphological complexity.

In this article, we are studying the differences between the European languages using statistical and unsupervised methods. The analysis is conducted in different levels of language, lexical, morphological and syntactic. Our premise is that the difficulty of the translation could be perceived as differences or similarities in different levels of language. The results are compared to linguistic groupings. The analyses of this paper are based on the concept of Kolmogorov complexity, which is used to compare the language structure in syntactic and morphological levels. The way the languages convey information in these levels is taken as a measure of similarity or dissimilarity between languages and the results are compared to classical linguistic classification. The results will serve as a tool in developing machine translation system(s), e.g., in the following way: if source language conveys more information in the morphological level and the target language more in the syntactic level, it is clear that the (machine) translator must be able to transfer the information from one level to another.

We study the impact of variance estimation on a receiver, where the soft information provided by the channel decoder is utilized in the multiuser detection. Specifically, we consider a receiver structure that resembles the structure of an iterative decoder for a serially concatenated code, but where the inner constituent decoder is replaced by an interference cancellation (IC) unit. The system performance with three different variance estimator methods is studied through numerical simulations. The results show that the variance estimation method has a great impact on the receiver performance especially in the case, where the different users are received with different powers.

In this paper we propose a novel receiver structure that utilizes the soft information provided by the channel decoder in the multiuser detection. This receiver structure is based on a single-user version of the uncoded maximum a priori criterion. We derive the optimal multiuser likelihood calculation algorithm for this structure. In addition, suboptimal algorithms are proposed that are based on interference cancellation. Numerical simulations are reported which indicate that these suboptimal algorithms have a close-to-optimal coded BER performance even in the presence of high-power interferers.

The author proposes a modified maximal ratio combining (MRC) scheme that is more suitable for RAKE receivers in wideband code division multiple access (WCDMA) mobile terminals than the conventional MRC scheme. This modified MRC scheme is derived analytically assuming unequal noise powers and imperfect channel estimates in different RAKE fingers.