Search

This work presents a pipeline for processing digitally scanned documents, reading their textual content, and storing it in a dataset for the purpose of information retrieval. The pipeline is able to handle images of various quality, whether they were obtained by a digital scanner or camera. The image can contain multiple pages in any layout, but an approximate upright orientation is assumed. The pipeline uses Faster R-CNN to detect individual pages. These are then processed by a deskew algorithm to correct the orientation, and finally read by the Tesseract OCR system that has been retrained on a large set of synthetic images and a small set of annotated real-world documents. By applying the pipeline, we were able to increase the word recall to 60.56% which is an absolute gain of 19.19% from the baseline solution that uses only Tesseract OCR. A demo of the proposed pipeline can be found at https://archivkgb.zcu.cz/.

In this paper, we focus on the issue of speaker diarization of data from a real call center. We have previously proposed a specialized solution to the problem, which employed additional knowledge about the identities of the phone operators (in our case, the language counselors from the Language Consulting Center), thus improving performance over the baseline. But a recent end-to-end diarization method, EEND, has since proven very successful on other data and was shown to surpass the previous state of the art in the field. Thus, we chose to compare this new method with our own previous approach. Using an existing implementation of the EEND method (adapted using a small amount of the target data from the Language Consulting Center), we successfully surpass the performance of our previous approach (17.42% vs. 19.39% DER), without the need for any additional information about speaker identities. The majority of the remaining diarization error of the EEND system is due to incorrect decisions between speech and silence, rather than speaker confusion. For comparison, we also show the results of a more standard diarization approach, represented by the method used in the Kaldi toolkit.

V tomto článku presentujeme proces předzpracování naskenovaných historických dokumentů do elektronické textové formy, která může poté být zindexována a uložena do databáze. Povaha dokumentů představuje velkou výzvu pro standardní automatizované techniky. Nejenže se jedná o kombinaci ručně psaných dokumentů a dokumentů psaných na stroji, ale dramaticky se liší i jejich kvalita a jednotlivé skeny často obsahují více než jen jednu stránku. Navíc, jazyk jednotlivých dokumentů se střídá převážně mezi ruštinou a ukrajinštinou, nicméně se v dokumentech vyskytují i úplně jiné jazyky. Tento článek se zaměřuje na segmentaci, klasifikaci typu dokumentu, a celkové předzpracování dokumentů. Nad výstupem těchto metod je pak otestován standardní OCR software a jeho úspěšnost je vyhodnocena na velmi jednoduché baseline úloze. In this paper, we propose a pipeline for processing of scanned historical documents into the electronic text form that could then be indexed and stored in a database. The nature of the documents presents a substantial challenge for standard automated techniques — not only there is a mix of typewritten and handwritten documents of varying quality but the scanned pages often contain multiple documents at once. Moreover, the language of the texts alternates mostly between Russian and Ukrainian but other languages also occur. The paper focuses mainly on segmentation, document type classification, and image preprocessing of the scanned documents; the output of those methods is then passed to the off-the-shelf OCR software and a baseline performance is evaluated on a simplified OCR task.

V tomto článku popisujeme diarizaci mono telefonních dat z Jazykové poradny Ústavu pro jazyk český. Náš navrhovaný přístup k diarizaci využívá informace o identitě jednoho z účastníků hovoru. V klasickém přístupu k diarizaci nahrazujeme shlukování x-vektorů identifikací řečníka. In this paper, we describe a diarization of mono channel telephone recordings from The Language Consulting Center providing the Czech language consultancy service. In our proposed approach to a diarization, we use information about the known identity of one speaker (the language counsellor) acquired from the text transcription at the beginning of the conversation. In the state-of-the-art diarization based on the x-vectors clustering, we replace the clustering step by the identification of each segment of the recording against the counsellor’s identity x-vector and the general x-vector model that represents the client. Our proposed diarization without resegmentation step can be used as an online approach. Because of the uniqueness of our data, we compare our results with the Kaldi diarization as the baseline system.

In this paper, we present our system developed by the team from the New Technologies for the Information Society (NTIS) research center of the University of West Bohemia in Pilsen, for the Second DIHARD Speech Diarization Challenge. The base of our system follows the currently-standard approach of segmentation, i/x-vector extraction, clustering, and resegmentation. The hyperparameters for each of the subsystems were selected according to the domain classifier trained on the development set of DIHARD II. We compared our system with results from the Kaldi diarization (with i/x-vectors) and combined these systems. At the time of writing of this abstract, our best submission achieved a DER of 23.47% and a JER of 48.99% on the evaluation set (in Track 1 using reference SAD)., Comment: Submitted to Interspeech 2019

Přítomnost překrývající se řeči má značný negativní vliv na úspěšnost systémů pro diarizaci řečníků. Tento článek se věnuje aplikaci konvoluční neuronové síť pro detekci takovýchto řečových intervalů a následnému vyhodnocení z hlediska potenciálního zlepšení diarizace řečníků. Síť je trénována na speciálně vytvořených umělých datech, zatímco vyhodnocení je prováděno na datasetech AMI Corpus a SSPNet Conflict Corpus. The presence of overlapping speech has a significant negative impact on the performance of speaker diarization systems. In this paper, we employ a convolutional neural network for the detection of such speech intervals and evaluate it in terms of the potential improvements to speaker diarization. We train the network on specifically-created synthetic data, while the evaluation is performed on the AMI Corpus and the SSPNet Conflict Corpus.

V tomto článku popisujeme diarizaci archivu Jazykové poradny vznikajícím v rámci projektu "Zpřístupnění dotazů jazykové poradny v lingvisticky strukturované databázi". Jedna část tohoto archivu je nahraná pouze v mono kvalit, naším úkolem je proto rozdělit data pomocí diarizace. Náš přístup využívá informace o identitě jazykového poradce získané z přepisu jeho představení na začátku každého z hovorů. Protože naše data jsou jedinenčná, pro porovnání uvádíme také výsledky dostupného systému diarizace Kaldi. In this paper, we describe a diarization of the archive data from the project “Access to a Linguistically Structured Database of Enquiries from the Language Consulting Center”. This project is attempting to provide improved access to the large archives of the Czech language of mainly telephone conversations collected continuously by The Language Consulting Center. One part of this archives contains mono recordings, where the data of the client and the language counsellor are mixed in one channel. In our proposed approach to a diarization, we used the information about the identity of the language counsellor acquired from the text transcription on the beginning of the conversation. For the initial stage of the diarization, our system based on clustering the x-vectors was adopted. The resegmentation step is used for refining the boundaries of speaker changes by the pre-trained Gaussian mixture model of the counsellor. Because of the uniqueness of our data, we compared our results with the Kaldi diarization as the baseline system.

In this paper, we describe the initial stages of the project “Access to a Linguistically Structured Database of Enquiries from the Language Consulting Center”. This project is attempting to provide an improved access to the large archives of mainly telephone conversations collected continuously by the Institute of the Czech Language. The main goal is to open up the unique Czech data acquired from the queries to the Language Consulting Center and to build the semi-automatic system that will facilitate searching and categorizing of these queries. For this purpose, the Automatic Speech Recognizer (ASR) and the language processing methods are being designed. The vocabulary used in such queries contains many unusual words unlike the common speech (e.g. linguistic terms). In order to train the ASR system, it is necessary to manually transcribe a large amount of speech data, identify the appropriate vocabulary, and obtain relevant text for language modeling purposes. In this paper, the proposed telephone system for recording the new data and the baseline speech recognition on these data is described. The first experiments with the topic detection on these data aimed at discovering what can be found in them and also how to preprocess them is also described.

In this paper, we propose a speaker change detection system based on lexical information from the transcribed speech. For this purpose, we applied a recurrent neural network to decide if there is an end of an utterance at the end of a spoken word. Our motivation is to use the transcription of the conversation as an additional feature for a speaker diarization system to refine the segmentation step to achieve better accuracy of the whole diarization system. We compare the proposed speaker change detection system based on transcription (text) with our previous system based on information from spectrogram (audio) and combine these two modalities to improve the results of diarization. We cut the conversation into segments according to the detected changes and represent them by an i-vector. We conducted experiments on the English part of the CallHome corpus. The results indicate improvement in speaker change detection (by 0.5% relatively) and also in speaker diarization (by 1% relatively) when both modalities are used.

In offline speaker diarization systems, particularly those aimed at telephone speech, the accuracy of the initial segmentation of a conversation is often a secondary concern. Imprecise segment boundaries are typically corrected during resegmentation, which is performed as the final step of the diarization process. However, such resegmentation is generally not possible in online systems, where past decisions are usually unchangeable. In such situations, correct segmentation becomes critical. In this paper, we evaluate several different segmentation approaches in the context of online diarization by comparing the overall performance of an i-vector-based diarization system set to operate in a sequential manner.

The goal of this paper is to evaluate the contribution of speaker change detection (SCD) to the performance of a speaker diarization system in the telephone domain. We compare the overall performance of an i-vector based system using both SCD-based segmentation and a naive constant length segmentation with overlapping segments. The diarization system performs K-means clustering of i-vectors which represent the individual segments, followed by a resegmentation step. Experiments were done on the English part of the CallHome corpus. The final results indicate that the use of speaker change detection is beneficial, but the differences between the two segmentation approaches are diminished by the use of resegmentation.

Multi-label classification plays the key role in modern categorization systems. Its goal is to find a set of labels belonging to each data item. In the multi-label document classification unlike in the multi-class classification, where only the best topic is chosen, the classifier must decide if a document does or does not belong to each topic from the predefined topic set. We are using the generative classifier to tackle this task, but the problem with this approach is that the threshold for the positive classification must be set. This threshold can vary for each document depending on the content of the document (words used, length of the document, ...). In this paper we use the Unconstrained Cohort Normalization, primary proposed for speaker identification/verification task, for robustly finding the threshold defining the boundary between the correc and the incorrect topics of a document. In our former experiments we have proposed a method for finding this threshold inspired by another normalization technique called World Model score normalization. Comparison of these normalization methods has shown that better results can be achieved from the Unconstrained Cohort Normalization.

The aim of this work is to propose a refinement of the shift-MLLR (shift Maximum Likelihood Linear Regression) adaptation of an acoustics model in the case of limited amount of adaptation data, which can lead to ill-conditioned transformations matrices. We try to suppress the influence of badly estimated transformation parameters utilizing the Artificial Neural Network (ANN), especially Convolutional Neural Network (CNN) with bottleneck layer on the end. The badly estimated shift-MLLR transformation is propagated through an ANN (suitably trained beforehand), and the output of the net is used as the new refined transformation. To train the ANN the well and the badly conditioned shift-MLLR transformations are used as outputs and inputs of ANN, respectively.

Adaptation techniques are necessary in automatic speech recognizers to improve a recognition accuracy. Linear Transformation methods (MLLR or fMLLR) are the most favorite in the case of limited available data. The fMLLR is the feature-space transformation. This is the advantage with contrast to MLLR that transforms the entire acoustic model. The classical fMLLR estimation involves maximization of the likelihood criterion based on individual Gaussian components statistic. We proposed an approach which takes into account the overall likelihood of a HMM state. It estimates the transformation to optimize the ML criterion of HMM directly using gradient descent algorithm.

In the paper recent methods used in the task of speaker recognition are presented. At first, the extraction of so called i-vectors from GMM based supervectors is discussed. These i-vectors are of low dimension and lie in a subspace denoted as Total Variability Space (TVS). The focus of the paper is put on Probabilistic Linear Discriminant Analysis (PLDA), which is used as a generative model in the TVS. The influence of development data is analyzed utilizing distinct speech corpora. It is shown that it is preferable to cluster available speech corpora to classes, train one PLDA model for each class and fuse the results at the end. Experiments are presented on NIST Speaker Recognition Evaluation (SRE) 2008 and NIST SRE 2010.

The worst problem the adaptation is dealing with is the lack of adaptation data. This work focuses on the feature Maximum Likelihood Linear Regression (fMLLR) adaptation where the number of free parameters to be estimated significantly decreases in comparison with other adaptation methods. However, the number of free parameters of fMLLR transform is still too high to be estimated properly when dealing with extremely small data sets. We described and compared various methods used to avoid this problem, namely the initialization of the fMLLR transform and a linear combination of basis matrices varying in the choice of the basis estimation (eigen decomposition, factor analysis, independent component analysis and maximum likelihood estimation). Initialization methods compensate the absence of the test speaker’s data utilizing other suitable data. Methods using linear combination of basis matrices reduce the number of estimated fMLLR parameters to a smaller number of weights to be estimated. Experiments are aimed to compare results of proposed basis and initialization methods.

One of the most utilized adaptation techniques is the feature Maximum Likelihood Linear Regression (fMLLR). In comparison with other adaptation methods the number of free parameters to be estimated significantly decreases. Thus, the method is well suited for situations with small amount of adaptation data. However, fMLLR still fails in situations with extremely small data sets. Such situations can be solved through proper initialization of fMLLR estimation adding some a-priori information. In this paper a novel approach is proposed solving the problem of fMLLR initialization involving statistics from speakers acoustically close to the speaker to be adapted. Proposed initialization suitably substitutes missing adaptation data with similar data from a training database, fMLLR estimation becomes well-conditioned, and the accuracy of the recognition system increases even in situations with extremely small data sets.

This paper deals with robust estimations of data statistics used for the adaptation. The statistics are accumulated before the adaptation process from available adaptation data. In general, only small amount of adaptation data is assumed. These data are often corrupted by noise, channel, they do not contain only clean speech. Also, when training Hidden Markov Models (HMM) several assumptions are made that could not have been fulfilled in the praxis, etc. Therefore, we described several techniques that aim to make the adaptation as robust as possible in order to increase the accuracy of the adapted system. One of the methods consists in initialization of the adaptation statistics in order to prevent ill-conditioned transformation matrices. Another problem arises when an acoustic feature is assigned to an improper HMM state even if the reference transcription is available. Such situations can occur because of the forced alignment process used to align frames to states. Thus, it is quite handy to accumulate data statistic utilizing only reliable frames (in the sense of data likelihood). We are focusing on Maximum Likelihood Linear Transformations and the experiments were performed utilizing the feature Maximum Likelihood Linear Regression (fMLLR). Experiments are aimed to describe the behavior of the system extended by proposed methods.

This paper deals with a combination of basic adaptation techniques of Hidden Markov Model used in the speech recognition. The adaptation methods approach the data only through their statistics, which have to be accumulated before the adaptation process. When performing two adaptations subsequently, the data statistics have to be accumulated twice in each of the adaptation passes. However, when the adaptation methods are chosen with care, the data statistics may be accumulated only once, as proposed in this paper. This significantly reduces the time consumption and avoids the need to store all the adaptation data. Combination of Maximum A-Posteriori Probability and feature Maximum Likelihood Linear Regression adaptation is considered. Motivation for such an approach could be the on-line adaptation, where the time consumption is of big importance.

The article introduces an expert system for the speaker verification task. Our main purpose was to design a tool for the combination of various speaker verification systems proposed for various operating conditions. First of all, the essential ideas are explained that made us design the expert system. Next section describes the structure of a rule-based expert system and subsequently an oriented graph is proposed for the representation of the topology of the system. The expert rules exploited by the system are derived automatically from the input data and we have implemented also a certainty factor to acquire more reliable decisions. The experiments show that the proposed system has the capability to significantly improve the verification results in trials with various operating conditions.

In the paper Factor Analysis (FA) and Nuisance Attribute Projection (NAP) are reviewed, analyzed and compared. Since nowadays FA become a part of most state-of-the-art recognition systems (used e.g. in the concept of i-vectors or PLDA models) it is of relevance to examine different insights into the problem. NAP was chosen as a counterpart to FA as an advanced PCA like method often utilized in speaker recognition systems along with FA. It is demonstrated how can be both FA and NAP expressed as solutions of Least Squares (LS), the consequences of the LS formulation are discussed, and it is shown in what extent do solutions of NAP and FA overlap.