Search

In current dialogue systems the use of speech as an input modality is common. But this modality is only one of those human beings use. In human-human interaction people use gestures to point or facial expressions to show their moods as well. To give modern systems a chance to read information from all modalities used by humans, these systems must have multimodal user interfaces. The SmartKom system has such a multimodal interface that analyzes facial expression, speech and gesture simultaneously. Here we present the module that fulfills the task of facial expression analysis in order to identify the internal state of a user.

The domain of the speech recognition and dialog system EVAR is train time table inquiry. We observed that in real human-human dialogs when the officer transmits the information, the customer very often interrupts. Many of these interruptions are just repetitions of the time of day given by the officer. The functional role of these interruptions is often determined by prosodic cues only. An important result of experiments where naive persons used the EVAR system is that it is hard to follow the train connection given via speech synthesis. In this case it is even more important than in human-human dialogs that the user has the opportunity to interact during the answer phase. Therefore we extended the dialog module to allow the user to repeat the time of day and we added a prosody module guiding the continuation of the dialog by analyzing the intonation contour of this utterance.

In this paper first experiments with naive persons using the speech understanding and dialog system EVAR are discussed. The domain of EVAR is train table inquiry. We observed that in real human-human dialogs when the officer transmits the information the customer very often interrupts. Many of these interruptions are just repetitions of the time of day given by the officer. The functional role of these interruptions is determined by prosodic cues only. An important result of the experiments with EVAR is that it is hard to follow the system giving the train connection via speech synthesis. In this case it is even more important than in human-human dialogs that the user has the opportunity to interact during the answer phase. Therefore we extended the dialog module to allow the user to repeat the time of day and we added a prosody module guiding the continuation of the dialog.

A set of features for word-level confidence estimation is developed. The features should be easy to implement and should require no additional knowledge beyond the information which is available from the speech recognizer and the training data. We compare a number of features based on a common scoring method, the normalized cross entropy. We also study different ways to combine the features. An artificial neural network leads to the best performance, and a recognition rate of 76% is achieved. The approach is extended not only to detect recognition errors but also to distinguish between insertion and substitution errors.

Laryngealizations are irregular voiced portions of speech, which can have morpho-syntactic functions and can disturb the automatic computation of FO. Two methods for the automatic detection of laryngealizations are described in this paper: With a Gaussian classifier using spectral and cepstral features a recognition rate of 80% (false alarm rate of 8%) could be achieved. As an alternative a “non-standard” method has been developed: an artificial neural network (ANN) was used for non-linear inverse filtering of speech signals. The inversely filtered signal was directly used as input for another ANN, which was trained to detect laryngealizations. In preliminary experiments we obtained a recognition rate of 65% (12% false alarms).

Using sentence templates and a stochastic context-free grammar a large corpus (10,000 sentences) has been created, where prosodic phrase boundaries are labeled in the sentences automatically during sentence generation. With perception experiments on a subset of 500 utterances we verified that 92% of the automatically marked boundaries were perceived as prosodically marked. In initial automatic classification experiments for three levels of boundaries recognition rates up to 81% could be achieved.

Although the recognition of spontaneous speech is the ultimate aim of speech understanding systems it has rarely been investigated so far. In this article first analyses of a German database containing identical utterances of spontaneous and read speech are presented. We describe the differences in prosody between these two registers and report results of a classifier that was trained using prosodic features to discriminate spontaneous and read speech. A systematic difference could be observed that is however rather complex and partly speaker dependent

In this paper, we show how prosodic information can be used in automatic dialogue systems and give some examples of promising new approaches. Most of these examples are taken from our own work in the V erbmobil speech-to-speech translation system and in the EVAR train timetable dialogue system. In a `prosodic orbit', we first present units, phenomena, annotations and statistical methods from the signal (acoustics) to the dialogue understanding phase. We show then, how prosody can be used together with other knowledge sources for the task of resegmentation if a first segmentation turns out to be wrong, and how an integrated approach leads to better results than a sequential use of the different knowledge sources; then we present a hybrid approach which is used to perform a shallow parsing and which uses prosody to guide the parsing; finally, we show how a critical system evaluation can help to improve the overall performance of automatic dialogue systems.

We present the prosody module of the VERBMOBlL speech-to-speech translation system, the world wide first complete system, which successfully uses prosodic information in the linguistic analysis. This is achieved by computing probabilities for clause boundaries, accentuation, and different types of sentence mood for each of the word hypotheses computed by the word recognizer. These probabilities guide the search of the linguistic analysis. Disambiguation is already achieved during the analysis and not by a prosodic verification of different linguistic hypotheses. So far, the most useful prosodic information is provided by clause boundaries. These are detected with a recognition rate of 94%. For the parsing of word hypotheses graphs, the use of clause boundary probabilities yields a speed-up of 92% and a 96% reduction of alternative readings.

In automatic speech understanding, division of continuous running speech into syntactic chunks is a great problem. Syntactic boundaries are often marked by prosodic means. For the training of statistical models for prosodic boundaries large databases are necessary. For the German Verbmobil (VM) project (automatic speech-to-speech translation), we developed a syntactic‐prosodic labelling scheme where diAerent types of syntactic boundaries are labelled for a large spontaneous speech corpus. This labelling scheme is presented and compared with other labelling schemes for perceptual‐prosodic, syntactic, and dialogue act boundaries. Interlabeller consistencies and estimation of eAort needed are discussed. We compare the results of classifiers (multi-layer perceptrons (MLPs) and n-gram language models) trained on these syntactic‐prosodic boundary labels with classifiers trained on perceptual‐prosodic and pure syntactic labels. The main advantage of the rough syntactic‐prosodic labels presented in this paper is that large amounts of data can be labelled with relatively little eAort. The classifiers trained with these labels turned out to be superior with respect to purely prosodic or syntactic labelling schemes, yielding recognition rates of up to 96% for the two-class-problem ‘boundary versus no boundary’. The use of boundary information leads to a marked improvement in the syntactic processing of the VM system. ” 1998 Elsevier Science B.V. All rights reserved.

In traditional classification problems, the reference needed for training a classifier is given and considered to be absolutely correct. However, this does not apply to all tasks. In emotion recognition in non-acted speech, for instance, one often does not know which emotion was really intended by the speaker. Hence, the data is annotated by a group of human labelers who do not agree on one common class in most cases. Often, similar classes are confused systematically. We propose a new entropy-based method to evaluate classification results taking into account these systematic confusions. We can show that a classifier which achieves a recognition rate of "only" about 60 % on a four-class-problem performs as well as our five human labelers on average.

We show how prosody can be used in speech understanding systems. This is demonstrated with the VERBMOBIL speech to-speech translation system which, to our knowledge, is the first complete system which successfully uses prosodic information in the linguistic analysis. Prosody is used by computing probabilities for clause boundaries, accentuation, and different types, of sentence mood for each of the word hypotheses computed by the word recognizer. These probabilities guide the search of the linguistic analysis. Disambiguation is already achieved during the analysis and not by a prosodic verification of different linguistic hypotheses. So far, the most useful prosodic information is provided by clause boundaries. These are detected with a recognition rate of 94%. For the parsing of word hypotheses graphs, the use of clause boundary probabilities yields a speed-up of 92% and a 96% reduction of alternative readings.

Title ofthe paper: Investigations of population genetic parameters using multilocus DNA fingerprinting in 4 goat breeds in Germany Conventional fmgerprint banding pattems were produced by Southern blotting and digoxigenm labeled oligonucleotid probes (GTG)5 and (GT)g. Digitised images of the blots were analysed with the RFLPscan Computer program. These probes produced more bands (20–39 distinct informative bands within each group) in these goat breeds than in sheep breeds. Typically, in individual goats, an average of 8–11 bands were detected in the range of 4.3–23 kb. Analysis ofthe Polymorphie banding pattems showed that 4.5 to 8.3 genetic loci were represented by these probes. The analysis of pooled DNA samples (10–28 animals per lane) overestimated similarity values (S) between different breeds. Average similarity values within groups (based on RFLPs of individuals) ränge from 0.42–0.61, while similarity values between different groups ränge from 0.36–0.47. The results indicated that native Thüringer Wald Ziege are different from the Toggenburger goats.The Heterozygosity index (H) for each breed calculated from (GTG)5 fingerprints (H = 0.42–0.62) were in good agreement (negative correlated) with band sharing valve. These H values are lower than those obtained in a similar study of 19 sheep breeds in which the ränge was (0.68–0.82). The goat breed with the highest H value was the Bunte Deutsche Edelziege (H = 0,62). Ranking the breeds by heterozygosity index and by inversely band sharing index gives: BDE > German TOGO > WDE, TWZ > Swiss TOGO. The results are discussed with respect to breeding programs and the influence of small population size.

In automatic speech understanding, the division of continuously running speech into syntactic chunks is a great problem. Syntactic boundaries are often marked by prosodic means. For the training of statistic models for prosodic boundaries large databases are necessary. For the German VERBMOBIL project (automatic speech-to-speech translation), we developed a syntactic-prosodic labeling scheme where two main types of boundaries (major syntactic boundaries and syntactically ambiguous boundaries) and some other special boundaries are labeled for a large VERBMOBIL spontaneous speech corpus. We compare the results of classifiers (multilayer perceptrons and language models) trained on these syntactic-prosodic boundary labels with classifiers trained on perceptual-prosodic and pure syntactic labels. The main advantage of the rough syntactic-prosodic labels presented in this paper is that large amounts of data could be labeled within a short time. Therefore, the classifiers trained with these labels turned out to be superior (recognition rates of up to 96%).

A frequent phenomenon in spoken dialogs of the information seeking type are short elliptic utterances whose mood (declarative or interrogative) can only be distinguished by intonation. The main acoustic evidence is conveyed by the fundamental frequency or F/sub 0/-contour. Many algorithms for F/sub 0/ determination have been reported in the literature. A common problem are irregularities of speech known as 'laryngealizations'. This article describes an approach based on neural network techniques for the improved determination of fundamental frequency. First, an improved version of the authors' neural network algorithm for reconstruction of the voice source signal (glottis signal) is presented. Second, the reconstructed voice source signal is used as input to another neural network distinguishing the three classes 'voiceless', 'voiced non-laryngealized', and 'voiced laryngealized'. Third, the results are used to improve an existing F/sub 0/ algorithm. Results of this approach are presented and discussed in the context of the application in a spoken dialog system. >

The present paper is devoted the development and formal representation of a descriptive model for an information technology to automate the morphological analysis of cytologic preparations (a tumor of the lymphatic system). The theoretical basis of the model is a descriptive approach to image analysis and understanding and its main mathematical tools. Practical application of the algebraic tools of the descriptive approach is demonstrated, and the algorithmic scheme of the technology is described in the language of descriptive image algebras.

Fatty acids are a group of lipids that consist of a carboxyl group plus a long hydrocarbon chain. Saturated fatty acids have single carbon-carbon bonds, whereas unsaturated fatty acids contain one or more double bonds within the hydrocarbon chain. Polyunsaturated fatty acids (PUFAs) have two main functions: the first is a critical role in membrane structure where they confer fluidity and flexibility, and the second is their role as precursors of prostaglandins, leukotrienes, and glycolipids, all of which are essential in the regulation of various biological functions. Mammals lack the enzyme to create double bonds between the ninth carbon (from the carboxyl end) and the terminal methyl group and thus cannot synthesize linoleic acid (18:2n-6) and α-linolenic acid (18:3n-3), making these fatty acids an essential component of the mammalian diet (vitamin F). Plants, including spinach, produce these essential PUFAs by desaturating oleic acid (18:1n-9) at positions C-12 and C-15 to produce 18:2n-6 (linoleic acid) and 18:3n-3 (linolenic acid). Thus, the fatty acid desaturases constitute an important category of enzymes involved in the synthesis of PUFAs. The genes for the desaturases have been identified in various plant systems and in simple organisms including Caenorhabditis elegans (1). These enzymes have also been shown to be functional in mammalian cells on transfection (2). The article by Saeki et al. (3) in a recent issue of PNAS takes this research a significant step further by showing constitutive expression and activity of the Δ12 spinach desaturase gene (FAD2) in transgenic pigs. Levels of linoleic acid (18:2n-6) in adipocytes collected from transgenic pigs and cultured in vitro were ≈10-fold higher than in wild-type control cells. Biopsied white adipose tissue from transgenic pigs contained ≈20% more linoleic acid than that of the nontransgenic controls. This study is the first time that a plant gene has been functionally …

This paper presents a neural network based system for 3-D object recognition and localization. A new appearance-based approach is developed for recognition and pose estimation of 3-D objects from a single 2-D perspective view. Three-layer perceptrons are widely used in the whole image analysis process. A feature vector derived by a nonlinear principal component network is used to model object appearance. A neural classifier which receives the feature vector is then configured for recognition purpose. Object pose parameters are obtained by neural estimators trained on the same feature vector. Performance is tested on a data set consisting of more than 70 000 images of 14 objects. Comparative study with statistical approach is carried out.

In current in vitro production (IVP) systems, oocytes lack in vivo dominant and preovulatory follicular development, which may compromise pregnancy and viability of calves born. When an oocyte sets off in vivo on the road toward fertilization, it contains numerous transcripts and proteins necessary to survive the first few cell cycles of embryonic development. It is not yet known during which period of development the oocyte builds up the store, possibly primarily during the major growth phase of the oocyte, which is completed at the time a follicle reaches the size of 3 mm. Here, we investigated to what extent the later phases of follicular development, such as prematuration in the dominant follicle before the LH surge and ensuing final maturation in the preovulatory follicle, contribute to oocyte competence and development into viable biastocysts. Recent studies on in vivo vs in vitro oocyte maturation employed oocytes from an identical preovulatory development by applying ovum pick-up (OPU) twice (before and 24 h after the LH surge) in each cow treated for superovulation with a controlled LH surge. The embryo recovery rates at Day 7 of IVC after IVF were similar: 44% (97/219) for in vivo- vs 41% (87/213) for in vitro-matured oocytes, which shows that the natural environment during final maturation is not essential for the mere in vitro development of the prematured oocyte beyond the 8- to 16-cell stage. However, in vivo maturation appeared to contribute to the oocyte's quality in a more subtle way, as indicated by a significant increase in the proportion of expanded blastocysts and a more physiological degree of chromosome aberrations of the embryos. In blastocysts derived from in vivo-matured oocytes, 21% of the embryos were mixoploid vs 50% from in vitro-matured oocytes, concomitant with a higher number of cells (96 vs 54 per normal blastocyst). The expression pattern of a set of six developmentally important genes was, however, not significantly altered in blastocysts derived from in vivo-matured oocytes. Certain deviations were observed compared with the levels of entirely in vivo-developed control blastocysts, which suggests that the beneficial effects of in vivo maturation are possibly exerted at initial stages of embryonic development. Prematuration in vivo, occurring in a dominant follicle developing from about 8 mm into the preovulatory follicle, is accompanied by changes in protein synthesis of the cumulus oocyte complex (COC). Presumably, the differentially expressed proteins are involved in equipping the oocyte with further developmental competence. Although we have unraveled some important biochemical and cellular biological features of the oocyte, further research on in vivo processes is essential to improve in vitro embryo production in practice.

We present a new kernel method for appearance-based object recognition, highly robust to noise and occlusion. It consists of a fully connected Markov random field that integrates results of Spin Glass theory with Gibbs probability distributions via nonlinear kernel mapping. We call this model Spin Glass-Markov Random Field. We present theoretical analysis and several experiments that show its effectiveness and robustness to noise and occlusion. We obtain in both cases excellent results. Particularly, we achieve a recognition rate above 93% with just 40% of visible portion of the object.

This contribution discusses the thermodynamic phases and storage capacity of an extension of the Hopfield-Little model of associative memory via kernel functions. The analysis is presented for the case of polynomial and Gaussian kernels in a replica symmetry ansatz. As a general result we found for both kernels that the storage capacity increases considerably compared to the Hopfield-Little model.

Shape and color information are important cues for object recognition. An ideal system should give the option to use both forms of information, as well as the option to use just one of the two. We present in this paper a kernel method that achieves this goal. It is based on results of statistical physics ofd isordered systems combined with Gibbs distributions via kernel functions. Experimental results on a database of 100 objects confirm the effectiveness of the proposed approach.

This paper addresses the problem of image segmentation using motion and luminance information. We use the dominant motion model to calculate both the background and foreground motion in a robust estimation framework and then combine it with the result of static segmentation using the watershed algorithm to segment the foreground from the background. In this paper, the previous pixel-based (or over a small neighborhood) motion measure is replaced by the patch-based motion measure in motion segmentation. Experimental results are given to show the efficiency of our method.

Within the EC-funded project SQEL,the German EVAR spoken dialogue system has been extended with respect to multilinguality and multifunctionality. The current demonstrator can handle four different languages and domains: German, Slovak, and Czech (and their national train connections), and Slovenian (European flights). It is currently being extended to the Czech language (Czech train connections). The SQEL demonstrator can also access databases on the WWW, which enables users without an internet connection to meet their information needs by just using the phone. The system starts up in German with a German opening phrase and the user is free to use any of the implemented languages. A multilingual word recognizer implicitly identifies the language, which is then associated with the appropriate domain and database. For the remainder of the dialogue, the corresponding monolingual recognizer is used instead. Experiments to date have shown that the multilingual and the (respective) monolingual recognizers attain comparable word accuracy rates, although the former is less efficient. The existence of language-independent task parameters, such as goal and source location, has meant that porting the system to a new language involves mainly the development of lexica and grammars (appart from the word recognizers) and not an extensive restructuring of the interpretation process within the Dialogue Manager. The latter is sufficiently flexible to switch between the different domains and languages.

This article presents the speech understanding and dialog system EVAR. All levels of linguistic knowledge are used both to control the analysis process and for the interpretation of an utterance. All kinds of knowledge are integrated in a homogeneous knowledge base. The control algorithm used for the analysis is defined within the representation scheme and does not depend on the application. One of the aims of EVAR is to develop a system structure where linguistic and non-linguistic expectations could be used not only for the interpretation but also as predictions for the recognition process.

The authors give a detailed account of a system environment for treating general problems of image and speech understanding. A framework for representing declarative and procedural knowledge based on a suitable definition of a semantic network is provided. The syntax and semantics of the network are clearly defined. The pragmatics of the network in its use for pattern understanding are defined by several rules which are problem-independent. This allows one to formulate problem-independent control algorithms. Complete software environments are available to handle the described structures. The general applicability of the network system is demonstrated by short descriptions of three applications from different task domains. >

This is the winter semester release of the slides of the lecture "Pattern Recognition" of the Winter Term 20/21. Slides are released under Creative Commons License 4.0 BY. The sources are available here:https://github.com/akmaier/pr-slides