Search

Abstractr We propose Quantized Dialog , a novel approach for the development of conversational systems. The methodology relies on the semantic quantization and clustering of the dialog utterances in order to reduce the dialog interaction space, making prediction of the next utterance more tractable. The effectiveness of this method is showcased using the goal-oriented dataset of the sixth Dialog System Technology Challenge (DSTC6). We compare the performance of Quantized Dialog based on an n-gram language model for next-utterance prediction against other models that employ popular deep-learning architectures, such as multi-layer neural network classifiers, memory networks, long short-term memory recurrent neural networks and convolutional neural networks. The experimental results demonstrate the promising potential of the new quantized approach in goal-oriented dialog prediction. Highlights • Quantized Dialog relies on semantic quantization and clustering of dialog utterances. • Once the proper clusters are obtained, the dialog policy can be learned efficiently. • A language model is obtained by learning transitions probabilities between clusters. • All five language models tested show very good performance (accuracy close to 100). • Quantized Dialog appealingly simple and promising option for conversational systems. [ABSTRACT FROM AUTHOR]

Highlights • An end-to-end goal-oriented dialog learning system is proposed. • Learning is conducted using a Bi-LSTM memory network model. • Performance increased with the use of metadata. • The Bi-LSTM memory network improves on the original and dynamic memory networks. Abstract We develop a model to satisfy the requirements of Dialog System Technology Challenge 6 (DSTC6) Track 1: building an end-to-end dialog systems for goal-oriented applications. This task involves learning a dialog policy from transactional dialogs in a given domain. Automatic system responses are generated using given task-oriented dialog data (http://workshop.colips.org/dstc6/index.html). As this task has a similar structure to a question answering task (Weston et al., 2015), we employ the MemN2N architecture (Sukhbaatar et al., 2015), which outperforms models based on recurrent neural networks or long short-term memory (LSTM). However, two problems arise when applying this model to the DSTC6 task. First, we encounter an out-of-vocabulary problem, which we resolve by categorizing the metadata types of words that exist in the knowledge base; the metadata is similar to the named entity. Second, the original memory network model has a weak ability to reflect sufficient temporal information, because it only uses sentence-level embeddings. Therefore, we add bidirectional LSTM (Bi-LSTM) at the beginning of the model to better reflect temporal information. The experimental results demonstrate that our model reflects temporal features well. Furthermore, our model achieves state-of-the-art performance among the memory networks, and is comparable to hybrid code networks (Ham et al., 2017) and hierarchical LSTM model (Bai et al., 2017) which is not an end-to-end architecture. [ABSTRACT FROM AUTHOR]

Highlights • The proposed system extends some of modules to be learned from data. • The proposed system is adaptable to changes in the knowledge base. • Extensions to hybrid code networks achieve the perfect accuracy in all DSTC6 test datasets. • The source code is available for download from github. Abstract Goal-oriented dialog systems require a different approach from chit-chat conversational systems in that they should perform various subtasks as well as continue the conversation itself. Since these systems typically interact with an external knowledge base that changes over time, it is desirable to incorporate domain knowledge to deal with such changes, yet with minimum human effort. This paper presents an extended version of the Hybrid Code Network (HCN) developed for the Facebook AI research (FAIR) dialog dataset used in the Sixth Dialog System Technology Challenge (DSTC6). Compared to the original HCN, the system was more adaptable to changes in the knowledge base due to the modules that are extended to be learned from data. Using the proposed learning scheme with fairly elementary domain-specific rules, the proposed model achieved 100% accuracy in all test datasets. [ABSTRACT FROM AUTHOR]

In this paper, we address issues in situated language understanding in a moving car. More specifically, we propose a reference resolution method to identify user queries about specific target objects in their surroundings. We investigate methods of predicting which target object is likely to be queried given a visual scene and what kind of linguistic cues users naturally provide to describe a given target object in a situated environment. We propose methods to incorporate the visual saliency of the visual scene as a prior. Crowdsourced statistics of how people describe an object are also used as a prior. We have collected situated utterances from drivers using our research system, which was embedded in a real vehicle. We demonstrate that the proposed algorithms improve target identification rate by 15.1% absolute over the baseline method that does not use visual saliency-based prior and depends on public database with a limited number of category information. [ABSTRACT FROM AUTHOR]

Abstract: Automatically detecting human social intentions and attitudes from spoken conversation is an important task for speech processing and social computing. We describe a system for detecting interpersonal stance: whether a speaker is flirtatious, friendly, awkward, or assertive. We make use of a new spoken corpus of over 1000 4-min speed-dates. Participants rated themselves and their interlocutors for these interpersonal stances, allowing us to build detectors for style both as interpreted by the speaker and as perceived by the hearer. We use lexical, prosodic, and dialog features in an SVM classifier to detect very clear styles (the strongest 10% in each stance) with up to 75% accuracy on previously seen speakers (50% baseline) and up to 59% accuracy on new speakers (48% baseline). A feature analysis suggests that flirtation is marked by joint focus on the woman as a target of the conversation, awkwardness by decreased speaker involvement, and friendliness by a conversational style including other-directed laughter and appreciations. Our work has implications for our understanding of interpersonal stance, their linguistic expression, and their automatic extraction. [Copyright &y& Elsevier]

Abstract: This paper describes a comprehensive usability evaluation of an automated telephone banking system which employs text-to-speech (TTS) synthesis in offering additional detail on customers’ account transactions. The paper describes a series of four experiments in which TTS was employed to offer an extra level of detail to recent transactions listings within an established banking service which otherwise uses recorded speech from a professional recording artist. Results from the experiments show that participants welcome the added value of TTS in being able to provide additional detail on their account transactions, but that TTS should be used minimally in the service. [ABSTRACT FROM AUTHOR]

Abstract: This paper studies the multifunctionality of dialogue utterances, i.e. the phenomenon that utterances in dialogue often have more than one communicative function. It is argued that this phenomenon can be explained by analyzing the participation in dialogue as involving the performance of several types of activity in parallel, relating to different dimensions of communication. The multifunctionality of dialogue utterances is studied by (1) redefining the notion of ‘utterance’ in a rigorous manner (calling the revised notion ‘functional segment’), and (2) empirically investigating the multifunctionality of functional segments in a corpus of dialogues, annotated with a rich, multidimensional annotation schema. It is shown that, when communicative functions are assigned to functional segments, thereby eliminating every form of segmentation-related multifunctionality, an average multifunctionality is found between 1.8 and 3.6, depending on what is considered to count as a segment''s communicative function. Moreover, a good understanding of the nature of the relations among the various multiple functions that a segment may have, and of the relations between functional segments and other units in dialogue segmentation, opens the way for defining a multidimensional computational update semantics for dialogue interpretation. [Copyright &y& Elsevier]

Abstract: This paper describes an initial prototype of the Companions project (www.companions-project.org): the Senior Companion (SC), designed to be a platform to display novel approaches to: [(1)] The use of Information Extraction (IE) techniques to extract the content of incoming dialogue utterances after an ASR phase. [(2)] The conversion of the input to RDF form to allow the generation of new facts from existing ones, under the control of a Dialogue Manager (DM), that also has access to stored knowledge and knowledge accessed in real time from the web, all in RDF form. [(3)] A DM expressed as a stack and network virtual machine that models mixed initiative in dialogue control. [(4)] A tuned dialogue act detector based on corpus evidence. The prototype platform was evaluated, and we describe this; it is also designed to support more extensive forms of emotion detection carried by both speech and lexical content, as well as extended forms of machine learning. We describe preliminary studies and results for these, in particular a novel approach to enabling reinforcement learning for open dialogue systems through the detection of emotion in the speech signal and its deployment as a form of a learned DM, at a higher level than the DM virtual machine and able to direct the SC''s responses to a more emotionally appropriate part of its repertoire. [Copyright &y& Elsevier]

Abstract: This paper proposes a novel user intention simulation method which is data-driven but can integrate diverse user discourse knowledge to simulate various types of user behaviors. A method of data-driven user intention modeling based on logistic regression is introduced in the Markov logic framework. Human dialog knowledge is designed into two layers, domain and discourse knowledge, and integrated with the data-driven model in generation time. Three types of user knowledge, i.e., cooperative, corrective and self-directing, are designed and integrated to generate behaviors of corresponding user-types. In experiments to investigate the patterns of simulated users, the approach successfully generated cooperative, corrective and self-directing user intention patterns. [ABSTRACT FROM AUTHOR]

Abstract: Multimodal conversational spoken dialogues using physical and virtual agents provide a potential interface to motivate and support users in the domain of health and fitness. This paper describes how such multimodal conversational Companions can be implemented to support their owners in various pervasive and mobile settings. We present concrete system architectures, virtual, physical and mobile multimodal interfaces, and interaction management techniques for such Companions. In particular how knowledge representation and separation of low-level interaction modelling from high-level reasoning at the domain level makes it possible to implement distributed, but still coherent, interaction with Companions. The distribution is enabled by using a dialogue plan to communicate information from domain level planner to dialogue management and from there to a separate mobile interface. The model enables each part of the system to handle the same information from its own perspective without containing overlapping logic, and makes it possible to separate task-specific and conversational dialogue management from each other. In addition to technical descriptions, results from the first evaluations of the Companions interfaces are presented. [ABSTRACT FROM AUTHOR]

Abstract: This paper argues that the problems of dialogue management (DM) and Natural Language Generation (NLG) in dialogue systems are closely related and can be fruitfully treated statistically, in a joint optimisation framework such as that provided by Reinforcement Learning (RL). We first review recent results and methods in automatic learning of dialogue management strategies for spoken and multimodal dialogue systems, and then show how these techniques can also be used for the related problem of Natural Language Generation. This approach promises a number of theoretical and practical benefits such as fine-grained adaptation, generalisation, and automatic (global) optimisation, and we compare it to related work in statistical/trainable NLG. A demonstration of the proposed approach is then developed, showing combined DM and NLG policy learning for adaptive information presentation decisions. A joint DM and NLG policy learned in the framework shows a statistically significant 27% relative increase in reward over a baseline policy, which is learned in the same way only without the joint optimisation. We thereby show that that NLG problems can be approached statistically, in combination with dialogue management decisions, and we show how to jointly optimise NLG and DM using Reinforcement Learning. [ABSTRACT FROM AUTHOR]

Abstract: A common task for spoken dialog systems (SDS) is to help users select a suitable option (e.g., flight, hotel, and restaurant) from the set of options available. As the number of options increases, the system must have strategies for generating summaries that enable the user to browse the option space efficiently and successfully. In the user-model based summarize and refine approach (UMSR, Demberg and Moore, 2006), options are clustered to maximize utility with respect to a user model, and linguistic devices such as discourse cues and adverbials are used to highlight the trade-offs among the presented items. In a Wizard-of-Oz experiment, we show that the UMSR approach leads to improvements in task success, efficiency, and user satisfaction compared to an approach that clusters the available options to maximize coverage of the domain (Polifroni et al., 2003). In both a laboratory experiment and a web-based experimental paradigm employing the Amazon Mechanical Turk platform, we show that the discourse cues in UMSR summaries help users compare different options and choose between options, even though they do not improve verbatim recall. This effect was observed for both written and spoken stimuli. [ABSTRACT FROM AUTHOR]

Abstract: We describe the design and evaluation of two different dynamic student uncertainty adaptations in wizarded versions of a spoken dialogue tutoring system. The two adaptive systems adapt to each student turn based on its uncertainty, after an unseen human “wizard” performs speech recognition and natural language understanding and annotates the turn for uncertainty. The design of our two uncertainty adaptations is based on a hypothesis in the literature that uncertainty is an “opportunity to learn”; both adaptations use additional substantive content to respond to uncertain turns, but the two adaptations vary in the complexity of these responses. The evaluation of our two uncertainty adaptations represents one of the first controlled experiments to investigate whether substantive dynamic responses to student affect can significantly improve performance in computer tutors. To our knowledge we are the first study to show that dynamically responding to uncertainty can significantly improve learning during computer tutoring. We also highlight our ongoing evaluation of our uncertainty-adaptive systems with respect to other important performance metrics, and we discuss how our corpus can be used by the wider computer speech and language community as a linguistic resource supporting further research on effective affect-adaptive spoken dialogue systems in general. [Copyright &y& Elsevier]

Abstract: The automatic recognition of user’s communicative style within a spoken dialog system framework, including the affective aspects, has received increased attention in the past few years. For dialog systems, it is important to know not only what was said but also how something was communicated, so that the system can engage the user in a richer and more natural interaction. This paper addresses the problem of automatically detecting “frustration”, “politeness”, and “neutral” attitudes from a child’s speech communication cues, elicited in spontaneous dialog interactions with computer characters. Several information sources such as acoustic, lexical, and contextual features, as well as, their combinations are used for this purpose. The study is based on a Wizard-of-Oz dialog corpus of 103 children, 7–14 years of age, playing a voice activated computer game. Three-way classification experiments, as well as, pairwise classification between polite vs. others and frustrated vs. others were performed. Experimental results show that lexical information has more discriminative power than acoustic and contextual cues for detection of politeness, whereas context and acoustic features perform best for frustration detection. Furthermore, the fusion of acoustic, lexical and contextual information provided significantly better classification results. Results also showed that classification performance varies with age and gender. Specifically, for the “politeness” detection task, higher classification accuracy was achieved for females and 10–11 years-olds, compared to males and other age groups, respectively. [Copyright &y& Elsevier]

Abstract: When conveying information about spatial situations and goals, speakers adapt flexibly to their addressee in order to reach the communicative goal efficiently and effortlessly. Our aim is to equip a dialogue system with the abilities required for such a natural, adaptive dialogue. In this paper we investigate the strategies people use to convey route information in relation to a map by presenting two parallel studies involving human–human and human–computer interaction. We compare the instructions given to a human interaction partner with those given to a dialogue system which reacts by basic verbal responses and dynamic visualization of the route in the map. The language produced by human route givers is analyzed with respect to a range of communicative as well as cognitively crucial features, particularly perspective choice and references to locations across levels of granularity. Results reveal that speakers produce systematically different instructions with respect to these features, depending on the nature of the interaction partner, human or dialogue system. Our further analysis of clarification and reference resolution strategies produced by human route followers provides insights into dialogue strategies that future systems should be equipped with. [Copyright &y& Elsevier]

Abstract: Spoken dialog systems have difficulty selecting which action to take in a given situation because recognition and understanding errors are prevalent due to noise and unexpected inputs. To solve this problem, this paper presents a hybrid approach to improving robustness of the dialog manager by using agenda-based and example-based dialog modeling. This approach can exploit n-best hypotheses to determine the current dialog state in the dialog manager and keep track of the dialog state using a discourse interpretation algorithm based on an agenda graph and a focus stack. Given the agenda graph and multiple recognition hypotheses, the system can predict the next action to maximize multi-level score functions and trigger error recovery strategies to handle exceptional cases due to misunderstandings or unexpected focus shifts. The proposed method was tested by developing a spoken dialog system for a building guidance domain in an intelligent service robot. This system was then evaluated by simulated and real users. The experimental results show that our approach can effectively develop robust dialog management for spoken dialog systems. [Copyright &y& Elsevier]

Abstract: This paper explains how Partially Observable Markov Decision Processes (POMDPs) can provide a principled mathematical framework for modelling the inherent uncertainty in spoken dialogue systems. It briefly summarises the basic mathematics and explains why exact optimisation is intractable. It then describes in some detail a form of approximation called the Hidden Information State model which does scale and which can be used to build practical systems. A prototype HIS system for the tourist information domain is evaluated and compared with a baseline MDP system using both user simulations and a live user trial. The results give strong support to the central contention that the POMDP-based framework is both a tractable and powerful approach to building more robust spoken dialogue systems. [Copyright &y& Elsevier]

Abstract: Prosody is an important cue for identifying dialog acts. In this paper, we show that modeling the sequence of acoustic–prosodic values as -gram features with a maximum entropy model for dialog act (DA) tagging can perform better than conventional approaches that use coarse representation of the prosodic contour through summative statistics of the prosodic contour. The proposed scheme for exploiting prosody results in an absolute improvement of 8.7% over the use of most other widely used representations of acoustic correlates of prosody. The proposed scheme is discriminative and exploits context in the form of lexical, syntactic and prosodic cues from preceding discourse segments. Such a decoding scheme facilitates online DA tagging and offers robustness in the decoding process, unlike greedy decoding schemes that can potentially propagate errors. Our approach is different from traditional DA systems that use the entire conversation for offline dialog act decoding with the aid of a discourse model. In contrast, we use only static features and approximate the previous dialog act tags in terms of lexical, syntactic and prosodic information extracted from previous utterances. Experiments on the Switchboard-DAMSL corpus, using only lexical, syntactic and prosodic cues from three previous utterances, yield a DA tagging accuracy of 72% compared to the best case scenario with accurate knowledge of previous DA tags (oracle), which results in 74% accuracy. [Copyright &y& Elsevier]

Abstract: This paper describes in detail a novel approach to the reordering challenge in statistical machine translation (SMT). This Ngram-based reordering (NbR) approach uses the powerful techniques of SMT systems to generate a weighted reordering graph. Thus, statistical criteria reordering constraints are supplied to an SMT system, and this allows an extension to the SMT decoding search. The NbR approach is capable of generalizing reorderings that have been learned during training, through the use of word classes instead of words themselves. Improvement in translation performance is demonstrated with the EPPS task (Spanish and German to English) and the BTEC task (Arabic to English). [Copyright &y& Elsevier]

Abstract: In this paper, we describe RavenClaw, a plan-based, task-independent dialog management framework. RavenClaw isolates the domain-specific aspects of the dialog control logic from domain-independent conversational skills, and in the process facilitates rapid development of mixed-initiative systems operating in complex, task-oriented domains. System developers can focus exclusively on describing the dialog task control logic, while a large number of domain-independent conversational skills such as error handling, timing and turn-taking are transparently supported and enforced by the RavenClaw dialog engine. To date, RavenClaw has been used to construct and deploy a large number of systems, spanning different domains and interaction styles, such as information access, guidance through procedures, command-and-control, medical diagnosis, etc. The framework has easily adapted to all of these domains, indicating a high degree of versatility and scalability. [Copyright &y& Elsevier]

Abstract: This paper presents empirical results of an analysis on the role of prosody in the recognition of dialogue acts and utterance mood in a practical dialogue corpus in Mexican Spanish. The work is configured as a series of machine-learning experimental conditions in which models are created by using intonational and other data as predictors and dialogue act tagging data as targets. We show that utterance mood can be predicted from intonational information, and that this mood information can then be used to recognize the dialogue act. [Copyright &y& Elsevier]

Abstract: The Gaussian mixture model – Universal background model (GMM–UBM) system is one of the predominant approaches for text-independent speaker verification, because both the target speaker model and the impostor model (UBM) have generalization ability to handle “unseen” acoustic patterns. However, since GMM–UBM uses a common anti-model, namely UBM, for all target speakers, it tends to be weak in rejecting impostors’ voices that are similar to the target speaker’s voice. To overcome this limitation, we propose a discriminative feedback adaptation (DFA) framework that reinforces the discriminability between the target speaker model and the anti-model, while preserving the generalization ability of the GMM–UBM approach. This is achieved by adapting the UBM to a target speaker dependent anti-model based on a minimum verification squared-error criterion, rather than estimating the model from scratch by applying the conventional discriminative training schemes. The results of experiments conducted on the NIST2001-SRE database show that DFA substantially improves the performance of the conventional GMM–UBM approach. [Copyright &y& Elsevier]

The aim of this study was to investigate the influence of modality on collaboration processes between human and computer. Spoken and written interactions with a natural language dialogue system were compared using two real information-retrieval systems. In order to look for a restaurant (Experiment 1) or plan a trip (Experiment 2), participants performed several task-oriented dialogue scenarios. Although the spoken interaction mode was less efficient, it promoted collaboration, the use of personal pronouns and the literal form of the system''s command utterances. Overall, in the written mode, the emphasis was on the task and its performance, rather than on dialogue. These findings are discussed with respect to the effect of communication mode on collaboration in human–computer dialogue. [Copyright &y& Elsevier]

This paper argues that interactive knowledge acquisition systems would benefit from a tighter and more thorough incorporation of tutoring and learning principles. Current acquisition systems learn from users in a passive manner, and could instead be designed to incorporate the proactive capabilities that one expects of a good student. We first describe our analysis of the literature on teacher–student interaction and present a compilation of tutoring and learning principles that are relevant to interactive knowledge acquisition systems. We then point out what tutoring and learning principles have been used to date in the acquisition literature, though unintentionally and implicitly, and discuss how a more thorough and explicit representation of these principles would help improve how computers learn from users. We present our design and an initial implementation of an acquisition dialogue system called SLICK that represents acquisition principles and goals explicitly and declaratively, making the system actively reason about various acquisition tasks and generate its interactions dynamically. Finally, we discuss promising directions in designing acquisition systems by structuring interactions with users according to tutoring and learning principles. [Copyright &y& Elsevier]

Abstract: In the present paper, we investigate the validity and reliability of de-facto evaluation standards, defined for measuring or predicting the quality of the interaction with spoken dialogue systems. Two experiments have been carried out with a dialogue system for controlling domestic devices. During these experiments, subjective judgments of quality have been collected by two questionnaire methods (ITU-T Rec. P.851 and SASSI), and parameters describing the interaction have been logged and annotated. Both metrics served the derivation of prediction models according to the PARADISE approach. Although the limited database allows only tentative conclusions to be drawn, the results suggest that both questionnaire methods provide valid measurements of a large number of different quality aspects; most of the perceptive dimensions underlying the subjective judgments can also be measured with a high reliability. The extracted parameters mainly describe quality aspects which are directly linked to the system, environmental and task characteristics. Used as an input to prediction models, the parameters provide helpful information for system design and optimization, but not general predictions of system usability and acceptability. [Copyright &y& Elsevier]

Abstract: In this paper, we evaluate mass knowledge acquisition using modified ALICE chatterbots. In particular we investigate the potential of allowing subjects to modify chatterbot responses to see if distributed learning from a web environment can succeed. This experiment looks at dividing knowledge into general conversation and domain specific categories for which we have selected telecommunications. It was found that subject participation in knowledge acquisition can contribute a significant improvement to both the conversational and telecommunications knowledge bases. We further found that participants were more satisfied with domain-specific responses rather than general conversation. [Copyright &y& Elsevier]

Abstract: Dialogue systems for health communication hold out the promise of providing intelligent assistance to patients through natural interfaces that require no training to use. But in order to make the development of such systems cost effective, we must be able to use generic techniques and components which are then specialized as needed to the specific health problem and patient population. In this paper, we describe Chester, a prototype intelligent assistant that interacts with its user via conversational natural spoken language to provide them with information and advice regarding their prescribed medications. Chester builds on our prior experience constructing conversational assistants in other domains. The emphasis of this paper is on the portability of our generic spoken dialogue technology, and presents a case study of the application of these techniques to the development of a dialogue system for health communication. [Copyright &y& Elsevier]

Abstract: System-initiated digressive proposals may be used to introduce new and unexpected information into automated telephone services. These digressions may be viewed as particularly pronounced forms of unsolicited interruptions as they contain information not directly related to the caller''s intended activity. In human–human conversation, interruptions are considered to be speech acts which intrinsically threaten both the positive and negative face wants of the addressee and conversants adopt specific verbal strategies to mitigate the negative impact of their interruptions. A question therefore arises whether the introduction of face-redressive expressions, based on human–human conversational strategies, into the design of system-initiated proposals in automated services can mitigate the negative impact of the interruptions. A usability experiment was conducted to examine the effectiveness of three contrasting politeness strategies for system-initiated digressions in a mass-market telephone banking dialogue using speech recognition technology. Participants (N=111) experienced these proposals while using the automated service to perform banking tasks. Results indicated that all these system-initiated digressions—irrespective of politeness strategy employed—had a negative impact on the user attitudes towards the service. This paper reports these results and explores participants’ perceptions of the politeness styles and registers employed in the system-initiated proposals. [Copyright &y& Elsevier]