Search

Enabling machines to emotion recognition in conversation is challenging, mainly because the information in human dialogue innately conveys emotions by long-term experience, abundant knowledge, context, and the intricate patterns between the affective states. We address the task of emotion recognition in conversations using external knowledge to enhance semantics. We propose KES model, a new framework that incorporates different elements of external knowledge and conversational semantic role labeling, where build upon them to learn interactions between interlocutors participating in a conversation. We design a self-attention layer specialized for enhanced semantic text features with external commonsense knowledge. Then, two different networks composed of LSTM are responsible for tracking individual internal state and context external state. In addition, the proposed model has experimented on three datasets in emotion detection in conversation. The experimental results show that our model outperforms the state-of-the-art approaches on most of the tested datasets.

Metonymy is one of the types of common figurative languages and often used in human conversation without any difficulties. However, metonymy recognition in NLP requires a deep semantic/contextual processing to interpretation because it is highly related to the discourse of the contexts. Moreover, the fact that few available datasets of figurative languages make it more problematic. Motivated by the shortcomings of metonymy recognition, we develop several new data sets, including the Chinese version of the data, and design an end-to-end neural network metonymy recognizer. Our framework is based on the semantic priority interrupt theory and additional knowledge is introduced which makes to learn contexts effectively. Through a series of experiments, we show that our method is comparable to the state-of-the-art metonymy recognition method, especially we verified that metonymy trigger words information contributes to performance improvement in our model.

Sarcasm is a form of figurative language where the literal meaning of words cannot hold, and instead the opposite interpretation is intended in a text. Sarcasm detection is a significant task to mine fine-grained information, which is a much more difficult challenge for sentiment analysis. Both industry and academia have realized the importance of sarcasm detection. However, most existing methods do not work very well. Using a neural architecture, we propose a novel multi-dimension question answering (MQA) network in order to detect sarcasm. MQA not only introduces the abundant semantic information to understand the ambiguity of sarcasm by multi-dimension representations, but also builds the conversation context information by deep memory question answering network based on bidirectional LSTM and attention mechanism to discover sarcasm. The experimental results show that our model has ability to obviously outperform other state-of-the-art methods, and then further examples also verifies the advancement and effectiveness of our proposed network for detecting sarcasm.

Chatbot (and voicebot) applications are increasingly adopted in various domains such as e-commerce or customer services as a direct communication channel between companies and end-users. Multiple frameworks have been developed to ease their definition and deployment. While these frameworks are efficient to design simple chatbot applications, they still require advanced technical knowledge to define complex interactions and are difficult to evolve along with the company needs (e.g. it is typically impossible to change the NL engine provider). In addition, the deployment of a chatbot application usually requires a deep understanding of the targeted platforms, especially back-end connections, increasing the development and maintenance costs. In this paper, we introduce the Xatkit framework. Xatkit tackles these issues by providing a set of Domain Specific Languages to define chatbots (and voicebots and bots in general) in a platform-independent way. Xatkit also comes with a runtime engine that automatically deploys the chatbot application and manages the defined conversation logic over the platforms of choice. Xatkit's modular architecture facilitates the separate evolution of any of its components. Xatkit is open source and fully available online.

The article analyses tools and applies the base for processes to the areas of artificial intelligence that are Immersed in the daily lives of people, companies, institutions, and the industry, in constant interaction, without the human beings realizing the process. For achieving this, work has been done on the subarea of conversational agents that offers new ways and experiences of communicating with users. Software modules that have been designed to recognize the patterns of human writing generating an appropriate response. This research presents an analysis of the evolving needs of users and the support of emerging technologies concerning automated communication and the implementation of a prototype of a conversational agent or chatbot that uses artificial intelligence and is focused on natural language processing; the prototype can establish a conversation with students naturally.

When breakdowns occur during a human-chatbot conversation, the lack of transparency and the “black-box” nature of task-oriented chatbots can make it difficult for end users to understand what went wrong and why. Inspired by recent HCI research on explainable AI solutions, we explored the design of in-application explainable chatbot interfaces (ChatrEx) that explain the underlying working of a chatbot during a breakdown. ChatrEx-VINC provides visual example-based step-by-step explanations in-context of the chat window whereas ChatrEx-VST provides explanations as a visual tour overlaid on the application interface. We implemented these chatbots for complex spreadsheet tasks and our comparative observational study (N=14) showed that the explanations provided by both ChatrEx-VINC and ChatrEx-VST enhanced users' understanding of the reasons for a breakdown and improved users' perceptions of usefulness, transparency, and trust. We identify several opportunities for future research to exploit explainable chatbot interfaces and better support human-chatbot interaction.

In the digital era, cyber security has become a serious problem. Information penetrates, wholesale fraud, manual human test breaking, and other comparable occurrences proliferate, influencing a large number of individuals just as organizations. The hindrances have consistently been endless in creating appropriate controls and procedures and putting them in place with utmost precision in order to deal with cyber-attacks. To recent developments in artificial intelligence, the danger of cyber - attacks has increased drastically. AI has affected everything from healthcare to robots. Because malicious hackers couldn’t keep this ball of fire from them, “normal” cyber-attacks have grown in to the “intelligent” cyber attacks. In this paper, The most promising artificial intelligence approaches are discussed. Researchers look at how such techniques may be used for cyber security. At last, the conversation concludes with a discussion about artificial intelligence’s future and cyber security.

Just as the development of everything must follow certain laws, the development of informatization in cities (towns) also has certain laws, and a round of deep-seated trans-boundary integration innovation has quietly taken place. In the development process of "smart city", urbanization and informatization are gradually intersecting and deeply integrating. From the conventional conversation of smart city to the current construction of new smart city, it is essential to construct a set of scientific evaluation index system to guide the construction. This paper analyzes various evaluation index systems constructed in the traditional smart city construction research, which can better lead the build of the new smart city evaluation index system.

Communication over the phone has always been one of the most prone communication channels as it just requires a phone number to get connected and audio to communicate. Other than the already "spam" categorized calls of telecommunication companies providing offers and services, there are many other kinds of suspicious and unauthorized phone calls where the caller just stays behind the screen pretending to be a legit and an authorized person while being a non-legit one with malicious intent. People often come across the real-life incidents of such fraudulent activities including fund transfer on a call through illegal affairs or using the phone to make a threat or blackmail call to somebody. The solution for those fraudulent activity is to have software that can automatically categorize the call as non-legit or legit call and also store important points of conversations in the form of text-based notes for future reference, along with the audio recordings. The solution, therefore, is an ML based mobile application that can run in the background, record the call and analyse the conversations by using the ML algorithms to build the classification model. The classification obtained would then lead to the call being categorized as legit/spam/threat/blackmail and all the conversations would also be converted and stored as notes along with the audio recording for future reference.

Most of modern day Information Processing applications require Natural Language Processing. Most of the applications need extracting, processing of text to be done in their surface forms. Applications like document classification, text-extraction and machine translation comes under Natural Language Processing. Knowledge based approach of Natural Language Processing deals with representing and acquiring of knowledge to solve problems like ambiguity resolution. Ambiguities in Customer service conversation requires high level processing in order to generate the appropriate automated replies. Sentiment analysis is the interpretation of emotions and attitude of the user. Emotions can depend on mood of the user. In this work negative/positive probability is used to show the sentiment of a user which in turn can help the chat bot to generate a suitable reply to the user and escalate the conversation to a real representative when it sense that user is frustrated beyond a cutoff.

As indicated by the WHO, the variety of mental issues incorporates dementia, discouragement, schizophrenia and other psychoses, intellectual disabilities and developmental disorders. Sadness alone influences in excess of 300 million individuals. Indeed, even with admittance to proficient medical care and social administrations, those tormented with emotional well-being conditions wonder whether or not to profit from the therapy because of the stigma encompassing the disease. Accordingly, to this we would like to build an artificial intelligent agent that responds and helps the patient using an interactive chat bot. With the recent changes in Deep learning technology and specifically in NLP with models such as BERT and Open-AI GPT getting efficient contextual responses is not a task too hard. The chatbot will be able to identify, address and help relieve the mental stress of the user through soothing message responses similar to a therapist’s conversation. Also, other relaxation techniques are provided on the same platform for further support to the distressed user.

The closed design of virtual reality (VR) head-mounted displays substantially limits users’ awareness of their real-world surroundings. This presents challenges when another person in the same physical space needs to interrupt the VR user for a brief conversation. Such interruptions, e.g., tapping a VR user on the shoulder, can cause a disruptive break in presence (BIP), which affects their place and plausibility illusions, and may cause a drop in performance of their virtual activity. Recent findings related to the concept of diegesis, which denotes the internal consistency of an experience/story, suggest potential benefits of integrating registered virtual representations for physical interactors, especially when these appear internally consistent in VR. In this paper, we present a human-subject study we conducted to compare and evaluate five different diegetic and non-diegetic methods to facilitate cross-reality interruptions in a virtual office environment, where a user’s task was briefly interrupted by a physical person. We created a Cross-Reality Interaction Questionnaire (CRIQ) to capture the quality of the interaction from the VR user’s perspective. Our results show that the diegetic representations afforded reasonably high senses of co-presence, the highest quality interactions, the highest place illusions, and caused the least disruption of the participants’ virtual experiences. We discuss our findings as well as implications for practical applications that aim to leverage virtual representations to ease cross-reality interruptions.

Artificial Intelligence (AI) is the field of computer science progressing rapidly, simulating human behavior and thinking and affecting already the most of economic and social activities. The use of AI has undoubtedly positive outcomes but also implies high-risks. One of the most common AI applications is the case of facial recognition, a remote biometric identification system that raises the conversation around the intrusion into individuals’ privacy. In an attempt to balance ethical values and legal principles concerning, inter alia, automated recognition, European Commission puts forward a regulatory framework with specific objectives. The paper aims to highlight the precautionary principle that EU enforces in the case of facial recognition practice, establishing security mechanisms with the aim of mining, re-using and sharing data that are required for the buildout of a reliable Artificial Intelligence.

Based on previous studies on the co-occurrence between the types of laughter and conversation contents, this paper uses the attention mechanism to further study the types of laughter and the specific wording in the conversation. We assume that the types of laughter also co-occurs with specific words, as well as a certain pause, in the conversation. We have established a neural network learning model based on the attention mechanism. The results of our model show that certain words in the conversation are related to specific types of laughter.

Virtual reality (VR) is a promising tool for the tourism industry as it allows an amazing replica of locations in the real world offering the opportunity to provide prospective customers with a virtual travel experience. Especially with VR headsets that allow fully immersive travel experiences where the users are essentially placed within a virtual world, which they can move around and, in some cases, interact with. However, even if vision alone is often perceptually dominant, interaction between users and virtual content is a crucial element of any immersive visualization environment in both virtual reality and 360-degree content (photos and videos). In this article we describe a way to improve the interaction between users and virtual content by the means of non-playable (NPC) characters which have the ability to provide a natural language conversation during an immersive experience.

The COVID-19 pandemic has changed the way people do activities, including learning and teaching activities. The learning and teaching process during the pandemic is carried out online where teachers and students need certain technologies such as the WhatsApp application, one of the conversation applications is the choice of teachers and students in the learning process because it is considered effective and easy to use in discussing, sending materials, and sending student answers. However, it is not uncommon for photos sent via WhatsApp to have a low resolution, making it difficult for teachers to correct and students to understand the material. From these problems, an intelligent assignment management system will be made that integrates the feature of collecting tasks collected in the form of photos with handwriting text detection into digital text. This application utilizes the text conversion process with Handwriting Text Recognition. In Handwriting Text Recognition, there are six stages that must be carried out starting from preprocessing, segmentation, normalization, feature extraction, classification, and transcription. With the development of this CEBAN (Cek dan Koreksi Jawaban / Check and Correction Answers) Android application, handwriting on images or photos will be easily converted into digital text which is then used by teachers or correctors easily in reading student answers who collect assignments, especially pictures. This of course will shorten the correction time because the text in digital form is no longer in the form of photos or images which most likely have different resolutions, different types of handwriting, and image clarity which will make it difficult for teachers to correct answers.

The unprecedented COVID scenario has exponentially increased the exposure of internet to adolescent age group of the world population. The sudden extended online presence of this age group has created an area of concern in parenting fraternity as this is accompanied by restrictions on the normal social life of children. However, study states that direct parental control methods lessen the trust and the bond between the parent and the teen. If the teens are explained about the possible consequences of their online actions in a friendly manner, it helps them to understand the need of self-regulation and caution when they are being part of online community actively or passively. In this work, a self- regulation model is proposed. The self-regulation model is developed as a Mobile app called “Teen-alyse”. This app is designed to help in achieving a balance in parental guidelines and teen self-regulation. Further, the idea of active mediation has been included which encourages both, parent, and teen to have conversation for the time limit set for the mobile apps. By applying an analysis algorithm, the teen's YouTube activities are classified as flagged and not flagged contents and a report is generated, which alerts teens as well as parents. The work proposes the model of teen self-regulating app by following values of parental control, teen self-monitoring and active mediation.

Emoticons are miniature pictures that are customarily used in internet community Communications in the 21st century. The fusion of textual and imagery contained in the same message develops today's modern way of conversation. In spite of being universally utilized in online media, Emoticons basic interpretation has received very little observation from a “Natural Language Processing” point of view. In this paper, we investigate the relation between facial expressions and emoticons, studying the novel task of predicting which emojis are evoked by the user's facial expressions. We experimented with variants of word embedding techniques, and train various models based on MNBs and LSTMs in this task respectively. The experimental results show that our model can predict reasonable emoticons from emotions.

With the development of human-machine dialogue technology, more and more companies have launched their cognitive service products, such as Virtual Personal Assistant (VPA), smart speakers, shopping guide robots, etc. However, in these practical applications, most of the bots passively respond to user's utterances, lacking user preference knowledge and the proactive consciousness to lead the dialogue. Therefore, it is essential that bots proactively and naturally lead the dialogue from chitchat to service recommendation to meet user's requirements. To address this challenge, bots not only needs to detect the user's dialogue goal in real time, but also needs to plan a goal sequence based on user profile. In this paper, we propose DGPF, a Dialogue Goal Planning Framework. DGPF plans a reasonable goal sequence grounded on user's interests and personal KB before the conversation, additionally predicts user's true intent (i.e. dialogue goal) and judges whether the goal is completed based on the utterances during the conversation. DGPF includes a novel joint learning model that can simultaneously fix the two sub-tasks of goal completion estimation as well as current goal prediction, and improve each other's performance interactively. Our experimental results on the open dataset DuRecDial have been significantly improved compared to the baseline, which proves the effectiveness of our framework.

The significant development of chatbot technology began with the implementation of the Seq2Seq (Sequence-to-Sequence) model which is based on two RNN (Recurrent Neural Network) architecture. Long-term training with vanilla RNN has the problem of vanishing or exploding gradient, so it is necessary to do research that focuses more on optimization steps for this problem. The optimization carried out on the Seq2Seq model is to apply a LSTM (Long-Short Term Memory) gate mechanism with a stable ability to understand long-term information. A GRU (Gated Recurrent Unit) is faster in training and an attention mechanism that can handle long sequence data. Also, gradient descent can minimize the error rate. A combination of gate types, gradient descent, and attention mechanisms needs to be done to produce a better conversational model. The test results of the conversational model performance were obtained by taking into account the metric scores in accuracy, precision, recall, F1, and loss by using 80% train data and 20% test data from 5000 lines of Banjarese conversation. In the 50th epoch, the best score was from testing the LSTM gate model with metric values of accuracy 64,67%, precision 85,20%, recall 62,49%, F1 72,09%, and loss 3,6073.

Mental Health and Physical Health together are equally responsible for the overall well-being of an individual. We often give a preference to what we see (the body) rather than what we feel (the mind), resulting in negligence of our mental health. Moreover, getting treated for a mental illness is something considered as “negative”; hence with the stigma against mental issues, people do not openly talk about it. Over the years, therapy has proved quite effective in ameliorating one's mental state, but it fails to meet the expense of every pocket. Therefore, we propose EDRA - a chatbot created in order to communicate with the user and provide an environment just like in therapy. This virtual therapy works with the help of two principal models, the Emotion Recognition System and Deep Neural Network Model. The Emotion Recognition System helps to recognize the mood/emotion of the user with the text input it receives from the user, and with the help of this emotion, the Deep Neural Network Model yields a relevant response. On termination of the conversation, a report of the user's mental state is formed, and accordingly, specific remedies are suggested to lighten the user's mood.

Recently, deep neural network (DNN) based chat-bots offer great promise for generating responses. With architecture using an encoder-decoder integrated with attention mechanism, they showed a good one for generating natural responses, however they still are be short-sighted in predicting response one at a time while ignoring their influence on future outcomes. This is one of the reason DNN based models usually tend to generate responses not relevant to previous responses. In this paper, we show how to apply reinforcement learning to those models that can help them achieve a specific goal. By simulating dialogues between two pretrained chatbots with DNN, followed by using policy gradient methods to reward sequences. We also present our forward-looking function for desirable goal which will be used to improve the models. The experimental results show that the proposed model generates appropriate responses to content with more information relevant to the conversation context. In addition, our model improves up to 43% BLEU score compared to the baseline.

The sign language recognition system plays an important role in the field of human-computer interaction. In the daily life of hearing-impaired people, sign language is used as the main tool to communicate with the world. Although sign language can satisfy simple conversation, it is difficult to deal with in some situations where a lot of conversation is required such as medical emergencies or educational consultation. This paper proposes a sign language recognition system based on Wi-Fi to improve the life of the disabled. The proposed system collects the Channel State Information (CSI) due to the change of hand movement. Through the analysis of all subcarriers, the amplitude of CSI is determined to reflect the characteristics of different sign languages, some high-frequency noise is removed in the amplitude of CSI to obtain a smoother signal Gesture feature. We propose a gesture feature extraction method based on the variance of time series and DTW algorithm is used to recognize nine common Japanese sign language gestures. We set two daily conditions to test the system, and the experimental results show that the system performs well in different conditions.

Despite sign language being the primary medium of conversation with deaf and dumb (DD) individuals, it is hard to grasp for any class of people, making communication immensely challenging. Bengali being one of the most spoken languages globally, significant research works on Bangla Sign Language (BdSL) have come to light to deal with this challenge. In recent years, researchers from diverse backgrounds have proposed solutions to automate the process of BdSL recognition. In this literature review, we have explored the trends of research on BdSL by comparing incorporated features and evaluation results of systems between 2002-2021 and techniques applied on different existing and new datasets. We have also collected and combined metadata from datasets of all BdSL Alphabets and numerics used so far. The literature findings of this paper indicate that most proposed models perform well on images containing static and single-handed signs but, performance falls significantly in complex backgrounds. Moreover, we focused on identifying insights and similarities of the existing system, finding research gaps and propose possible future directions. We expect this article will draw the attention of more researchers into this domain and this is the first identifiable academic literature review of BdSL recognition systems to the best of our knowledge.

As the applications of intelligent agents (IAs) are gradually increasing in daily life, they are expected to have reasonable social intelligence to interact with people by appropriately interpreting human behavior and intention. This paper presents a method to estimate whether people have willingness to join in a conversation, which helps to endow IAs with the capability of detecting potential participants. The method is built on the CNN-LSTM network, which takes image features and social signals as input, making use of general information conveyed in images, semantic social cues proven by social psychology studies, and temporal information in the sequence of inputs. The network is designed to have a multi-branch structure with the flexibility of accommodating different types of inputs. We also discuss the signal transition in multiparty human-robot interaction scenarios. The method is evaluated on three datasets with social signals and/or images as inputs. The results show that the proposed method can infer human engagement intention well.

To explore the interaction design of an autonomous social robot stationed in a hotel room, we conducted a Wizard of Oz study. We developed a teleoperated robotic system that appears to move autonomously through voice-to-synthesis processing. Comparing the evaluation of the latest autonomous case with one of these teleoperated cases, the results show that it is possible to construct a robotic system that is more highly rated in terms of warmth, competence, and enjoyment of conversation. The results also suggest novel forms of the hotel room robot’s interactive services, such as a hotel-life management service and conversation partner service as a role of a listening presence, which draws out and understands with the guests’ talk.

Globalization and the effect of recent infectious disease are changing the remote conversations as a new normal in business meetings, social provision and casual chatting. Previous research show that the remote conversations have a difficulty on showing the presence or attention to the other interlocutors, especially in the situation where the majority of interlocutor share the same place and one or a few interlocutor participate from difference place. This paper proposed "Parappa", a remote video conversation system, for those unbalanced condition by utilizing both physical and virtual approaches to share the nonverbal behaviors of remotely-participating interlocutor. The proposed system is constructed with a rotatable screen which projects the life-size avatar of remote interlocutor. The rotation of the screen represents the body posture and the projected avatar show the gesture. The results of preliminary analyses of eye gaze activities and frequency of the screen rotation during conversation suggests the possibilities that proposed system enables to show the presence of remote interlocutors.

In this work, we designed a teachable robot that encourages a pair of students to discuss their thoughts and teaching decisions during the tutoring session. The robot adapts to the students’ talking activity and adjusts the frequency and type of encouragement. We hypothesize that the robot’s encouragement of group discussion can enhance the social engagement of group members, leading to improved learning and enjoyment. We ran a user study (n = 68), where a pair of participants (dyad) worked together to teach a humanoid robot about rocks and minerals. In the adaptive condition, the robot uses reinforcement learning to maximise interaction between the dyad members. Results show that the adaptive robot was successful in creating more dialogue between dyad members and in increasing task engagement, but did not affect learning or enjoyment. Over time, the adaptive robot was also able to encourage both members to contribute more equally to the conversation.

In this study, we developed a new rule-based dialog system that includes both turn-taking detection and response-switching functions. The traditional rule-based part allows the system user to design and customize topics and conversation transitions easily. In addition to the traditional rule-based part, the turn-taking detection function can detect the human speaker’s continuous speech and control the system’s speech response according to the detected continuous speech. This mechanism enables conversations without interfering with the speech of the human speaker. The turn-taking detection was created using a deep neural network (DNN). Using the turn-taking detection result, the response switching function can change three types of response transitions: a back-channeling response, topic changing, and normal conversation transition.To confirm that our developed functions could improve the impressions of rule-based dialog systems, an experiment comparing the dialog system with and without our developed functions was conducted. The system with both turn-taking detection and response switching functions provided a relatively better impression to the participants. Although some improvements are needed for the dialog system, the results suggest that the use of a turn-taking detection function may enable a relatively smooth conversation even with a rule-based dialog system.

The world is digitizing, and many chores, such as bank transactions and internet payments, are being digitized as a result. With the advancement of technology, we may also consider digitizing doctor-patient interactions in order to avoid personally consulting a doctor. This paper suggests a diagnostic conversation by utilizing artificial intelligence to better identify the illness before a doctor's appointment and include specific knowledge regarding the disease. This would increase the insurance expenses, improve the exposure of a larger population to treatment, and expand the exposure of a broader community to medical assistance. Some chatbots function like a medical guidebook and will help patients/doctors search for knowledge about any illness, but to profit entirely from a chatbot, all necessary tools must be used in order to support them identify and include the details required for disease detection. Both remedies and signs for the actual disease are not known by the average layman. With certain things that do not create any pain, this chatbot does not travel to a hospital directly with checks, because it might be a waste of time. The chatbot uses NLP to master the human language, which aims to grasp patients' health issues. The patient will raise questions dependent on healthcare without needing to travel to a doctor or clinic.

The conversation in the text is an interesting research on Natural Language Processing. One of the text conversation tasks is to know the emotions of the people involved in the conversation. The conversation in social media like Twitter, Instagram, short message service, WhatsApp, and so on, often involves emotion. Somehow the comments are impulsive sentences that can stimulate emotions. Expressing emotions using text are rarely done and uncomfortable. However, with natural language technology development, expressing emotions using text can be succeeded with specific symbols. We call the specific symbols emojis. So many emojis can express emotions. This research proposes the emoji symbol as a character feature. We introduce the Emoji2Vec method and Long Short-term Memory with Attention. The Attention that is used has a complex topology. We compared the results of this study with the baseline model. The method we propose is better than the baseline model.

Emotion Recognition in Conversations (ERC) is the task of identifying the emotions of utterances from speakers in a conversation. The major challenge of ERC is how to aggregate useful contextual information from multiple utterances. Existing research in this area has mainly focused on (1) the context modeling of utterances and (2) the interaction between speakers in conversation. Another concern with ERC is that emphatically enhancing context modeling might over contextualize surrounding information and neglect the valuable influence of context-free semantic information from the utterance. In this work, we present the cross-level attention to preserve context-free semantics of utterances and prevent over-contextualizing. Additionally, multiparty attention masks are introduced to better model complex speaker interactions by separating the conversation into speakers and others. The proposed methods are integrated with the transformer-based architecture resulting in the Cross-Level Attention with Multi-Party mask model (CLAMP). The experimental results indicate that CLAMP empirically achieves competitive performance on the IEMOCAP dataset. The proposed method presents and improvement on context utilization when adding conversation utterances. Additionally the contribution of the proposed components is demonstrated in an ablation study.

The burden on family caregivers for in-home care is increasing in a super-aging society. Realizing conversational agents adapted to individual situations for in-home care assistance is a hard technical issue. The purpose of this paper is to realize a method that can create a personalized conversational agent by the end-user. The challenging points of this paper include two aspects: (1) Editing conversation scenarios easier for end-users. (2) Sharing edited conversation scenarios with others. As the approach, we first present a platform of personalized conversation scenario (PopCS). Then, we create a logs-as-scenarios concept to improve the personalized contents and the edited efficiency. By selectively sharing the conversation logs, we build a connection between the local conversational agent and the remote that seems like telemedicine. Since different family caregivers and healthcare specialists join in it, continuing to improve the conversation scenarios for in-home care assistance is promising.

Emotion Recognition in Conversation (ERC) is becoming increasingly popular due to the accessibility of an enormous measure of openly accessible conversational information. Moreover, it has potential applications in opinion mining, social media and the health care domain. In this paper, we propose a novel Context and Knowledge Enriched Transformer Framework (CKETF) in which we interpret the contextual information from the utterances using a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model and leverage additive attention based hierarchical transformer for encoding the knowledge sentences. Experiments on the knowledge-grounded Topical Chat dataset shows that both context and external knowledge are important for conversational emotion recognition. We demonstrate through extensive experiments and analysis that our proposed model significantly outperforms the current state-of-the-art methods.

An important research direction in automatic speech recognition (ASR) has centered around the development of effective methods to rerank the output hypotheses of an ASR system with more sophisticated language models (LMs) for further gains. A current mainstream school of thoughts for ASR N-best hypothesis reranking is to employ a recurrent neural network (RNN)-based LM or its variants, with performance superiority over the conventional n-gram LMs across a range of ASR tasks. In real scenarios such as a long conversation, a sequence of consecutive sentences may jointly contain ample cues of conversation-level information such as topical coherence, lexical entrainment and adjacency pairs, which however remains to be underexplored. In view of this, we first formulate ASR N-best reranking as a prediction problem, putting forward an effective cross-sentence neural LM approach that reranks the ASR N-best hypotheses of an upcoming sentence by taking into consideration the word usage in its precedent sentences. Furthermore, we also explore to extract task-specific global topical information of the cross-sentence history in an unsupervised manner for better ASR performance. Extensive experiments conducted on the AMI conversational benchmark corpus indicate the effectiveness and feasibility of our methods in comparison to several state-of-the-art reranking methods.