Search

Multi-modal sentiment analysis plays an important role for providing better interactive experiences to users. Each modality in multi-modal data can provide different viewpoints or reveal unique aspects of a user's emotional state. In this work, we use text, audio and visual modalities from MOSI dataset and we propose a novel fusion technique using a multi-head attention LSTM network. Finally, we perform a classification task and evaluate its performance., 2 pages, 1 figure, 1 table

Storytelling has been established as a proven method to effectively communicate and assist in knowledge transfer. In recent years, there has been growing interest in improving the training and learning domain by using advanced technology such as Virtual Reality (VR). However, a gap exists between storytelling and VR, and it is as yet unclear how they can be combined to form an effective system that not only maintains the level of engagement and immersion provided by VR technology but also provides the core strengths of storytelling. In this paper, we present vIS, a Vocational Immersive Storytelling system, which bridges the gap between storytelling and VR. vIS focuses on vocational training, in which users are trained on how to use a mechanical micrometer by employing a creative fictional story embedded inside a virtual manufacturing plant&rsquo, s workplace. For the evaluation, a two-phase user study with 30 participants was conducted to measure the system&rsquo, s effectiveness and improvements in long-term training, as well as to examine user experience against traditional methods of training&mdash, 2D videos and textual manuals. The results indicate that the user&rsquo, s ability to retain their training after seven days was nearly equal for vIS and the 2D video-based technique and was considerably higher than the text-based technique.

As mobile technologies have become ubiquitous in recent years, computer-based cognitive tests have become more popular and efficient. In this work, we focus on assessing motor function in children by analyzing their gait movements. Although there has been a lot of research on designing automated assessment systems for gait analysis, most of these efforts use obtrusive wearable sensors for measuring body movements. We have devised a computer vision-based assessment system that only requires a camera which makes it easier to employ in school or home environments. A dataset has been created with 27 children performing the test. Furthermore, in order to improve the accuracy of the system, a deep learning based model was pre-trained on NTU-RGB+D 120 dataset and then it was fine-tuned on our gait dataset. The results highlight the efficacy of proposed work for automating the assessment of children’s performances by achieving 76.61% classification accuracy.

Training and assessment of a new worker is the most vital part of any vocational industry. The landscape of jobs and work is changing rapidly, thanks to the emerging new technology and the advancement of knowledge in the scientific fields. This technological and scientific revolution presents an opportunity in the creation of new industries and occupations, enhanced productivity and quality of work life, and the potential for more people to participate in the workforce. But these come at risk and disadvantage of an increased cost of training as well as lack of proper training in a few industries. In this paper, we propose to build a vIIS framework - A Vocational Interactive Immersive Storytelling system that uses storytelling in an Interactive virtual environment to train and assess a worker a new skill. The major advantage of this vIIS system is that it provides constant feedback in an engaging immersive virtual reality environment and has better memory retention and recall of the trained task. For assessment, we intend to collect data such as task completion time, accuracy, error rate, qualitative feedback and also measure transfer learning.

Physical fatigue due to muscle exhaustion is a symptom that can be very common in daily life. However fatigue can sometimes be suspect of more severe diseases such as multiple sclerosis and needs to be assessed appropriately. Despite the need to monitor fatigue, describing it in an objective and quantifiable manner is still an open problem due to the great levels of subjectivity involved. In this work we propose a novel method towards detecting physical fatigue. We design our approach based on objective EMG measurements and we aim to identify the presence of physical fatigue based on subjective user-reports. Based on our analysis we highlight the significance of our findings and we discuss how machine learning based modeling can become useful towards understanding fatigue and designing adaptive rehabilitation scenarios.

In this paper, we present Rewind/Remind (RR), a tool designed to help people assess and improve their associative memory. Rewind/Remind also serves as a tool for therapists by providing cognitive training based on memory research on helpful encoding strategies. RR is a flexible web browser application that is composed of two different interfaces: the Patient Interface (PI), and the Therapist Interface (TI). The PI consists of two different games -- Game 1: "Find the Link" and Game 2: "Create the Link". For Game 1, users have to find the link that connects two objects, while for Game 2, users have to define an association between objects. The TI provides analytical reports, allows for off-line monitoring of user performance, and gives the option to modify the sequence of tasks. These novel functionalities and approaches introduce promising new directions for cognitive remediation.

Lego construction task paradigms are utilized in order to develop logico-mathematical abilities through visuospatial memory. This study aims to assess the relationship between cognition and performance in a simulated industrial environment by employing humanoid robots to assess the stated metrics. This system proposes to develop a smart vocational assessment and intervention service system that assesses a worker's needs for training and rehabilitation in an experimental setup that simulates a factory. The proposed approach collects and analyzes multi-sensing data and recommends personalized interventions that can improve the performance of and individual worker. In our implementation, Aldebaran's NAO robot gradually learns the features and thresholds needed to construct a decision tree that gradually learns the expected Lego model by interacting with the user. The results from 615 test samples show that the NAO robot is able to correctly identify the Lego blocks configuration assembled by the user with an accuracy 81% of the time. Finally, we discuss the limitations of the proposed solution and we suggest future contributions that can overpass these limitations and boost the accuracy of our proposed solution.

There is a recent trend of research and applications of Cyber-Physical Systems (CPS) in manufacturing to enhance human-robot collaboration and production. In this paper, we propose a CPS framework for personalized Human-Robot Collaboration and Training to promote safe human-robot collaboration in manufacturing environments. We propose a human-centric CPS approach that focuses on multimodal human behavior monitoring and assessment, to promote human worker safety and enable human training in Human-Robot Collaboration tasks. We present the architecture of our proposed system, our experimental testbed and our proposed methods for multimodal physiological sensing, human state monitoring and interactive robot adaptation, to enable personalized interaction.

This paper proposes a novel system for assessing physical exercises specifically designed for cognitive behavior monitoring. The proposed system provides decision support to experts for helping with early childhood development. Our work is based on the well-established framework of Head-Toes-Knees-Shoulders (HTKS) that is known for its sufficient psychometric properties and its ability to assess cognitive dysfunctions. HTKS serves as a useful measure for behavioral self-regulation. Our system, CogniLearn, automates capturing and motion analysis of users performing the HTKS game and provides detailed evaluations using state-of-the-art computer vision and deep learning based techniques for activity recognition and evaluation. The proposed system is supported by an intuitive and specifically designed user interface that can help human experts to cross-validate and/or refine their diagnosis. To evaluate our system, we created a novel dataset, that we made open to the public to encourage further experimentation. The dataset consists of 15 subjects performing 4 different variations of the HTKS task and contains in total more than 60,000 RGB frames, of which 4,443 are fully annotated.

Extensive research has been carried out in using computer-based techniques to train and prepare workers for various industry positions. Most of this research focuses on how to best enable the workers to perform a type of task safely and efficiently. In fact, many of the accidents in manufacturing and construction environments are due to the lack of proper training needed for employees. In this study, we compare the impact of three types of training approaches on the planning and problem-solving abilities of a trainee while he/she performs the Towers of Hanoi (TOH) task. The three approaches are (a) traditional (with a human trainer), (b) gamification (game-based training simulation), and (c) computer-aided training. The aim of this study is to evaluate a worker’s level of functioning and problem-solving skills based on a specific training approach. Exact assessment of functional capacities is an important prerequisite to ensure effective and personalized training. The study uses workplace simulation to collect different types of performance data and assess the impact of these training approaches.

One of the traditional methods of learning a new language, or Second-Language Acquisition (SLA), is immersion, seen today as one of the most effective learning methods. Using this method, the learner relocates to a new place where the target language is the dominant language, and tries to learn the language my immersing themselves in the local environment. One of the disadvantages of this method is that relocating to a new place is not always a viable option. That's where Virtual Reality (VR) can come to our rescue. VR is an immersive technology that allows the user to wear a head-mounted display to be immersed in a life-like virtual environment. In our research, we explore the possibility of utilizing the power of VR for SLA by building a system called Ogma, which helps users immerse themselves in an environment representing a foreign place. In our pilot study, we built a VR system and performed user studies by comparing learning performance using VR against the traditional method. In both cases, users were given a set of ten Swedish words to learn. Results indicated that percentage retention using our VR method was significantly higher than that of the traditional method. In addition, the effectiveness and enjoyability ratings given by users were significantly higher for the VR method. This proves that our system has potential impact on SLA by using VR technology, without suffering the disadvantages of the traditional method.

In this paper, we present an adaptive cognitive music game designed to monitor and improve the attention levels of people with dementia. The goal of this game is to provide a customized protocol based on user needs and preferences, following the Reinforcement Learning (RL) framework. The game adjusts its parameters (e.g., difficulty level) so as to help the user complete the task successfully, while keeping them engaged. The main contribution of this paper is an interactive learning and adaptation framework that enables and facilitates the adaptation of the robot behavior towards new users, providing a safe, tailored and efficient interaction.

In this paper, we present an interactive learning and adaptation framework. The framework combines Interactive Reinforcement Learning methods to effectively adapt and refine a learned policy to cope with new users. We argue that implicit feedback provided by the primary user and guidance from a secondary user can be integrated to the adaptation mechanism, resulting at a tailored and safe interaction. We illustrate this framework with a use case in Robot Assisted Therapy, presenting a Robot Yoga Trainer that monitors a yoga training session, adjusting the session parameters based on human motion activity recognition and evaluation through depth data, to assist the user complete the session, following a Reinforcement Learning approach.

Automatic patient thought record categorization (TR) is important in cognitive behavior therapy, which is an useful augmentation of standard clinic treatment for major depressive disorder. Because both collecting and labeling TR data are expensive, it is usually cost prohibitive to require a large amount of TR data, as well as their corresponding category labels, to train a classification model with high classification accuracy. Because in practice we only have very limited amount of labeled and unlabeled training TR data, traditional semi-supervised learning methods and transfer learning methods, which are the most commonly used strategies to deal with the lack of training data in statistical learning, cannot work well in the task of automatic TR categorization. To address this challenge, we propose to tackle the TR categorization problem from a new perspective via self-taught learning, an emerging technique in machine learning. Self-taught learning is a special type of transfer learning. Instead of requiring labeled data from an auxiliary domain that are relevant to the classification task of interest as in traditional transfer learning methods, it learns the inherent structures of the auxiliary data and does not require their labels. As a result, a classifier achieves decent classification accuracy using the limited amount of labeled TR texts, with the assistance from the large amount of text data obtained from some inexpensive, or even no-cost, resources. That is, a cost-effective TR categorization system can be built that may be particularly useful for diagnosis of patients and training of new therapists. By further taking into account the discrete nature input text data, instead of using the traditional Gaussian sparse coding in self-taught learning, we use exponential family sparse coding to better simulate the distribution of the input data. We apply the proposed method to the task of classifying patient homework texts. Experimental results show the effectiveness of the proposed automatic TR classification framework.

In the digital city of the future there is the vision of seamless virtual and physical access for every home and between each home and the workplace, as well as critical city infrastructure such as the post office, the bank, hospitals, transportation systems, and other entities. This paper provides an overview of technical and other issues in extending at home (@home) assistive technologies for the elderly and the disabled. The paper starts by giving a vision of what this city is supposed to look like and how a human is to act, navigate and function in it. A framework for extending assistive technologies is proposed that considers individuals belonging to special groups of interest and locations other than their home. Technology has already reached the state of ubiquitous and pervasive sensor devices measuring everything, from temperature to human behavior. Implanting intelligence into and connecting such devices will be of immense use in preventive healthcare, security in industrial installations, greater energy efficiency, and numerous other applications. The paper reviews enabling technologies that exist and focuses on healthcare applications that support a longer and higher quality of life at home for the elderly and the disabled. It discusses intelligent platforms involving agents, context-aware and location-based services, and classification systems that enable advanced monitoring and interpretation of patient status and optimization of the environment to improve medical assessments. The paper concludes with a discussion of some of the challenges that exist in extending @home assistive technologies to @city assistive technologies.

Risk attitude reflects an intelligent agent's preference for different uncertain rewards. Cost is the resource consumed; wealth is the total amount of the resource an agent holds. In a risk-aware system whose risk attitude is not independent of wealth, utility is a function of wealth. Given the utility function is one-switch, if we simply use cost based utility functions for the reasoning, unless the initial wealth is zero, we cannot precisely obtain the optimum preference in every decision step. A bridge algorithm between cost and wealth helps us solve this problem. We provide a framework of the bridge algorithm for risk-aware Markov decision processes. We present an example of the block-world problem to explain the algorithm. An effective bridge algorithm helps planners to make reasonable decisions according to their risk attitudes, without changing the structure of Markov domains. A bridge between cost and wealth enables us to deal with planning domains using the powerful backward induction approach instead of decision tree. This will have profound theoretical and realistic influence on artificial intelligence, economics, health care, as well as other areas concerning risk.

Hierarchical algorithms for Markov decision processes have been proved to be useful for the problem domains with multiple subtasks. Although the existing hierarchical approaches are strong in task decomposition, they are weak in task abstraction, which is more important for task analysis and modeling. In this paper, we propose a task-oriented design to strengthen the task abstraction. Our approach learns an episodic task model from the problem domain, with which the planner obtains the same control effect, with concise structure and much improved performance than the original model. According to our analysis and experimental evaluation, our approach has better performance than the existing hierarchical algorithms, such as MAXQ and HEXQ.

Nonverbal communication helps to compensate many hidden meanings omitted from spoken language. In some situations, nonverbal communication may even take the place of verbal communication. In this paper, we introduce how nonverbal acoustic communication can be utilized in human-computer interaction. We first overview acoustic techniques for nonverbal communication. Then we provide a design framework of nonverbal communication based intelligent agents. We demonstrate an example of the design using an experimental iPad game.

Virtual Reality (VR) is becoming a tool that is more often used in various types of activities, including rehabilitation. However, studies using VR rehabilitation mainly focus on comparing the performances of participants, but not their opinions. In this paper, we present a virtual version of the Box and Blocks Test. We also present the results of a pilot study where participants completed a physical version of the Box and Blocks Test and the virtual version, comparing their scores and opinions. We also compare how the participants viewed the passage of time while performing both versions as a way to see how engaged they were during the task.

In this paper, we present an interactive learning and adaptation framework that facilitates the adaptation of an interactive agent to a new user. We argue that Interactive Reinforcement Learning methods can be utilized and integrated to the adaptation mechanism, enabling the agent to refine its learned policy in order to cope with different users. We illustrate our framework with a use case in the domain of Robot Assisted Therapy. We present our results of the learning and adaptation experiments against different simulated users, showing the motivation of our work and discussing future directions towards the definition and implementation of our proposed framework.

This paper proposes a deep learning classification method for frame-wise recognition of human activities, using raw color (RGB) information. In particular, we present a Convolutional Neural Network (CNN) classification approach for recognising three basic motion activity classes, that cover the vast majority of human activities in the context of a home monitoring environment, namely: sitting, walking and standing up. A real-world fully annotated dataset has been compiled, in the context of an assisted living home environment. Through extensive experimentation we have highlighted the benefits of deep learning architectures against traditional shallow classifiers functioning on hand-crafted features, on the task of activity recognition. Our approach proves the robustness and the quality of CNN classifiers that lies on learning highly invariant features. Our ultimate goal is to tackle the challenging task of activity recognition in environments that are characterized with high levels of inherent noise.

Physical exercising is an essential part of any rehabilitation plan. The subject must be committed to a daily exercising routine, as well as to a frequent contact with the therapist. Rehabilitation plans can be quite expensive and time-consuming. On the other hand, tele-rehabilitation systems can be really helpful and efficient for both subjects and therapists. In this paper, we present ReAdapt, an adaptive module for a tele-rehabilitation system that takes into consideration the progress and performance of the exercising utilizing multisensing data and adjusts the session difficulty resulting to a personalized session. Multimodal data such as speech, facial expressions and body motion are being collected during the exercising and feed the system to decide on the exercise and session difficulty. We formulate the problem as a Markov Decision Process and apply a Reinforcement Learning algorithm to train and evaluate the system on simulated data.

A great deal of recent research has focused on social and assistive robots that can achieve a more natural and realistic interaction between the agent and its environment. Following this direction, this paper aims to establish a computational framework that can associate objects with their uses and their basic characteristics in an automated manner. The goal is to continually enrich the robot's knowledge regarding objects that are important to the user, through verbal interaction. We address the problem of learning correlations between object properties and human needs by associating visual with verbal information. Although the visual information can be acquired directly by the robot, the verbal information is acquired via interaction with a human user. Users provide descriptions of the objects for which the robot has captured visual information, and these two sources of information are combined automatically. We present a general model for learning these associations using Gaussian Mixture Models. Since learning is based on a probabilistic model, the approach handles uncertainty, redundancy, and irrelevant information. We illustrate the capabilities of our approach by presenting the results of an initial experiment run in a laboratory environment, and we describe the set of modules that support the proposed framework.

The estimation of human attention as input modality has been suggested as a method for an advanced human-computer interaction. With an increasing interest and development of augmented reality tools, the advent of Microsoft HoloLens glasses and increasingly affordable wearable eye-tracking devices, monitoring the human attention will soon become ubiquitous. Also visual heat-maps have become very popular and simpler to create in the 2D space over the last few years. They are very compelling and can be effective in summarizing and communicating data. The innovation in our work is the implementation of visual 3D heat-maps of the real world combined with advanced Computer Vision libraries. Finally, we have incorporated the visual 3D heat-maps for rehabilitation purposes that deal with the loss of concentration in children with learning disabilities, or disabled patients to select items of interest for them across a room.

The spherical harmonic (SPHARM) description is a powerful surface modeling technique used by many applications in biomedical imaging. However, earlier SPHARM studies employ a spherical parameterization procedure that can be applied only to voxel surfaces. This paper presents CALD, a new spherical parameterization algorithm that makes the SPHARM model applicable to general triangle meshes. The CALD algorithm starts from an initial mapping and performs novel local and global smoothing methods alternately until a solution is approached. This new algorithm can also be used for processing of 3D closed surfaces in many different areas, including medical image analysis, computer vision, and computer graphics.

We present a new framework for 3D surface object classification that combines a powerful shape description method with suitable pattern classification techniques. Spherical harmonic parameterization and normalization techniques are used to describe a surface shape and derive a dual high dimensional landmark representation. A point distribution model is applied to reduce the dimensionality. Fisher's linear discriminants and support vector machines are used for classification. Several feature selection schemes are proposed for learning better classifiers. After showing the effectiveness of this framework using simulated shape data, we apply it to real hippocampal data in schizophrenia and perform extensive experimental studies by examining different combinations of techniques. We achieve best leave-one-out cross-validation accuracies of 93% (whole set, N = 56) and 90% (right-handed males, N = 39), respectively, which are competitive with the best results in previous studies using different techniques on similar types of data. Furthermore, to help medical diagnosis in practice, we employ a threshold-free receiver operating characteristic (ROC) approach as an alternative evaluation of classification results as well as propose a new method for visualizing discriminative patterns.

In this paper, we investigate the performance of a series of classification methods for the prediction of the hospital Length of Stay (LOS), based on two temporally sequential clinical scenarios. We used a 2012 Medicare Provider Analysis and Review (MedPar) dataset, which contains records of Medicare beneficiaries who used inpatient hospital services. Our subset included 300,000 randomly selected cases. During the prepossessing we added new features and linked our data with external datasets, using common key identifiers. In the first scenario our goal was to predict the LOS using a subset of information which is readily available to the clinician upon the patient admission, while the second scenario assumes that there is available additional data (information on the patient diagnosis and clinical procedures). For our experiments we used three different classifiers: Naive Bayes, AdaBoost and C4.5 Decision tree, for two different LOS cut-off points (4 day and 12 day hospital stay). The overall performance of our classifiers was ranging from fair to very good. On the other hand the true positive rate, that is the correct classification of the long hospital stays, was low, with an exception of Naive Bayes, which demonstrated significantly better performance in the second scenario. Our results indicate that Naive Bayes may be used for the prediction of the in-hospital LOS. Our analysis also indicates that the MedPar data combined with other data resources has the potential to provide a good basis for robust prediction analytics in hospitals.

Object recognition serves obvious purposes in assisted living environments, where robotic devices can be used as companions to assist humans in need. The recent introduction of vision based sensors, which are able to extract depth sensing information about the environment, in addition to the traditional RGB video, presents new opportunities and challenges for more accurate object recognition.The current work, presents an object recognition approach that uses RGB-D point cloud data and a novel feature extraction methodology, in combination with well-known supervised learning algorithms, to achieve accurate, real-time recognition of a large number of objects. In our experiments, we use a dataset of household objects organized into 51 categories, and evaluate the recognition accuracy and time efficiency of a set of different supervised learning methods.

As the diffusion of computing technology into everyday life continues at an exponential rate, the technology is becoming increasingly embedded in artefacts and environments. One important environment for this diffusion is the domestic environment or ‘‘Oikos’’ as it was known in ancient Greece. People’s living environment is particularly important when one takes into account the world’s ageing population and that as people grow older, they will increasingly rely on technology. People will, however, have differing levels of computer knowledge and assisted living needs. In addition, awareness towards the elderly and the need of constant medical supervision for chronic patients or habitants of remote, isolated and underserved locations has risen recently. To this end, services once available at a few centralized locations, such as high quality medical services, are now intended to be made available through a network anytime, anyplace and to anyone [1]. In such an environment, provision of medical data communication is currently dependent on lowbandwidth point-to-point telemedicine systems [2–6]. An easily field-deployable ad hoc wireless network or even a basic sensor network [7] could provide a significant improvement in real-time patient monitoring and emergency communications capability in general. This special issue comes after the successful organization of the 1st ACM International Conference on ‘‘PErvasive Technologies Related to Assistive Environments—PETRA 2008’’, which was held from 15th till 19th of July 2008, in Athens, Greece (http://www.petrae.org/). In this special issue, we have tried to include extended in-depth analysis of the best works announced in the PETRA 2008 conference, dealing with the application of pervasive ubiquitous computing to assistive environments, taking also into account the target audience of the PUC journal. As already mentioned, the word ‘‘assistive’’ is used in a general way that includes not only assisting persons with obvious physical limitations and disabilities, but also empowering any human to improve his/her quality of life by enhancing sensing at all levels [8, 9]. The research works included in this special issue show how pervasive and ubiquitous technologies can put computers to the service of man and become assistive. As a result, the special issue is comprised by papers describing systems, algorithms and methodologies from 3 major thematic areas: (1) activity and behaviour recognition [10–12], (2) advanced and personalized human computer interfaces [13, 14] and (3) biomedical monitoring platforms [15, 16, 18]. More specifically, Bamis et al. [10] in their paper present a scalable framework for detailed behaviour interpretation of elders. The paper reports their early deployment experiences and presents the current progress in three main components: sensors, middleware and behaviour interpretation mechanisms that aim to make effective monitoring and assistive services a reality. I. Maglogiannis (&) Department of Computer Science and Biomedical Informatics, University of Central Greece, Papasiopoulou 2-4, 35100 Lamia, Greece e-mail: imaglo@ucg.gr

This paper highlights the problems associated with placing technical multimedia materials on-line on the World Wide Web. It discusses obstacles in (a) automating the collection and processing of the multimedia components, and (b) limitations in the interface design imposed by restrictions associated with continuous media and especially audio. Using as paradigm the publishing of multimedia technical conference proceedings (“DAGS‘95 Conference on Electronic Publishing and the Information Superhighway”), the paper proposes a manageable framework for the production of such proceedings, one that allows parallel development of the submitted papers in hypertext format and of the time-based media (talks) in multimedia format.

Multimedia authoring and publishing incorporates a variety of types of publications, from newly created multimedia presentations to digital libraries that incorporate a wide variety of pre-existing materials, from small self-published magazines to large productions that involve dozens or hundreds of workers and budgets nearing those of small feature films. In this paper, we consider an important form of electronic publication that is not frequently analyzed: the lecture, as reproduced and extended electronically. We present a methodology for transforming a recorded lecture into an appropriately useful and interactive multimedia publication—the hyper-reproduction—highlighting techniques appropriate for publications with limited resources (workers, time, bandwidth, etc.). We suggest techniques for automating parts of the construction and discuss the added capabilities of such reproductions and their effect on how users access, navigate, and retain information.

Creation of large and complex World Wide Web sites is hampered by the “page at a time” approach of many tools and the programming knowledge and custom software development required for automated solutions. This paper describes the development of Automatic Site Markup Language (ASML). ASML is a new markup language designed to automate and facilitate the production of large, complicated web sites which can include dynamic content or content that changes with time. ASML extends HTML with new, high-level features while still preserving complete compatibility with common browser and server technologies. It has powerful indexing and searching facilities, and enables the automatic translation of document formats. Most importantly, ASML provides HTML-like features at the site level rather than just the page level.

In this paper, we describe a framework for audiovisual human computer interaction using a PC. The architecture of our system is based on recognized speech from both audio and visual information, and a dialog system that is capable of deriving the user's intentions, taking into account contextual information. The design of our system goes beyond the traditional paradigm of using vocal commands to control a computer which is usually application dependant. In order to do so it employs an adaptive module, able to select an appropriate grammar that suits the program used at a particular time. Furthermore, our system utilizes the visual modality in addition to audio, for increased word accuracy. We carried out a number of experiments during which we acquired promising results, therefore showing the great prospect of our framework.

We present the application of a recently proposed probabilistic logical formalism, on the task of sensor data fusion in the USEFIL project. USEFIL seeks to extract valuable knowledge concerning the well-being of elderly people by combining information coming from low-cost, unobtrusive monitoring devices. The approach we adopt to device its data fusion component is based on the Event Calculus and the stochastic logic programming language ProbLog and aims towards constructing a semantic representation of the received data, usable by a Decision Support System that will assist elderly people in their every day activities and will provide to doctors, relatives and carers insights on the user's behaviour and health.

In this paper, we describe a novel methodology which employs machine learning as an alternative means to explore hospital characteristics and client satisfaction, for decision making and improved quality of care. We applied well known feature selection and data mining algorithms such as forward selection and Naive Bayes respectively, to determine patient satisfaction, which is an important indicator of quality of care in hospital settings. Our dataset comprised of three types of data, (i) patient perception about received care, (ii) nurse perception about the working environment and (iii) organizational attributes of the hospital. Our experimental results exhibited high classification accuracy (87%), allowing valid conclusions to be reached about the organizational and workforce factors which attribute to patient satisfaction. Our findings were validated using traditional statistical methods such as binomial correlation and linear regression.

In this paper, we describe several existing Augmented Reality projects that illustrate critical yet currently disparate applications of Augmented Reality. This paper then describes a partially implemented system (the Visually Pervasive Augmented World) that has begun work on merging all of these applications together in an attempt to provide as much functionality as possible, while keeping the hardware requirements as accessible to the general public as possible. Once completed, this system will offer complete customizability to users, allowing them to interact with and modify their augmented world through user-submitted static and animated 3D models, text, a custom scripting language, and location and object based Augmented Reality.

We propose a Bayesian filtering framework for online analysis of visual structured processes, which can be combined with the Echo State Network (ESN) to capture prior information. With the proposed method we mitigate the effective Markovian Behavior of the ESN. We are able to keep a set of hypotheses about the entire history of behaviors and to evaluate them online based on new observations. The performance is evaluated under two complex visual behavior understanding scenarios using public datasets: a visual process for a kitchen table preparation and a real life manufacturing process.

Assistive Environments is a rapidly growing research field where researchers try to introduce state of the art technology into everyday life aiming to improve its quality. Everyone can benefit from it but first and foremost people with special needs such as elderly, disabled or patients. The Major Research Instrument project aims to develop a state of the art web based application, named zScope, that will be able to monitor human behaviour. The proposed tool is part of zScope and is responsible for placing and monitoring the various sensors in the environment. It provides functionality for environment layout design and is able to recommend near optimal positions for each sensor, using the Extended Max Sum Decentralised Coordination [1] algorithm, monitor each sensor's health and respond to any failures that might occur.

Continuous location tracking in an assistive environment is considered to be the fundamental step for most of the monitoring applications, as it provides a very good indication about different ongoing activities. But, tracking in a multi-person environment, such as hospitals, requires simultaneous identification and localization mechanisms, which makes the system extremely complex. In this paper, we describe a system that uses RFID and other passive sensors deployed at various locations of the environment to continuously track and identify a person in a multi-person assistive environment. Given, sensor activations at different time points, the proposed system considers location uncertainties of each person and applies the nearest neighbor algorithm to update the location of a person.

Automatic patient though record categorization (TR) is important in Cognitive Behavior Therapy (CBT), which is an useful augmentation of standard clinic treatment for Major Depressive Disorder (MDD). Because both collection and labeling of TR data are expensive, it is cost prohibitive to require a large amount of TR data, as well as their cor-responding category labels, to train a classification model with high classification accuracy. As in practice we only have very limited amount of labeled and unlabeled training TR data, traditional semi-supervised learning methods and transfer learning methods, which are the most commonly used strategies to deal with the lack of training data in statistical learning, can not work well in the task of automatic TR categorization. With the recognition of these challenges, in this paper we propose to approach the TR categorization problem from a new perspective via self-taught learning, an emerging topic in machine learning. Self-taught learning is a special type of transfer learning, instead of requiring labeled data from an auxiliary domain that are relevant to the classification task of interest as in traditional transfer learning methods, it learns the inherent structures of the auxiliary data and does not require their labels. Consequently, we may learn a classifier using the limited amount of labeled TR texts to achieve decent classification accuracy, with the assistance from the large amount text data obtained from some cheap, or even no-cost, resources. That is, a cost effective TR categorization system can be built up, which is of particular use in practical diagnosis and training of new therapists. We demonstrate the proposed method in the task of classifying the real depression homework texts, where promising experimental results validate the effectiveness of our new method.

Sensor data generated by pervasive applications are very diverse and are rarely described in standard or established formats. Consequently, one of the greatest challenges in pervasive systems is to integrate heterogeneous repositories of sensor data into a single view. The traditional approach to data integration, where a global schema is designed to incorporate the local schemas, may not be suitable to sensor data due to their highly transient schemas and formats. Furthermore, researchers and professionals in healthcare need to combine relevant data from various data streams and other data sources, and to be able to perform searches over all of these collectively using a single interface or query. Often, users express their search in terms of a small set of predefined fields from a single schema that is the most familiar to them, but they want their search results to include data from other compatible schemas as well. We have designed and implemented a framework for a sensor data repository that gives access to heterogeneous sensor metadata schemas in a uniform way. In our framework, the user specifies a query in an arbitrary schema and specifies the mappings from this schema to all the collections he wants to access. To ease the task of mapping specification, our system remembers metadata mappings previously used and uses them to propose other relevant mapping choices for the unmapped metadata elements. That way, users may build their own metadata mappings based on earlier mappings, each time specifying (or improving) only those components that are different. We have created a repository using data collected from various pervasive applications in a healthcare environment, such as activity monitoring, fall detection, sleep-pattern identification, and medication reminder systems, which are currently undergoing at the Heracleia Lab. We have also developed a flexible query interface to retrieve relevant records from the repository that allows users to specify their own choices of mappings and to express conditions to effectively access fine-grained data.

Radio Frequency Identification (RFID) is an automatic identification (Auto-ID) Technology, which is most commonly used now days in healthcare for tracking and identifying objects. In the context of assistive environment, statistical query analysis over the history of Data generated from RFID Applications as well as real time monitoring of the patients or the elderly people (people who need assistance) are really important. But Data generated from these types of healthcare applications can be very large, if each individual object becomes RFID-Tagged. As a result, the RFID technology is also imposing a greater challenge to provide efficient query responses over these Data. In this paper, we show how to apply traditional Data Warehousing techniques to model these massive amounts of RFID Data. In short, we describe how to construct an RFID Warehouse so that important query analyses can be performed very efficiently. We also show how to process a continuous stream of RFID Data to answer real time queries using Sliding Window techniques. By the help of using synthetic Datasets, we conclude that querying over Data Warehouse is much faster than traditional Relational DBMS. We also find that the aforesaid performance improvement is expected to be much higher as the size of the Dataset increases.

One of the issues in healthcare systems or medical information systems is the reduction of medical errors to ensure patient safety. Inside an assistive environment, we apply RFID tags to monitor drug taking pattern and its consequences are reported to the care giver. This paper talks about an application which tracks the medicine intake pattern for the elderly using RFID readers and tags, motion sensors, and a wireless sensor mote. With the adoption of this ambient assistive technology in healthcare systems, the concept of heterogeneous sensor data management becomes an issue. In this paper, using a Web Based Caregiver Module makes the process of monitoring medicine intake for health-related matters of the elderly living alone simpler and easier. We also propose to use an energy efficient technique by using multiple sensor devices which employ a sequence of innetwork data fusion as needed.

In recent years, there is a growing interest about assisted living environments especially for the elderly who live alone, due to the increasing number of aged people. In order for them to live safe and healthy, we need to detect abnormal behavior that may cause severe and emergent situations for the elderly. In this work, we suggest a method that detects abnormal behavior using wireless sensor networks. We model an episode that is a series of events, which includes spatial and temporal information about the subject being monitored. We define a similarity scoring function that compares two episodes taking into consideration temporal aspects. We propose a way to determine a threshold to divide episodes into two groups that reduces wrong classification. Weights on individual functions that consist the similarity function are determined experimentally so that they can produce the good results in terms of area under curve in receiver operating characteristic (ROC) curve.

An advanced tutorial system for teaching the fundamentals of logic has been developed to run on unix work stations and commonly available microcomputers. An important part of this tutorial is the intelligent problem solving environment which allows students to practise writing logical sentences in mathematical notation. A natural language system for intelligent logic narrative analysis ( ilona ) allows students to type in their own logical sentences in plain English and then have the computer check their working when they write these in mathematical form. ilona is an intelligent tutoring system which allows students a great deal of initiative in problem solving and provides a degree of flexibility in answer evaluation not found in traditional cai systems. The concepts and structures used in the development of ilona are easily transferable to other domains.

There has been significant research in adapting the Semantic Web technologies to create flexible context aware pervasive systems to enhance fields such as assisted living or smart environments. Several ontology based techniques have been proposed to simplify modeling knowledge and its relationships, and several ontology centered middleware tools are currently being developed to provide flexible and viable solutions for application developers. However, middleware built on the basis of Semantic Web generally suffers from drawbacks in performance, which limits its practical applications in the real world. This paper proposes a framework to facilitate Quality of Service (QoS) in ontology centered context aware pervasive middleware. Our approach suggests that context-aware middleware that operate by contracting mutual agreements with the client applications and provide controls over the amount of data to be processed by them can achieve predictable performance and response times. We also propose a service contract scheme that allows both client applications and middleware to participate in the decisions regarding the necessary data transformations required by the different system components in order to improve the overall system performance.

Due to the increasing number of the elderly, more and more people need to have additional health care such as medical or environmental monitoring information at home or nursing facility. Most elderly people are likely to have a sudden behavioral changes due to their aging or existing health problems. Therefore, it is necessary to have an autonomous system that can monitor them in order to prevent emergent situation in advance. In this paper, we present a wireless sensor network system that can recognize human behavioral patterns of the elderly who lives alone. We model episodes that are series of events for a person who lives in an one-bedroom apartment. We propose data cleaning techniques in both sensor and base station sides for the erroneous environment of wireless sensor networks. Based on these techniques, we try to extract discrete events as close as possible to effective events. We introduce non-real time analysis to recognize human behavioral patterns on the centralized system, which can be further extended to a real-time analysis. We also adopt an existing search technique to apply it to detect similar or abnormal behavior. We experiment the proposed system by gathering behavioral pattern data from the miniature one-bedroom apartment that is equipped with SunSPOTs in our HERACLEIA Laboratory. We look up the resulting episodes from our experiment in the dictionary that is a set of predetermined episodes using the suggested algorithm.