Search

This article devises a photograph-based monitoring model to estimate the real-time PM2.5 concentrations, overcoming currently popular electrochemical sensor-based PM2.5 monitoring methods’ shortcomings such as low-density spatial distribution and time delay. Combining the proposed monitoring model, the photographs taken by various camera devices (e.g., surveillance camera, automobile data recorder, and mobile phone) can widely monitor PM2.5 concentration in megacities. This is beneficial to offering helpful decision-making information for atmospheric forecast and control, thus reducing the epidemic of COVID-19. To specify, the proposed model fuses Information Abundance measurement and Wide and Deep learning, dubbed as IAWD, for PM2.5 monitoring. First, our model extracts two categories of features in a newly proposed DS transform space to measure the information abundance (IA) of a given photograph since the growth of PM2.5 concentration decreases its IA. Second, to simultaneously possess the advantages of memorization and generalization, a new wide and deep neural network is devised to learn a nonlinear mapping between the above-mentioned extracted features and the groundtruth PM2.5 concentration. Experiments on two recently established datasets totally including more than 100 000 photographs demonstrate the effectiveness of our extracted features and the superiority of our proposed IAWD model as compared to state-of-the-art relevant computing techniques.

The internet of things (IoT) is rapidly expanding and improving operations in a wide range of real-world applications, from consumer IoT and enterprise IoT to manufacturing and industrial IoT (IIoT). Consumer markets, wearable devices, healthcare, smart buildings, agriculture, and smart cities are just a few examples. This paper discusses the current state of the IoT ecosystem, its primary applications and benefits, important architectural stages, some of the problems and challenges it faces, and its future. This paper explains how an appropriate IoT architecture that saves data, analyzes it, and recommends corrective action improves the process’s ground reality. The IoT system architecture is divided into three layers: device, gateway, and platform. This then cascades into the four stages of the IoT architectural layout: sensors and actuators; gateways and data acquisition systems; edge IT data processing; and datacenter and cloud, which use high-end apps to collect data, evaluate it, process it, and provide remedial solutions. This elegant combination provides excellent value in automatic action. In the future, IoT will continue to serve as the foundation for many technologies. Machine learning will become more popular in the coming years as IoT networks take center stage in a variety of industries.

Current research of designing prosthetic robotic hands mainly focuses on improving their functionality by devising new mechanical structures and actuation systems. Most of existing work relies on a single structure/system (e.g., bone-only or tissue-only) and ignores the fact that the human hand is composed of multiple functional structures (e.g., skin, bones, muscles, and tendons). This may increase the difficulty of the design process and lower the flexibility of the fabricated hand. To tackle this problem, this paper proposes a three-dimensional (3D) printable multi-layer design that models the hand with the layers of skin, tissues, and bones. The proposed design first obtains the 3D surface model of a target hand via 3D scanning, and then generates the 3D bone models from the surface model based on a fast template matching method. To overcome the disadvantage of the rigid bone layer in deformation, the tissue layer is introduced and represented by a concentric tube-based structure, of which the deformability can be explicitly controlled by a parameter. The experimental results show that the proposed design outperforms previous designs remarkably. With the proposed design, prosthetic robotic hands can be produced quickly with low cost and be customizable and deformable.

This open access book details the relationship between the artist and their created works, using tools such as information technology, computer environments, and interactive devices, for a range of information sources and application domains. This has produced new kinds of created works which can be viewed, explored, and interacted with, either as an installation or via a virtual environment such as the Internet. These processes generate new dimensions of understanding and experience for both the artist and the public’s relationships with the works that are produced. This has raised a variety of interdisciplinary opportunities and issues, and these are examined. The symbiotic relationship between artistic works and the cultural context in which they are produced is reviewed. Technology can provide continuity by making traditional methods and techniques more efficient and effective. It can also provide discontinuity by opening up new perspectives and paradigms. This can generate new ideas, and produce a greater understanding of artistic processes and how they are implemented in practice. Tools have been used from the earliest times to create and modify artistic works. For example, naturally occurring pigments have been used for cave paintings. What has been created provides insight into the cultural context and social environment at the time of creation. There is an interplay between the goal of the creator, the selection and use of appropriate tools, and the materials and representations chosen. Technology, Design and the Arts - Opportunities and Challenges is relevant for artists and technologists and those engaged in interdisciplinary research and development at the boundaries between these disciplines.

In this work we have enhanced the perception of a humanoid robot by integrating it with a social state estimation system. We present a user study of the humanoid Nao robot as a social mediator, comprising two sets of experiments. In the first sets of experiments, the participants rate their understanding of feedback messages delivered via the Nao robot. They also assess two modalities to deliver the feedback: audio only and audio combined with gestures. In almost all cases there is an improvement of 10% or more when audio and gesture modalities are combined to deliver feedback messages. For the second sets of experiments the sociofeedback system was integrated with the Nao robot. The participants engage in two-person scenario-based conversations while the Nao robot acts as a mediator. The sociofeedback system analyzes the conversations and provides feedback via Nao. Subsequently, the participants assess the received sociofeedback with respect to various aspects, including its content, appropriateness, and timing. Participants also evaluate their overall perception of Nao as social mediator via the Godspeed questionnaire. The results indicate that the social feedback system is able to detect the social scenario with 93.8% accuracy and that Nao can be effectively used to provide sociofeedback in discussions. The results of this paper pave the way to natural human-robot interactions for social mediators in multi-party dialog systems. NRF (Natl Research Foundation, S’pore) Accepted version

New challenges have been brought out along with the emerging of 3D-related technologies, such as virtual reality, augmented reality (AR), and mixed reality. Free viewpoint video (FVV), due to its applications in remote surveillance, remote education, and so on, based on the flexible selection of direction and viewpoint, has been perceived as the development direction of next-generation video technologies and has drawn a wide range of researchers' attention. Since FVV images are synthesized via a depth image-based rendering (DIBR) procedure in the "blind" environment (without reference images), a reliable real-time blind quality evaluation and monitoring system is urgently required. But existing assessment metrics do not render human judgments faithfully mainly because geometric distortions are generated by DIBR. To this end, this paper proposes a novel referenceless quality metric of DIBR-synthesized images using the autoregression (AR)-based local image description. It was found that, after the AR prediction, the reconstructed error between a DIBR-synthesized image and its AR-predicted image can accurately capture the geometry distortion. The visual saliency is then leveraged to modify the proposed blind quality metric to a sizable margin. Experiments validate the superiority of our no-reference quality method as compared with prevailing full-, reduced-, and no-reference models. MOE (Min. of Education, S’pore)

Schizophrenia is a chronic mental disorder that contributes to poor function and quality of life. We are aiming to design objective assessment tools of schizophrenia. In earlier work, we investigated non-verbal quantitative cues for this purpose. In this paper, we explore linguistic cues, extracted from interviews with patients with schizophrenia and healthy control subjects, conducted by trained psychologists. Specifically, we analyzed the interviews of 47 patients and 24 healthy age-matched control subjects. We applied automated speech recognition and linguistic tools to capture the linguistic categories of emotional and psychological states. Based on those linguistic categories, we applied a binary classifier to distinguish patients from matched control subjects, leading to a classification accuracy of about 86% (by leave-one-out cross-validation); this result seems to suggest that patients with schizophrenia tend to talk about different topics and use different words. We provided an in-depth discussion of the most salient lexical features, which may provide some insights into the linguistic alterations in patients. NRF (Natl Research Foundation, S’pore) NMRC (Natl Medical Research Council, S’pore) Accepted version

Human hand is the best sample for humanoid robotic hand and a nature grasping is the final target that most robotic hands are pursuing. Many prior researches had been done in virtual and real for simulation the human grasping. Unfortunately, there is no perfect solution to duplicate the nature grasping of human. The main difficulty comes from three points. 1. How to 3D modelling and fabricate the real hand. 2. How actuated the robotic hand as real hand. 3. How to grasp objects in different shapes like human hand. To deal with these three problems and further to provide a partial solution for duplicate human grasping, this paper introduces our method to solve these problems from robotic hand design, fabrication, actuation and grasping plan. Our modelling progress takes only around 12 minutes that include 10 minutes of 3D scanning of a real human hand and two minutes for changing the scanned model to an articulated model by running our algorithm. Our grasping plan is based on the sampled trajectory and easy to implement for grasping different objects. Followed these steps, a seven DOF robotic hand is created and tested in the experiments. Published version

Schizophrenia is a long-term mental disease associated with language impairments that affect about one percent of the population. Traditional assessment of schizophrenic patients is conducted by trained professionals, which requires tremendous resources of time and effort. This study is part of a larger research objective committed to creating automated platforms to aid clinical diagnosis and understanding of schizophrenia. We have analyzed non-verbal cues and movement signals in our previous work. In this study, we explore the feasibility of using automatic transcriptions of interviews to classify patients and predict the observability of negative symptoms in schizophrenic patients. Interview recordings of 50 schizophrenia patients and 25 age-matched healthy controls were automatically transcribed by a speech recognition toolkit. After which, Natural Language Processing techniques were applied to automatically extract the lexical features and document vectors of transcriptions. Using these features, we applied ensemble machine learning algorithm (by leave-one-out cross-validation) to predict the Negative Symptom Assessment subject ratings of schizophrenic patients, and to classify patients from controls, achieving a maximum accuracy of 78.7%. These results indicate that schizophrenic patients exhibit significant differences in lexical usage compared with healthy controls, and the possibility of using these lexical features in the understanding and diagnosis of schizophrenia. NRF (Natl Research Foundation, S’pore) NMRC (Natl Medical Research Council, S’pore) Accepted version

It is often desirable for humanoid robots to have dexterous hands and perform human-like grasps. This requires a deliberate design of hands, in addition to a good actuation system and an efficient 3D vision system. In this paper, we present a simple method to produce a customized articulated robotic hand for a humanoid robot (eg., Nadine social robot). Our method acquires the 3D shape by 3D scanning, which can cost-effectively generate customized hands. Guided by the human hand anatomy and inspired by the progress of 3D printing technology, we model the structure of the robotic hand using standard shapes such as cuboids, spheres and cylinders, which are adapted to match the scanned shape and fabricated using 3D printing. Moreover, we choose flexible resin materials11https://formlabs.com/3d-printers/form-2/, which are supported by 3D printers, to make the fabricated hand flexible enough for natural deformation. As a result, our designed robotic hand has six degrees of freedom and together with a cable-driven actuation system can achieve the range of motion as a human hand. Experimental results demonstrated that our robotic hand is capable of grasping a variety of objects with different shapes. NRF (Natl Research Foundation, S’pore) Accepted version

This paper presents a system that aims to achieve autonomous grasping for micro-controller based humanoid robots such as the Inmoov robot [1]. The system consists of a visual sensor, a central controller and a manipulator. We modify the open sourced objection detection software YOLO (You Only Look Once) v2 [2] and associate it with the visual sensor to make the sensor be able to detect not only the category of the target object but also the location with the help of a depth camera. We also estimate the dimensions (i.e., the height and width) of the target based on the bounding box technique (Fig. 1). After that, we send the information to the central controller (a humanoid robot), which controls the manipulator (customised robotic hand) to grasp the object with the help of inverse kinematics theory. We conduct experiments to test our method with the Inmoov robot. The experiments show that our method is capable of detecting the object and driving the robotic hands to grasp the target object. Open image in new window Fig. 1. Autonomous grasping with real-time object detection

Negative symptoms in schizophrenia are associated with significant burden and possess little to no robust treatments in clinical practice today. One key obstacle impeding the development of better treatment methods is the lack of an objective measure. Since negative symptoms almost always adversely affect speech production in patients, speech dysfunction have been considered as a viable objective measure. However, researchers have mostly focused on the verbal aspects of speech, with scant attention to the non-verbal cues in speech. In this paper, we have explored non-verbal speech cues as objective measures of negative symptoms of schizophrenia. We collected an interview corpus of 54 subjects with schizophrenia and 26 healthy controls. In order to validate the non-verbal speech cues, we computed the correlation between these cues and the NSA-16 ratings assigned by expert clinicians. Significant correlations were obtained between these non-verbal speech cues and certain NSA indicators. For instance, the correlation between Turn Duration and Restricted Speech is -0.5, Response time and NSA Communication is 0.4, therefore indicating that poor communication is reflected in the objective measures, thus validating our claims. Moreover, certain NSA indices can be classified into observable and non-observable classes from the non-verbal speech cues by means of supervised classification methods. In particular the accuracy for Restricted speech quantity and Prolonged response time are 80% and 70% respectively. We were also able to classify healthy and patients using non-verbal speech features with 81.3% accuracy. NRF (Natl Research Foundation, S’pore) NMRC (Natl Medical Research Council, S’pore) Published version

In this paper, we present a novel method for real-time 3D hand pose estimation from single depth images using 3D Convolutional Neural Networks (CNNs). Image-based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial information. Our proposed 3D CNN-based method, taking a 3D volumetric representation of the hand depth image as input and extracting 3D features from the volumetric input, can capture the 3D spatial structure of the hand and accurately regress full 3D hand pose in a single pass. In order to make the 3D CNN robust to variations in hand sizes and global orientations, we perform 3D data augmentation on the training data. To further improve the estimation accuracy, we propose applying the 3D deep network architectures and leveraging the complete hand surface as intermediate supervision for learning 3D hand pose from depth images. Extensive experiments on three challenging datasets demonstrate that our proposed approach outperforms baselines and state-of-the-art methods. A cross-dataset experiment also shows that our method has good generalization ability. Furthermore, our method is fast as our implementation runs at over 91 frames per second on a standard computer with a single GPU. NRF (Natl Research Foundation, S’pore) MOE (Min. of Education, S’pore) Accepted version

Today, to create and to simulate a virtual anatomical version of a subject is useful in the decision process of surgical treatments. The muscular activity is one of the factors which can contribute to abnormal movements such as in spasticity or static contracture. In this paper, we propose a numerical solution, based on the Finite Element (FE) method, able to estimate muscles deformations during contraction. Organized around a finite element solver and a volumetric environment, this solution is made of all the modeling and simulation processes from the discretization of the studied domain to the visualization of the results. The choices of materials and properties of the FE model are also presented such as the hyperelasticity, the contention model based on inter-meshes neighboring nodes pairing, and the estimation of nodal forces based on the subject-specific muscular forces and action lines.

Nonverbal communication is an important aspect of real-life face-to-face interaction and one of the most efficient ways to convey emotions, therefore users should be provided the means to replicate it in the virtual world. Because articulated embodiments are well suited to provide body communication in virtual environments, this paper first reviews some of the advantages and disadvantages of complex embodiments. After a brief introduction to nonverbal communication theories, we present our solution, taking into account the practical limitations of input devices and social science aspects. We introduce our sample of actions and implementation using our VLNET (Virtual Life Network) networked virtual environment and discuss the results of an informal evaluation experiment.

Object-driven interactions have various applications, especially in team activities and sports like volleyball, football, tennis, etc. We define a multi-party virtual world as an environment which includes multiple avatars and computer agents. Latency is one of the primary issues while building a multi-party setup. Delays due to latency cause inconsistencies and unnecessary halts resulting in unrealistic experience in VR. For interactions using objects, it becomes vital to handle the latencies. We present a time scaling algorithm for latency management and synchronization between multiple avatars, while they interact using objects in the VR environment. We propose two time scaling schemes—source and target based. Both these methodologies are dependent on which user’s virtual view (source or target) needs to be uniform. Incorporating the presented time scaling schemes, we develop a comprehensive multi-party virtual reality platform to simulate applications which involves interactions between agents and avatars using objects. We adopt a client–server architecture for building our multiuser VR platform which induces flexibility of adding multiple avatars and/or agents to the application. The modular nature of our proposed system enables extension to different VR applications that encompass communication using objects. We exemplify our framework by developing a multi-player VR volleyball game which employs the features of our proposed scheme. We demonstrate the significance of time scaling in the object-driven interactive multi-party VR framework through a comparative user study. We also evaluate the immersion experience in our multi-player volleyball game by a user survey. National Research Foundation (NRF) This research is supported by the program titled Realistic Immersion with Virtual Humans, a collaboration between Nanyang Technological University (NTU) Singapore and Institute of Software, Chinese Academy of Sciences (ISCAS) in Beijing, China. The program is supported by the National Research Foundation, Prime Minister’s Office, Singapore, under its NRF-NSFC Joint Research Grant Call (Data Science).

Articulated hand pose estimation is one of core technologies in human-computer interaction. Despite the recent progress, most existing methods still cannot achieve satisfactory performance, partly due to the difficulty of the embedded high-dimensional nonlinear regression problem. Most existing data-driven methods directly regress 3D hand pose from 2D depth image, which cannot fully utilize the depth information. In this paper, we propose a novel multi-view convolutional neural network (CNN)-based approach for 3D hand pose estimation. To better exploit 3D information in the depth image, we project the point cloud generated from the query depth image onto multiple views of two projection settings and integrate them for more robust estimation. Multi-view CNNs are trained to learn the mapping from projected images to heat-maps, which reflect probability distributions of joints on each view. These multi-view heat-maps are then fused to estimate the optimal 3D hand pose with learned pose priors, and the unreliable information in multi-view heat-maps is suppressed using a view selection method. Experimental results show that the proposed method is superior to the state-of-the-art methods on two challenging data sets. Furthermore, a cross-data set experiment also validates that our proposed approach has good generalization ability. NRF (Natl Research Foundation, S’pore) MOE (Min. of Education, S’pore) Accepted version

When building robotic hands, researchers are always face with two main issues of how to make robotic hands look human-like and how to make robotic hands function like real hands. Most existing solutions solve these issues by manually modelling the robotic hand [10-18]. However, the design processes are long, and it is difficult to duplicate the geometry shape of a human hand. To solve these two issues, this paper presents a simple and effective method that combines 3D printing and digitization techniques to create a 3D printable cable-driven robotic hand from scanning a physical hand. The method involves segmenting the 3D scanned hand model, adding joints, and converting it into a 3D printable model. Comparing to other robotic solutions, our solution retains more than 90% geometry information of a human hand1, which is attained from 3D scanning. Our modelling progress takes around 15 minutes that include 10 minutes of 3D scanning and five minutes for changing the scanned model to an articulated model by running our algorithm. Compared to other articulated modelling solutions [19, 20], our solution is compatible with an actuation system which provides our robotic hand with the ability to mimic different gestures. We have also developed a way of representing hand skeletons based on the hand anthropometric. As a proof of concept, we demonstrate our robotic hand's performance in the grasping experiments. NRF (Natl Research Foundation, S’pore) Accepted version

The positions of the hand joints are important high-level features for hand-based human-computer interaction. We present a novel method to predict the 3-D joint positions from the depth images and the parsed hand parts obtained with a pretrained classifier. The hand parts are utilized as the additional cue to resolve the multimodal predictions produced by the previous regression-based method without increasing the computational cost significantly. In addition, we further enforce the hand motion constraints to fuse the per-pixel prediction results. The posterior distribution of the joints is formulated as a weighted product of experts model based on the individual pixel predictions, which is maximized via the expectation–maximization algorithm on a learned low-dimensional space of the hand joint parameters. The experimental results show the proposed method improves the prediction accuracy considerably compared with the rivals that also regress for the joint locations from the depth images. Especially, we show that the regressor learned on synthesized dataset also gives accurate prediction on real-world depth images by enforcing the hand part correlations despite their discrepancies.

Negative symptoms of schizophrenia significantly affect the daily functioning of patients, especially movement and expressive gestures. The diagnosis of such symptoms is often difficult and require the expertise of a trained clinician. Apart from these subjective methods, there is little research on developing objective methods to quantify the symptoms. Therefore, we explore body movement signals as objective measures of negative symptoms. Specifically, we extract the signals from video recordings of patients being interviewed. We analysed the interviews of 69 paid participants (46 patients and 23 healthy controls) in this study. Correlation between movement signals (linear and angular speeds of upper limbs and head, acceleration and gesture angles) and subjective ratings (assigned during same interview) from the NSA-16 scale were calculated. As hypothesized, the movement signals correlated strongly with the movement impairment aspect of the NSA-16 questionnaire. Also, not quite surprisingly, strong correlations were obtained between the movement signals and speech items of NSA-16, indicating lack of associated gestures in patients during speech. These subjective ratings could also be reasonably predicted from the objective signals with an accuracy of 61-78 % using machine-learning algorithms with leave-one-out cross-validation technique. Furthermore, these objective measures can be reliably utilized to distinguish between the patient and healthy groups, as supervised learning methods can classify the two groups with 74-87 % accuracy. NRF (Natl Research Foundation, S’pore) NMRC (Natl Medical Research Council, S’pore) Accepted version

An accurate and efficient clothoid approximation approach is presented in this paper using Bézier curves based on the minimization of curvature profile difference. Compared with existing methods, the proposed approach is able to guarantee higher order geometric continuity with smaller approximation error in terms of position, orientation, and curvature. The approximation scheme takes place in three stages. First, a subset of clothoids with specific winding angle constraints referred to as elementary clothoids is approximated using quintic Bézier curves. Then, a basic clothoid defined in the first quadrant is formulated, which is composed of a series of transformed elementary clothoids. An adaptive sampling stra-tegy is applied to ensure that the resulting Bézier segments are computed within a specified accuracy and all the required information can be obtained offline and stored in a lookup table. Finally, a general clothoid with arbitrary parameters can be conveniently approximated based on the lookup table through appropriate geometric transformations. A comparison with the recent circular interpolation and rational Bézier curve based approximation shows that the proposed approach is able to achieve equivalent or greater computational efficiency in most scenarios. Accepted version

Vision-based hand pose estimation is important in human-computer interaction. While many recent works focus on full degree-of-freedom hand pose estimation, robust estimation of global hand pose remains a challenging problem. This paper presents a novel algorithm to optimize the leaf weights in a Hough forest to assist global hand pose estimation with a single depth camera. Different from traditional Hough forest, we propose to learn the vote weights stored at the leaf nodes of a forest in a principled way to minimize average pose prediction error, so that ambiguous votes are largely suppressed during prediction fusion. Experiments show that the proposed method largely improves pose estimation accuracy with optimized leaf weights on both synthesis and real datasets and performs favorably compared to state-of-the-art convolutional neural network-based methods. On real-world depth videos, the proposed method demonstrates improved robustness compared to several other recent hand tracking systems from both industry and academy. Moreover, we utilize the proposed method to build virtual/augmented reality applications to allow users to manipulate and examine virtual objects with bare hands. MOE (Min. of Education, S’pore)

This paper discusses the role of sensors and actuators in Human–Computer Interaction and Virtual Reality. We first introduce the need of sensory input and output and the user senses stimulation. We then present two experiences of immersive games with a comparison of the techniques 20 years ago and today and also discuss the importance of vision-based hand tracking. The main part of the paper is dedicated to haptic and tactile feedbacks with 4 different case studies: haptic feedback using a haptic workstation, a wearable system for mobility improvement of visually impaired people, a system for enhancing pilot performance, and actuators to feel the wind. Finally, we survey a few smart jackets with sensors and actuators. NRF (Natl Research Foundation, S’pore)

With the quick development and popularity of computers, computer-generated signals have drastically invaded into our daily lives. Screen content image is a typical example, since it also includes graphic and textual images as components as compared with natural scene images which have been deeply explored, and thus screen content image has posed novel challenges to current researches, such as compression, transmission, display, quality assessment, and more. In this paper, we focus our attention on evaluating the quality of screen content images based on the analysis of structural variation, which is caused by compression, transmission, and more. We classify structures into global and local structures, which correspond to basic and detailed perceptions of humans, respectively. The characteristics of graphic and textual images, e.g., limited color variations, and the human visual system are taken into consideration. Based on these concerns, we systematically combine the measurements of variations in the above-stated two types of structures to yield the final quality estimation of screen content images. Thorough experiments are conducted on three screen content image quality databases, in which the images are corrupted during capturing, compression, transmission, etc. Results demonstrate the superiority of our proposed quality model as compared with state-of-the-art relevant methods. MOE (Min. of Education, S’pore)

User ratings are the essence of recommender systems in e-commerce. Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of such systems and contribute to the well-known data sparsity and cold start problems. This article proposes a new information source for recommender systems, called prior ratings. Prior ratings are based on users’ experiences of virtual products in a mediated environment, and they can be submitted prior to purchase. A conceptual model of prior ratings is proposed, integrating the environmental factor presence whose effects on product evaluation have not been studied previously. A user study conducted in website and virtual store modalities demonstrates the validity of the conceptual model, in that users are more willing and confident to provide prior ratings in virtual environments. A method is proposed to show how to leverage prior ratings in collaborative filtering. Experimental results indicate the effectiveness of prior ratings in improving predictive performance.

In this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence-arousal-dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hidden emotions due to social conventions. To achieve such a higher degree of facial expression in a generic way, we propose a new model for mapping the valence-arousal-dominance emotion model onto a set of 12 scalar facial part actions built mostly by combining pairs of antagonist action units from the Facial Action Coding System. The proposed linear model can automatically drive a large number of autonomous virtual humans or support the interactive design of complex facial expressions over time. By design, our approach produces symmetric facial expressions, as expected for most of the emotional spectrum. However, more complex ambivalent feelings can be produced when differing emotions are applied on the left and right sides of the face. We conducted an experiment on static images produced by our approach to compare the expressive power of symmetric and asymmetric facial expressions for a set of eight basic and complex emotions. Results confirm both the pertinence of our general mapping for expressing basic emotions and the significant improvement brought by asymmetry for expressing ambivalent feelings. Copyright © 2013 John Wiley & Sons, Ltd.

The recent proliferation of virtual reality (VR) technology applications in the autism therapy to promote learning and positive behavior among such children has produced optimistic results in developing a variety of skills and abilities in them. Dolphin-assisted therapy has also become a topic of public and research interest for autism intervention and treatment. This paper will present an innovative design and development of a Virtual Dolphinarium for potential autism intervention. Instead of emulating the swimming with dolphins, our virtual dolphin interaction program will allow children with autism to act as dolphin trainers at the poolside and to learn (nonverbal) communication through hand gestures with the virtual dolphins. Immersive visualization and gesture-based interaction are implemented to engage children with autism within an immersive room equipped with a curved screen spanning a 320(°) and a high-end five-panel projection system. This paper will also report a pilot study to establish trial protocol of autism screening to explore the participants' readiness for the virtual dolphin interaction. This research will have two potential benefits in the sense of helping children with autism and protecting the endangered species.

Articulated hand pose estimation plays an important role in human-computer interaction. Despite the recent progress, the accuracy of existing methods is still not satisfactory, partially due to the difficulty of embedded high-dimensional and non-linear regression problem. Different from the existing discriminative methods that regress for the hand pose with a single depth image, we propose to first project the query depth image onto three orthogonal planes and utilize these multi-view projections to regress for 2D heat-maps which estimate the joint positions on each plane. These multi-view heat-maps are then fused to produce final 3D hand pose estimation with learned pose priors. Experiments show that the proposed method largely outperforms state-of-the-art on a challenging dataset. Moreover, a cross-dataset experiment also demonstrates the good generalization ability of the proposed method., Comment: 9 pages, 9 figures, published at Computer Vision and Pattern Recognition (CVPR) 2016

Humans are not rational beings. Deviations from rationality in human thinking are currently well documented [25] as non-reducible to rational pursuit of egoistic benefit or its occasional distortion with temporary emotional excitation, as it is often assumed. This occurs not only outside conceptual reasoning or rational goal realization but also subconsciously and often in certainty that they did not and could not take place 'in my case'. Non-rationality can no longer be perceived as a rare affective abnormality in otherwise rational thinking, but as a systemic, permanent quality, 'a design feature' of human cognition. While social psychology has systematically addressed non-rationality of human cognition (including its non-emotional aspects) for decades [63]. It is not the case for computer science, despite obvious relevance for individual and group behavior modeling. This paper proposes brief survey of work in computational disciplines related to human-like non-rationality modeling including: Social Signal Processing, Cognitive Architectures, Affective Computing, Human-Like Agents and Normative Multi-agent Systems. It attempts to establish a common terminology and conceptual frame for this extremely interdisciplinary issue, reveal assumptions about non-rationality underlying the discussed models and disciplines, their current limitations and potential in contributing to solution. Finally, it also presents ideas concerning possible directions of development, hopefully contributing to solution of this challenging issue.

This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the problem of dynamic navigation of individual and groups of characters in the multi-agent system. A natural interface is designed for the interaction between the user and the virtual characters, as well as the virtual environment, based on gesture recognition. To evaluate the efficiency of the dynamic navigation method, performance results are provided. The presented system has the potential to be used in the training and evaluation of emergency evacuation and other real-time applications of crowd simulation with interaction.

Computer-based simulations of human hip joints generally include investigating contacts happening among soft or hard tissues during hip movement. In many cases, hip movement is approximated as rotation about an estimated hip center. In this paper, we investigate the effect of different methods used for estimating hip joint center of rotation on the results acquired from hip simulation. For this reason, we use three dimensional models of hip tissues reconstructed from MRI datasets of 10 subjects, and estimate their center of rotation by applying five different methods (including both predictive and functional approaches). Then, we calculate the amount of angular and radial penetrations that happen among three dimensional meshes of cartilages, labrum, and femur bone, when hip is rotating about different estimated centers of rotation. The results indicate that hip simulation can be highly affected by the method used for estimating hip center of rotation. However, under some conditions (e.g. when Adduction or External Rotation are considered) we can expect to have a more robust simulation. In addition, it was observed that applying some methods (e.g. the predictive approach based on acetabulum) may result in less robust simulation, comparing to the other methods.

We present a visualization framework for exploring and analyzing data sets from biomechanical and neuromuscular simulations. These data sets describe versatile information related to the different stages of a motion analysis. In studying these data using a 3D visualization approach, interactive exploring is important, especially for supporting spatial analysis. Moreover, as these data contain many various but related elements, numerical analysis of neuromuscular simulations is complicated. Visualization techniques enhance the analysis process, thus improving the effectiveness of the experiments. Our approach allows convenient definitions of relationships between numerical data sets and 3D objects. Scientific simulation data sets appropriate for this style of analysis are present everywhere motion analysis is performed and are predominant in many clinical works. In this paper, we outline the functionalities of the framework as well as applications embedded within the OpenSim simulation platform. These functionalities form an effective approach specifically designed for the investigation of neuromuscular simulations. This claim is supported by evaluation experiments where the framework was used to analyze gaits and crouch motions.

In the recent years, usage of the third-person perspective (3PP) in virtual training methods has become increasingly viable and despite the growing interest in virtual reality and graphics underlying third-person perspective usage, not many studies have systematically looked at the dynamics and differences between the third and first-person perspectives (1PPs). The current study was designed to quantify the differences between the effects induced by training participants to the third-person and first-person perspectives in a ball catching task. Our results show that for a certain trajectory of the stimulus, the performance of the participants post3PP training is similar to their performance postnormal perspective training. Performance post1PP training varies significantly from both 3PP and the normal perspective.

This paper presents a novel concept: a graphical representation of human emotion extracted from text sentences. The major contributions of this paper are the following. First, we present a pipeline that extracts, processes, and renders emotion of 3D virtual human (VH). The extraction of emotion is based on data mining statistic of large cyberspace databases. Second, we propose methods to optimize this computational pipeline so that real-time virtual reality rendering can be achieved on common PCs. Third, we use the Poisson distribution to transfer database extracted lexical and language parameters into coherent intensities of valence and arousal—parameters of Russell’s circumplex model of emotion. The last contribution is a practical color interpretation of emotion that influences the emotional aspect of rendered VHs. To test our method’s efficiency, computational statistics related to classical or untypical cases of emotion are provided. In order to evaluate our approach, we applied our method to diverse areas such as cyberspace forums, comics, and theater dialogs.

In this paper, we propose to increase realism and complexity of haptic applications for Virtual Reality and CAD applications. To achieve this goal, we use a Haptic Workstation which allows to acquire the posture and position of both hands, and to apply forces on the fingertips and wrists. We propose techniques to calibrate and improve the comfort of these kinds of devices in order to integrate them into Virtual Environments. However, a two-handed haptic device does not present only advantages. Indeed, It is much more complicated to compute forces on two hand models, than on a single point or fingertip. For this reason, we propose a framework to optimize this computation.

Real-time crowd motion planning requires fast, realistic methods for path planning as well as obstacle avoidance. In a previous work (Morini et al. in Cyberworlds International Conference, pp. 144–151, 2007), we introduced a hybrid architecture to handle real-time motion planning of thousands of pedestrians. In this article, we present an extended version of our architecture, introducing two new features: an improved short-term collision avoidance algorithm, and simple efficient group behavior for crowds. Our approach allows the use of several motion planning algorithms of different precision for regions of varied interest. Pedestrian motion continuity is ensured when switching between such algorithms. To assess our architecture, several performance tests have been conducted, as well as a subjective test demonstrating the impact of using groups. Our results show that the architecture can plan motion in real time for several thousands of characters.

In this paper, we introduce a european re- search pro ject, interactive media with personal networked devices (INTERMEDIA) in which we seek to progress beyond home and device-centric convergence toward truly user-centric convergence of multimedia. Our vision is The User as Multimedia Central: the user as the point at which multimedia services and the means for interact- ing with them converge. This paper proposes the main research goals so that users can be provided with a per- sonalized interface and content independently of physical networked devices, space and time. As a case study, we describe an indoors, mobile mixed reality guide system: Chloe@University. With a see-through head-mounted dis- play (HMD) connected to a small wearable computing device, Chloe@University provides users with an efficient way of guiding in a building. A 3D virtual character in front of the user guides him/her to the required destina- tion.