Search

This article devises a photograph-based monitoring model to estimate the real-time PM2.5 concentrations, overcoming currently popular electrochemical sensor-based PM2.5 monitoring methods’ shortcomings such as low-density spatial distribution and time delay. Combining the proposed monitoring model, the photographs taken by various camera devices (e.g., surveillance camera, automobile data recorder, and mobile phone) can widely monitor PM2.5 concentration in megacities. This is beneficial to offering helpful decision-making information for atmospheric forecast and control, thus reducing the epidemic of COVID-19. To specify, the proposed model fuses Information Abundance measurement and Wide and Deep learning, dubbed as IAWD, for PM2.5 monitoring. First, our model extracts two categories of features in a newly proposed DS transform space to measure the information abundance (IA) of a given photograph since the growth of PM2.5 concentration decreases its IA. Second, to simultaneously possess the advantages of memorization and generalization, a new wide and deep neural network is devised to learn a nonlinear mapping between the above-mentioned extracted features and the groundtruth PM2.5 concentration. Experiments on two recently established datasets totally including more than 100 000 photographs demonstrate the effectiveness of our extracted features and the superiority of our proposed IAWD model as compared to state-of-the-art relevant computing techniques.

This open access book details the relationship between the artist and their created works, using tools such as information technology, computer environments, and interactive devices, for a range of information sources and application domains. This has produced new kinds of created works which can be viewed, explored, and interacted with, either as an installation or via a virtual environment such as the Internet. These processes generate new dimensions of understanding and experience for both the artist and the public’s relationships with the works that are produced. This has raised a variety of interdisciplinary opportunities and issues, and these are examined. The symbiotic relationship between artistic works and the cultural context in which they are produced is reviewed. Technology can provide continuity by making traditional methods and techniques more efficient and effective. It can also provide discontinuity by opening up new perspectives and paradigms. This can generate new ideas, and produce a greater understanding of artistic processes and how they are implemented in practice. Tools have been used from the earliest times to create and modify artistic works. For example, naturally occurring pigments have been used for cave paintings. What has been created provides insight into the cultural context and social environment at the time of creation. There is an interplay between the goal of the creator, the selection and use of appropriate tools, and the materials and representations chosen. Technology, Design and the Arts - Opportunities and Challenges is relevant for artists and technologists and those engaged in interdisciplinary research and development at the boundaries between these disciplines.

Reference EPFL-ARTICLE-221843doi:10.1007/s00371-016-1271-7View record in Web of Science Record created on 2016-10-18, modified on 2016-10-18

Nonverbal communication is an important aspect of real-life face-to-face interaction and one of the most efficient ways to convey emotions, therefore users should be provided the means to replicate it in the virtual world. Because articulated embodiments are well suited to provide body communication in virtual environments, this paper first reviews some of the advantages and disadvantages of complex embodiments. After a brief introduction to nonverbal communication theories, we present our solution, taking into account the practical limitations of input devices and social science aspects. We introduce our sample of actions and implementation using our VLNET (Virtual Life Network) networked virtual environment and discuss the results of an informal evaluation experiment.

Hand pose tracking and gesture recognition are useful for human-computer interaction, while a major problem is the lack of discriminative features for compact hand representation. We present a robust hand parsing scheme to extract a high-level description of the hand from the depth image. A novel distance-adaptive selection method is proposed to get more discriminative depth-context features. Besides, we propose a Superpixel-Markov Random Field (SMRF) parsing scheme to enforce the spatial smoothness and the label co-occurrence prior to remove the misclassified regions. Compared to pixel-level filtering, the SMRF scheme is more suitable to model the misclassified regions. By fusing the temporal constraints, its performance can be further improved. Overall, the proposed hand parsing scheme is accurate and efficient. The tests on synthesized dataset show it gives much higher accuracy for single-frame parsing and enhanced robustness for continuous sequence parsing compared to benchmarks. The tests on real-world depth images of the hand and human body show the robustness to complex hand configurations of our method and its generalization power to different kinds of articulated objects.

Providing high quality recommendations is important for e-commerce systems to assist users in making effective selection decisions from a plethora of choices. Collaborative filtering is a widely accepted technique to generate recommendations based on the ratings of like-minded users. However, it suffers from several inherent issues such as data sparsity and cold start. To address these problems, we propose a novel method called ''Merge'' to incorporate social trust information (i.e., trusted neighbors explicitly specified by users) in providing recommendations. Specifically, ratings of a user's trusted neighbors are merged to complement and represent the preferences of the user and to find other users with similar preferences (i.e., similar users). In addition, the quality of merged ratings is measured by the confidence considering the number of ratings and the ratio of conflicts between positive and negative opinions. Further, the rating confidence is incorporated into the computation of user similarity. The prediction for a given item is generated by aggregating the ratings of similar users. Experimental results based on three real-world data sets demonstrate that our method outperforms other counterparts both in terms of accuracy and coverage.

In this paper, we propose a method to achieve effective facial emotional expressivity for embodied conversational agents by considering two types of asymmetry when exploiting the valence-arousal-dominance representation of emotions. Indeed, the asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hidden emotions due to social conventions. To achieve such a higher degree of facial expression in a generic way, we propose a new model for mapping the valence-arousal-dominance emotion model onto a set of 12 scalar facial part actions built mostly by combining pairs of antagonist action units from the Facial Action Coding System. The proposed linear model can automatically drive a large number of autonomous virtual humans or support the interactive design of complex facial expressions over time. By design, our approach produces symmetric facial expressions, as expected for most of the emotional spectrum. However, more complex ambivalent feelings can be produced when differing emotions are applied on the left and right sides of the face. We conducted an experiment on static images produced by our approach to compare the expressive power of symmetric and asymmetric facial expressions for a set of eight basic and complex emotions. Results confirm both the pertinence of our general mapping for expressing basic emotions and the significant improvement brought by asymmetry for expressing ambivalent feelings. Copyright © 2013 John Wiley & Sons, Ltd.

In this paper we present a novel vision-based markerless hand pose estimation scheme with the input of depth image sequences. The proposed scheme exploits both temporal constraints and spatial features of the input sequence, and focuses on hand parsing and 3D fingertip localization for hand pose estimation. The hand parsing algorithm incorporates a novel spatial-temporal feature into a Bayesian inference framework to assign the correct label to each image pixel. The 3D fingertip localization algorithm adapts a recently developed geodesic extrema extraction method to fingertip detection with the hand parsing algorithm, a novel path-reweighting method and K-means clustering in metric space. The detected 3D fingertip locations are finally used for hand pose estimation with an inverse kinematics solver. Quantitative experiments on synthetic data show the proposed hand pose estimation scheme can accurately capture the natural hand motion. A simulated water-oscillator application is also built to demonstrate the effectiveness of the proposed method in human-computer interaction scenarios.

The recent proliferation of virtual reality (VR) technology applications in the autism therapy to promote learning and positive behavior among such children has produced optimistic results in developing a variety of skills and abilities in them. Dolphin-assisted therapy has also become a topic of public and research interest for autism intervention and treatment. This paper will present an innovative design and development of a Virtual Dolphinarium for potential autism intervention. Instead of emulating the swimming with dolphins, our virtual dolphin interaction program will allow children with autism to act as dolphin trainers at the poolside and to learn (nonverbal) communication through hand gestures with the virtual dolphins. Immersive visualization and gesture-based interaction are implemented to engage children with autism within an immersive room equipped with a curved screen spanning a 320(°) and a high-end five-panel projection system. This paper will also report a pilot study to establish trial protocol of autism screening to explore the participants' readiness for the virtual dolphin interaction. This research will have two potential benefits in the sense of helping children with autism and protecting the endangered species.

This paper presents an immersive volleyball game, where a player plays not only against virtual opponents but also with support on his/her side of virtual teammates. This volleyball game has been implemented for several 3D displays such as a stereoscopic display, an autostereoscopic display, Oculus Rift glasses, and a 320o Immersive Room. In this paper, we also propose a user study of the relations between virtual humans and the sense of presence in the different 3D displays. We particularly study how surrounding virtual humans affect the sense of presence. Results show that users more significantly perceived spatial presence of virtual environment and social presence of virtual humans with the Oculus Rift and the Immersive Room.

This is the first book to describe how Autonomous Virtual Humans and Social Robots can interact with real people, be aware of the environment around them, and react to various situations. Researchers from around the world present the main techniques for tracking and analysing humans and their behaviour and contemplate the potential for these virtual humans and robots to replace or stand in for their human counterparts, tackling areas such as awareness and reactions to real world stimuli and using the same modalities as humans do: verbal and body gestures, facial expressions and gaze to aid seamless human-computer interaction (HCI). The research presented in this volume is split into three sections: User Understanding through Multisensory Perception: deals with the analysis and recognition of a given situation or stimuli, addressing issues of facial recognition, body gestures and sound localization. Facial and Body Modelling Animation: presents the methods used in modelling and animating faces and bodies to generate realistic motion. Modelling Human Behaviours: presents the behavioural aspects of virtual humans and social robots when interacting and reacting to real humans and each other. Context Aware Human-Robot and Human-Agent Interaction would be of great use to students, academics and industry specialists in areas like Robotics, HCI, and Computer Graphics.

Traversing dense crowds can be challenging, but it is especially difficult in a virtual environment where the user is limited to simple input devices. Predicting near-future user movements with a hidden Markov model allows nearby autonomous agents to react politely, that is, to specifically avoid impeding the movement of the user. This predictive model and simple avoidance scheme are tested by six subjects in 360 short interactive games and by a 10-participant two-alternative forced choice questionnaire. Polite agents are found to significantly improve the player's ability to navigate quickly and without collision, and the resulting character movements showed increased naturalness for two of the three game scenarios. Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the problem of dynamic navigation of individual and groups of characters in the multi-agent system. A natural interface is designed for the interaction between the user and the virtual characters, as well as the virtual environment, based on gesture recognition. To evaluate the efficiency of the dynamic navigation method, performance results are provided. The presented system has the potential to be used in the training and evaluation of emergency evacuation and other real-time applications of crowd simulation with interaction.

We present a visualization framework for exploring and analyzing data sets from biomechanical and neuromuscular simulations. These data sets describe versatile information related to the different stages of a motion analysis. In studying these data using a 3D visualization approach, interactive exploring is important, especially for supporting spatial analysis. Moreover, as these data contain many various but related elements, numerical analysis of neuromuscular simulations is complicated. Visualization techniques enhance the analysis process, thus improving the effectiveness of the experiments. Our approach allows convenient definitions of relationships between numerical data sets and 3D objects. Scientific simulation data sets appropriate for this style of analysis are present everywhere motion analysis is performed and are predominant in many clinical works. In this paper, we outline the functionalities of the framework as well as applications embedded within the OpenSim simulation platform. These functionalities form an effective approach specifically designed for the investigation of neuromuscular simulations. This claim is supported by evaluation experiments where the framework was used to analyze gaits and crouch motions.

In the recent years, usage of the third-person perspective (3PP) in virtual training methods has become increasingly viable and despite the growing interest in virtual reality and graphics underlying third-person perspective usage, not many studies have systematically looked at the dynamics and differences between the third and first-person perspectives (1PPs). The current study was designed to quantify the differences between the effects induced by training participants to the third-person and first-person perspectives in a ball catching task. Our results show that for a certain trajectory of the stimulus, the performance of the participants post3PP training is similar to their performance postnormal perspective training. Performance post1PP training varies significantly from both 3PP and the normal perspective.

This paper presents a novel concept: a graphical representation of human emotion extracted from text sentences. The major contributions of this paper are the following. First, we present a pipeline that extracts, processes, and renders emotion of 3D virtual human (VH). The extraction of emotion is based on data mining statistic of large cyberspace databases. Second, we propose methods to optimize this computational pipeline so that real-time virtual reality rendering can be achieved on common PCs. Third, we use the Poisson distribution to transfer database extracted lexical and language parameters into coherent intensities of valence and arousal—parameters of Russell’s circumplex model of emotion. The last contribution is a practical color interpretation of emotion that influences the emotional aspect of rendered VHs. To test our method’s efficiency, computational statistics related to classical or untypical cases of emotion are provided. In order to evaluate our approach, we applied our method to diverse areas such as cyberspace forums, comics, and theater dialogs.

In this paper, we introduce a european re- search pro ject, interactive media with personal networked devices (INTERMEDIA) in which we seek to progress beyond home and device-centric convergence toward truly user-centric convergence of multimedia. Our vision is The User as Multimedia Central: the user as the point at which multimedia services and the means for interact- ing with them converge. This paper proposes the main research goals so that users can be provided with a per- sonalized interface and content independently of physical networked devices, space and time. As a case study, we describe an indoors, mobile mixed reality guide system: Chloe@University. With a see-through head-mounted dis- play (HMD) connected to a small wearable computing device, Chloe@University provides users with an efficient way of guiding in a building. A 3D virtual character in front of the user guides him/her to the required destina- tion.

This paper introduces a framework for real-time simulation and rendering of crowds navigating in a virtual environment. The solution first consists in a specific environment preprocessing technique giving rise to navigation graphs, which are then used by the navigation and simulation tasks. Second, navigation planning interactively provides various solutions to the user queries, allowing to spread a crowd by individualizing trajectories. A scalable simulation model enables the management of large crowds, while saving computation time for rendering tasks. Pedestrian graphical models are divided into three rendering fidelities ranging from billboards to dynamic meshes, allowing close-up views of detailed digital actors with a large variety of locomotion animations. Examples illustrate our method in several environments with crowds of up to 35000 pedestrians with real-time performance. Copyright © 2006 John Wiley and Sons, Ltd.

Expressive facial animation synthesis of human like characters has had many approaches with good results. MPEG-4 standard has functioned as the basis of many of those approaches. In this paper we would like to lay out the knowledge of some of those approaches inside an ontology in order to support the modeling of emotional facial animation in virtual humans (VH). Inside this ontology we will present MPEG-4 facial animation concepts and its relationship with emotion through expression profiles that utilize psychological models of emotions. The ontology allows storing, indexing and retrieving prerecorded synthetic facial animations that can express a given emotion. Also this ontology can be used a refined knowledge base in regards to the emotional facial animation creation. This ontology is made using Web Ontology Language and the results are presented as answered queries. Copyright © 2006 John Wiley and Sons, Ltd.

This paper presents an original model with methodologies that integrate in a novel way different types of an autonomous virtual agent's perception in a virtual environment. Our first new approach permits the coherent management of the shared virtual environment for the simulations of an autonomous virtual agent (AVA). Our second approach allows the prediction or the estimation of both the orientation and the attention of an AVA in a virtual environment. By means of a test application with a 'virtual goalkeeper', we demonstrate the speed and the robustness of our technique

In this paper, we present research results and future challenges in creating realistic and believable Virtual Humans. To realize these modeling goals, real-time realistic representation is essential, but we also need interactive and perceptive Virtual Humans to populate the Virtual Worlds. Three levels of modeling should be considered to create these believable Virtual Humans: 1) realistic appearance modeling, 2) realistic, smooth and flexible motion modeling, and 3) realistic high-level behaviors modeling. At first, the issues of creating virtual humans with better skeleton and realistic deformable bodies are illustrated. To give a level of believable behavior, challenges are laid on generating on the fly flexible motion and complex behaviours of Virtual Humans inside their environments using a realistic perception of the environment. Interactivity and group behaviours are also important parameters to create believable Virtual Humans which have challenges in creating believable relationship between real and virtual humans based on emotion and personality, and simulating realistic and believable behaviors of groups and crowds. Finally, issues in generating realistic virtual clothed and haired people are presented.

The ability to build and control exposure through a seamless synergy of interaction and narration is a strongly required feature of a new type of immersive-VR training and therapy system. This paper presents a practical approach for immersive-VR training and therapy applications based on interactive storytelling. It provides detailed description of a working implementation of the Interactive Narration Space (INS); this approach combines and satisfies both interaction and narration requirements through the use of high-level social interaction. By introducing the Social Channel, we aim at minimizing the contradictions between control over the story required by the trainer/therapist and interaction required by the trainee/patient. These concepts and their practical realization have been investigated in the context of emergency-situation training and psychotherapeutic exposure, and could validate the usability of mediated interaction with a virtual assistant.

Our approach is based on the multi-sensory integration of the standard theory of neuroscience, where signals of a single object coming from distinct sensory systems are combined. The acquisition steps of signals, filtering, selection and simplification intervening before proprioception, active and predictive perception are integrated into virtual sensors and a virtual environment. We will focus on two aspects: 1) the assignment problem: determining which sensory stimuli belong to the same virtual object and (2) the sensory recoding problem: recoding signals in a common format before combining them. We have developed three novel methodologies to map the information coming from the virtual sensors of vision, audition and touch as well as that of the virtual environment in the form of a 'cognitive map'. Copyright © 2004 John Wiley and Sons, Ltd.

In order to guarantee an effective and punctual medical intervention to injured people involved in health emergency situations, where usually both professional and non-professional health operators are involved, a fast and accurate treatment has to be carried out. In case of catastrophic or very critical situations, non-professional operators who did not receive proper training (volunteers are among them) could be affected by psychological inhibitions. Their performances could slow down in such way that would affect the quality of the treatment and increase both direct and indirect costs. Our virtual reality system that is currently in use at the health care emergency center of San Martino Hospital in Genoa, Italy, has been designed and developed to check health emergency operators' capabilities to adopt correct decision-making procedures, to make optimal use of new technological equipment and to overcome psychological barriers. Our system is composed of (1) a high-end simulation PC, whose main functions are execution of the main software module, rendering of 3D scenes in stereo mode, rendering of sound, and control of data transmission from/to VR devices; (2) a low-end control PC, which controls the VR simulation running on the simulation PC, manages medical emergency simulation scenarios, introduces unexpected events to the simulation and controls the simulation difficulty level; (3) a magnetic-based motion tracking device used for head and hand tracking; (4) a wireless pair of shutter glasses together with a cathode ray tube wall projector; and (5) a high-end surround sound system. The expected benefits have been verified through the design and implementation of controlled clinical trials.

While virtual crowds are becoming common in non-real-time applications, the real-time domain is still relatively unexplored. In this paper we discuss the challenges involved in creating such simulations, especially the need to efficiently manage variety. We introduce the concept of levels of variety, then present our work on crowd behaviour simulation aimed at interactive real-time applications such as computer games or virtual environments. We define a modular behavioural architecture of a multi-agent system allowing autonomous and scripted behaviour of agents supporting variety. Finally we show applications of our system in a virtual reality training system and virtual heritage reconstruction

Real-time 3D graphics are being extensively used to build interactive virtual environments for a number of different applications. In many situations, virtual objects are required to exhibit complex behaviors, and the use of a versatile behavioral programming methodology is required. This paper presents a feature modeling approach to define behavioral information of virtual objects, giving special attention to their capabilities of interaction with virtual human actors. The proposed method is based on a complete definition and representation of interactive objects, which are called smart objects. This representation is based on the description of interaction features: parts, movements, graspable sites, functionalities, purposes, etc., including a behavioral automaton based on scripts and graphical state machines

Trust has been extensively studied and its effectiveness demonstrated in recommender systems. Due to the lack of explicit trust information in most systems, many trust metric approaches have been proposed to infer implicit trust from user ratings. However, previous works have not compared these different approaches, and oftentimes focus only on the performance of predictive item ratings. In this paper, we first analyse five kinds of trust metrics in light of the properties of trust. We conduct an empirical study to explore the ability of trust metrics to distinguish explicit trust from implicit trust and to generate accurate predictions. Experimental results on two real-world data sets show that existing trust metrics cannot provide satisfying performance, and indicate that future metrics should be designed more carefully.

This Chapter discusses the methods involved in the generation of large crowds of Virtual Humans in environments like cities. We focus on the geometric aspects of these methods in the different steps involved: scaler, simulator, renderer, path planner, and behaviour handler. We emphasize the application of these methods to the field of Cultural Heritage, recreating old cities with population living their life. In particular, we present examples from Pompeii and discuss the interaction between the environment and the behaviour of the Romans.

In neurofeedback, brain waves are transformed into sounds or music, graphics, and other representations, to provide real-time information on ongoing waves and patterns in the brain. Here we present various forms of neurofeedback, including sonification, sonification in combination with visualization, and at last, immersive neurofeedback, where auditory and visual feedback is provided in a multi-sided immersive environment in which participants are completely surrounded by virtual imagery and 3D sound. Neural feedback may potentially improve the user’s (or patient’s) ability to control brain activity, the diagnosis of medical conditions, and the rehabilitation of neurological or psychiatric disorders. Several psychological and medical studies have confirmed that virtual immersive activity is enjoyable, stimulating, and can have a healing effect. As an illustration, neurofeedback is generated from electroencephalograms (EEG) of Alzheimer’s disease (AD) patients and healthy subjects. The auditory, visual, and immersive representations of Alzheimer’s EEG differ substantially from healthy EEG, potentially yielding novel diagnostic tools. Moreover, such alternative representations of AD EEG are natural and intuitive, and hence easily accessible to laymen (AD patients and family members), and can provide insight into the abnormal brainwaves associated with AD.

We outline the creation of an Informed Environment, dedicated to urban life simulation. We propose methods and tools for creating and providing the information necessary for animating virtual humans in a city using an Informed Environment. The Informed Environment is based on a hierarchical decomposition of an urban scene into Environment Entities providing geometrical information as well as semantic notions, thus allowing a more realistic simulation of human behaviour. In this manner, virtual humans can integrate with a certain kind of urban knowledge

In this paper we present a virtual tennis game. We describe the creation and modeling of the virtual humans and body deformations, also showing the real-time animation and rendering aspects of the avatars. We focus on the animation of the virtual tennis ball and the behavior of a synthetic, autonomous referee who judges the tennis games. The networked, collaborative, virtual environment system is described with special reference to its interfaces to driver programs. We also mention the virtual reality (VR) devices that are used to merge the interactive players into the virtual tennis environment, together with the equipment and technologies employed for this exciting experience. We conclude with remarks on personal experiences during the game and on future research topics to improve parts of the presented system.

In networked virtual environments (NVEs), users are represented by their virtual embodiments. The articulated structure of these embodiments introduces a new complexity in the representation and streaming of animations, especially when the number of participants in the simulation increases. This requires real-time algorithms to decrease the networking overhead. The dead-reckoning technique is a way to reduce the required bit rate, and has been used for simple nonarticulated objects in popular NVE systems. We introduce a dead-reckoning technique for articulated virtual human figures, based on the MPEG-4 body animation specification, using Kalman filtering.

COVEN (Collaborative Virtual Environments) is a European project that seeks to develop a comprehensive approach to the issues in the development of collaborative virtual environment (CVE) technology. COVEN brings together twelve academic and industrial partners with a wide range of expertise in CSCW, networked VR, computer graphics, human factors, HCI, and telecommunications infrastructures. After two years of work, we are presenting the main features of our approach and results, our driving applications, the main components of our technical investigations, and our experimental activities. With different citizen and professional application scenarios as driving forces, COVEN is exploring the requirements and supporting techniques for collaborative interaction in scalable CVEs. Technical results are being integrated in an enriched networked VR platform based on the dVS and DIVE systems. Taking advantage of a dedicated Europe-wide ISDN and ATM network infrastructure, a large component of the project is a trial and experimentation activity that should allow a comprehensive understanding of the network requirements of these systems as well as their usability issues and human factors aspects.

We propose a model of a tennis game simulation with synthetic actors as players and a referee. The behavior of these actors is based on their synthetic vision and audition. Physical modeling of the ball dynamics and sound rendering enhance the realism. An interactive user can also play against a synthetic actor by using a Space Ball

The authors have been working on simulating virtual humans for several years. Until recently, these constructs could not act in real time. Today, however, many applications need to simulate in real time virtual humans that look realistic. They have invested considerable effort in developing and integrating several modules into a system capable of animating humans in real-time situations. This includes interactive modules for building realistic individuals and a texture-fitting method suitable for all parts of the head and body. Animating the body, including the hands and their deformations, is the key aspect of the system; to their knowledge, no competing system integrates all these functions. They also included facial animation, as demonstrated below with virtual tennis players. They have developed a single system containing all the modules needed for simulating real-time virtual humans in distant virtual environments (VEs). The system lets one rapidly clone any individual and animate the clone in various contexts. People cannot mistake virtual humans for real ones, but they think them recognizable and realistic.

In order to feel the sense of presence in a virtual environment, it is important for the participants to become a part of this environment and interact with it through natural behaviors. This interaction is even more important in networked collaborative virtual environments, in which the participants need to see and interact with each other. We present the Virtual Life Network (VLNET), a joint research effort in the field of networked collaborative virtual environments at MIRALab of the University of Geneva and the Computer Graphics Laboratory of the Swiss Federal Institute of Technology, Lausanne. In VLNET each participant is represented by a virtual human actor with realistic appearance and movements similar to the actual body. Interacting with the environment through his virtual body, the participant is perceived by himself and others in a natural way. Since it is generally not possible to track all degrees of freedom of the human body in order to reproduce realistic body motion, we introduce the motor functions that generate natural motion for standard tasks such as walking and arm motion; they are based on limited tracked information (hand and head positions). By using the same virtual human representation, but with the addition of high-level control, autonomous virtual actors can be introduced into the environment to perform some useful tasks or simply to make the environment more appealing. To further enhance the realistic feel of the virtual environment and to simplify object manipulation we provide the facility of defining object behaviors by attaching motor functions to the objects.

Virtual environments define a new interface for networked multimedia applications. The sense of “presence” in the virtual environment is an important requirement for collaborative activities involving multiple remote users working with social interactions. Using virtual actors within the shared environment is a supporting tool for presence. In this paper, we present a shared virtual life network with virtual humans that provides a natural interface for collaborative working and we describe the bridge we realized between this 3D shared world and the Web through a system of 3D snapshots.

This paper provides a detailed and complete description of merging virtual actors with animation in a real environment. It describes the tasks involved in each stage of integration, such as video acquisition, extraction of camera parameters, creation and animation of virtual actors, and rendering the final images. The most important problems are discussed: real objects hidden by virtual actors and vice versa, collision detection between the virtual actor and the real environment, correspondence between real and virtual cameras, and casting shadows of the virtual actors on the real world. Case studies are presented, such as the virtual actress Marilyn walking with real people on a real street, or sitting down on a real chair.

Using virtual humans to represent participants promotes realism in networked VEs. Different message types used to animate the human body and face impose varying network requirements, as analyzed here.

In this tutorial, we will describe both virtual characters and realistic humanoid social robots using the same high level models. Particularly, we will describe:1. How to capture real-time gestures and facial emotions from real people, how to recognize any real person, how to recognize certain sounds. We will present a state of the art and some new avenues of research.2. How to model a variety of interactive reactions of the virtual humans and social robots (facial expressions, gestures, multiparty dialog, etc) depending on the real scenes input parameters.3. How we can define Virtual Characters that have an emotional behavior (personality and mood and emotions) and how to allow them to remember us and have a believable relationship with us. This part is to allow virtual humans and social robots to have an individual and not automatic behaviour. This tutorial will also address the modelling of long-term and short-term memory and the interactions between users and virtual humans based on gaze and how to model visual attention. We will explain different methods to identify user actions and how to allow Virtual Characters to answer to them.4. This tutorial will also address the modelling of long-term and short-term memory and the interactions between users and virtual humans based on gaze and how to model visual attention. We will present the concepts of behavioral animation, group simulation, intercommunication between virtual humans, social humanoid robots and real people.Case studies will be presented from the Being There Centre (see http://imi.ntu.edu.sg/BeingThereCentre/Projects/Pages/Project4.aspx) where autonomous virtual humans and social robots react to a few actions from the real people.

Since the identification of the first cases of Autism Spectrum Disorder (ASD) in 1943 by Dr Leo Kanner, the number of reported ASD cases as well as research studies expanding our understanding of the disorder has increased rapidly and significantly. In terms of ASD treatment, there is a wide range of conventional as well as complementary and alternative interventions. Since the beginning of the new millennium, interest in Virtual-reality (VR) technology within the domains of special education and neuro-rehabilitative medicine has been growing rapidly. Today, the VR application research studies have offered more opportunities and a wider scope for developing innovative autism treatment. This chapter presents one VR application in autism treatment using artificial agents or bots (3D virtual pink dolphins) in a virtual environment (3D dolphinarium) through an interdisciplinary collaboration among engineering, new media, arts, and education at the Nanyang Technological University, Singapore. The collaborative effort has resulted in the development of the Learning Activity System Design for autistic children using virtual pink dolphins to serve as a platform for future development in the VR-based autism treatment.

A real-time system is proposed to quantitatively assess speaking mannerisms and social behavior from audio recordings of two-person dialogs. Speaking mannerisms are quantitatively assessed by low-level speech metrics such as volume, rate, and pitch of speech. The social behavior is quantified by sociometrics including level of interest, agreement, and dominance. Such quantitative measures can be used to provide real-time feedback to the speakers, for instance, to alarm to speaker when the voice is too strong (speaking mannerism), or when the conversation is not proceeding well due to disagreements or numerous interruptions (social behavior). In the proposed approach, machine learning algorithms are designed to compute the sociometrics (level of interest, agreement, and dominance) in real-time from combinations of low-level speech metrics. To this end, a corpus of 150 brief two-person dialogs in English was collected. Several experts assessed the sociometrics for each of those dialogs. Next, the resulting annotated dialogs are used to train the machine learning algorithms in a supervised manner. Through this training procedure, the algorithms learn how the sociometrics depend on the low-level speech metrics, and consequently, are able to compute the sociometrics from speech recordings in an automated fashion, without further help of experts. Numerical tests through leave-one-out cross-validation indicate that the accuracy of the algorithms for inferring the sociometrics is in the range of 80-90%. In future, those reliable predictions can be the key to real-time sociofeedback, where speakers will be provided feedback in real-time about their behavior in an ongoing discussion. Such technology may be helpful in many contexts, for instance in group meetings, counseling, or executive training. Accepted version

We present an overview of some principles of synthetic vision and audition for digital autonomous actors in virtual worlds. After a short review of the state-of-the-art we focus on some aspects of synthetic vision and virtual world constraints. Then, we present a simple real time structured sound renderer. This sound renderer is used as audition channel for synthetic and real actors and synchronized sound track generator for video film productions.

We present an interface for navigating digital collections based on a one-dimensional analog control and a data visualization based on old analog radios. Our system takes advantage of inertial control to browse a large data collection in a compelling way, reducing the complexity of similar interfaces present in both desktop-based and portable media players. This vintage radio interface has been used to navigate a digital music collection. We have compared the proposed interface with the current most popular hardware, the iPod. The results of user tests with 24 participants are presented and discussed. The insights gained are encouraging enough to continue the development of one-dimensional analog controls for content discovery and retrieval.

Non-verbal communication (NVC) makes up about two-thirds of all communication between two people or between one speaker and a group of listeners. However, this fundamental aspect of communicating is mostly omitted in 3D social forums or virtual world oriented games. This paper proposes an answer by presenting a multi-user 3D-chatting system enriched with NVC relative to motion. This event-based architecture tries to recreate a context by extracting emotional cues from dialogs and derives virtual human potential body expressions from that event triggered context model. We structure the paper by expounding the system architecture enabling the modeling NVC in a multi-user 3D-chatting environment. There, we present the transition from dialog-based emotional cues to body language, and the management of NVC events in the context of a virtual reality client-server system. Finally, we illustrate the results with graphical scenes and a statistical analysis representing the increase of events due to NVC.

Providing high quality recommendations is important for online systems to assist users who face a vast number of choices in making effective selection decisions. Collaborative filtering is a widely accepted technique to provide recommendations based on ratings of similar users. But it suffers from several issues like data sparsity and cold start. To address these issues, in this paper, we propose a simple but effective method, namely "Merge", to incorporate social trust information (i.e. trusted neighbors explicitly specified by users) in providing recommendations. More specifically, ratings of a user's trusted neighbors are merged to represent the preference of the user and to find similar other users for generating recommendations. Experimental results based on three real data sets demonstrate that our method is more effective than other approaches, both in accuracy and coverage of recommendations.

The last ten years have seen an explosion in the diversity of digital-life devices, e.g. music and video players. However, the interaction paradigm to use these devices has remained mostly unchanged. Remote controls are still the most common way to manage a digital-life device. Moreover, the interaction between devices themselves is still very limited and rarely addressed by a remote control interface. We present in this paper a study of tangible drag-and-drop, a remote control interface based on the well-known paradigm coming from the graphical user interface. This interaction technique aims at reducing the gap between the digital and physical worlds, enabling the transfer of digital data from one device to another. To validate such a concept, we present two prototypes, along with user studies and a general discussion about the tangible drag-and-drop technique.

Our goal is to create dancing crowds in cyber worlds and to use this feature to support creative endeavors such as pre-visualization of choreography and actual stage performances. In this paper we present a method of motion planning using dance motion clips. We describe a trial algorithm of collision avoidance using a grid map. We also present methods to create variety in animation for dance choreographies. As a result, we confirmed that most collisions could be avoided by the motion planning method. However, the need for some improvements in creating a conceptual dancing crowd was found.

Over the years, different approaches have been explored to build effective learning methods in virtual reality but the design of effective 3D manipulation techniques still remains an important research problem. To this end, it is important to quantify behavioral and brain mechanisms underlying the geometrical mappings of the body with the environment and external objects, both within the virtual environments (VE), the real world and relative to each other. The successful mapping of such interactions entails the study of fundamental components of these interactions, such as the origin of the visuo-spatial perspective (1PP, 3PP) and how they contribute to the user's performance in the virtual environments. Here, we report data using a novel set-up exposing participants - during free navigation - with a scene view from either 3PP or the habitual first-person perspective (1PP).

Garment design is traditionally carried out in two dimensions, and some software has been developed and applied in the garment industry in the design of garment panels. A new tool for the interactive design of garments in three dimensions is introduced. Making use of an elastic surface model, animation allows one to examine the garment design in three dimensions dynamically. The designer can use this tool to visualize his original ideas and changes interactively, and to see the garment vividly portrayed, including texture mapping on the final design, before the real cloth panels are cut. Application of this tool in the garment industry could reduce design time and costs substantially