Search

Degradation of the visual system can lead to a dramatic reduction of mobility by limiting a person to his sense of touch and hearing. This paper presents the development of an obstacle detection system for visually impaired people. While moving in his environment the user is alerted to close obstacles in range. The system we propose detects an obstacle surrounding the user by using a multi-sonar system and sending appropriate vibrotactile feedback. The system aims at increasing the mobility of visually impaired people by offering new sensing abilities.

This paper proposes a novel method to control virtual characters in dynamic environments. A virtual character is animated by a locomotion and jumping engine, enabling production of continuous parameterized motions. At any time during runtime, flat obstacles (e.g. a puddle of water) can be created and placed in front of a character. The method first decides whether the character is able to get around or jump over the obstacle. Then the motion parameters are accordingly modified. The transition from locomotion to jump is performed with an improved motion blending technique. While traditional blending approaches let the user choose the transition time and duration manually, our approach automatically controls transitions between motion patterns whose parameters are not known in advance. In addition, according to the animation context, blending operations are executed during a precise period of time to preserve specific physical properties. This ensures coherent movements over the parameter space of the original input motions. The initial locomotion type and speed are smoothly varied with respect to the required jump type and length. This variation is carefully computed in order to place the take-off foot as close to the created obstacle as possible.

A common problem in virtual character computer animation concerns the preservation of the basic foot-floor constraint (or footplant), consisting in detecting it before enforcing it. This paper describes a system capable of generating motion while continuously preserving the footplants for a real-time, dynamically evolving context. This system introduces a constraint detection method that improves classical techniques by adaptively selecting threshold values according to motion type and quality. The footplants are then enforced using a numerical inverse kinematics solver. As opposed to previous approaches, we define the footplant by attaching to it two effectors whose position at the beginning of the constraint can be modified, in order to place the foot on the ground, for example. However, the corrected posture at the constraint beginning is needed before it starts to ensure smoothness between the unconstrained and constrained states. We, therefore, present a new approach based on motion anticipation, which computes animation postures in advance, according to time-evolving motion parameters, such as locomotion speed and type. We illustrate our on-line approach with continuously modified locomotion patterns, and demonstrate its ability to correct motion artifacts, such as foot sliding, to change the constraint position and to modify from a straight to a curved walk motion.

This paper introduces a framework for real-time simulation and rendering of crowds navigating in a virtual environment. The solution first consists in a specific environment preprocessing technique giving rise to navigation graphs, which are then used by the navigation and simulation tasks. Second, navigation planning interactively provides various solutions to the user queries, allowing to spread a crowd by individualizing trajectories. A scalable simulation model enables the management of large crowds, while saving computation time for rendering tasks. Pedestrian graphical models are divided into three rendering fidelities ranging from billboards to dynamic meshes, allowing close-up views of detailed digital actors with a large variety of locomotion animations. Examples illustrate our method in several environments with crowds of up to 35000 pedestrians with real-time performance. Copyright © 2006 John Wiley and Sons, Ltd.

Expressive facial animation synthesis of human like characters has had many approaches with good results. MPEG-4 standard has functioned as the basis of many of those approaches. In this paper we would like to lay out the knowledge of some of those approaches inside an ontology in order to support the modeling of emotional facial animation in virtual humans (VH). Inside this ontology we will present MPEG-4 facial animation concepts and its relationship with emotion through expression profiles that utilize psychological models of emotions. The ontology allows storing, indexing and retrieving prerecorded synthetic facial animations that can express a given emotion. Also this ontology can be used a refined knowledge base in regards to the emotional facial animation creation. This ontology is made using Web Ontology Language and the results are presented as answered queries. Copyright © 2006 John Wiley and Sons, Ltd.

This paper presents an original model with methodologies that integrate in a novel way different types of an autonomous virtual agent's perception in a virtual environment. Our first new approach permits the coherent management of the shared virtual environment for the simulations of an autonomous virtual agent (AVA). Our second approach allows the prediction or the estimation of both the orientation and the attention of an AVA in a virtual environment. By means of a test application with a 'virtual goalkeeper', we demonstrate the speed and the robustness of our technique

This paper presents a sketching system for 3D organic shapes modeling and animation using virtual reality devices. On the hardware side, it is based on the Haptic Workstation™ which conveys force feedback on the user arms (upper body limbs), and a head mounted display to present the 3D generated images. On the software side, we use implicit surfaces modeling techniques such as metaballs. In fact, designers feel well comfortable with this kind of primitives due to their ability in the organic shapes creation such as virtual humans. The proposed system provides an efficient alternative to produce advanced 3D shapes sketching

In this paper, we present research results and future challenges in creating realistic and believable Virtual Humans. To realize these modeling goals, real-time realistic representation is essential, but we also need interactive and perceptive Virtual Humans to populate the Virtual Worlds. Three levels of modeling should be considered to create these believable Virtual Humans: 1) realistic appearance modeling, 2) realistic, smooth and flexible motion modeling, and 3) realistic high-level behaviors modeling. At first, the issues of creating virtual humans with better skeleton and realistic deformable bodies are illustrated. To give a level of believable behavior, challenges are laid on generating on the fly flexible motion and complex behaviours of Virtual Humans inside their environments using a realistic perception of the environment. Interactivity and group behaviours are also important parameters to create believable Virtual Humans which have challenges in creating believable relationship between real and virtual humans based on emotion and personality, and simulating realistic and believable behaviors of groups and crowds. Finally, issues in generating realistic virtual clothed and haired people are presented.

The ability to build and control exposure through a seamless synergy of interaction and narration is a strongly required feature of a new type of immersive-VR training and therapy system. This paper presents a practical approach for immersive-VR training and therapy applications based on interactive storytelling. It provides detailed description of a working implementation of the Interactive Narration Space (INS); this approach combines and satisfies both interaction and narration requirements through the use of high-level social interaction. By introducing the Social Channel, we aim at minimizing the contradictions between control over the story required by the trainer/therapist and interaction required by the trainee/patient. These concepts and their practical realization have been investigated in the context of emergency-situation training and psychotherapeutic exposure, and could validate the usability of mediated interaction with a virtual assistant.

A traditional multi-layered approach is adopted to human body modeling and deformation. The model is split into three general anatomical structures: the skeleton, musculature and skin. It is shown that each of these layers is modeled and deformed by using fast, procedural, ad-hoc methods that can painlessly be reimplemented. The modeling approach is generic enough to handle muscles of varying shape, size and characteristics and does not break in extreme skeleton poses. It is also described that the integrated MuscleBuilder system whose main features are: i) easy and quick creation of muscle deformation models; ii) automatic deformation of an overlying skin. It is shown that visually realistic results can be obtained at interactive frame rates with very little input from the designer.

Our approach is based on the multi-sensory integration of the standard theory of neuroscience, where signals of a single object coming from distinct sensory systems are combined. The acquisition steps of signals, filtering, selection and simplification intervening before proprioception, active and predictive perception are integrated into virtual sensors and a virtual environment. We will focus on two aspects: 1) the assignment problem: determining which sensory stimuli belong to the same virtual object and (2) the sensory recoding problem: recoding signals in a common format before combining them. We have developed three novel methodologies to map the information coming from the virtual sensors of vision, audition and touch as well as that of the virtual environment in the form of a 'cognitive map'. Copyright © 2004 John Wiley and Sons, Ltd.

In order to guarantee an effective and punctual medical intervention to injured people involved in health emergency situations, where usually both professional and non-professional health operators are involved, a fast and accurate treatment has to be carried out. In case of catastrophic or very critical situations, non-professional operators who did not receive proper training (volunteers are among them) could be affected by psychological inhibitions. Their performances could slow down in such way that would affect the quality of the treatment and increase both direct and indirect costs. Our virtual reality system that is currently in use at the health care emergency center of San Martino Hospital in Genoa, Italy, has been designed and developed to check health emergency operators' capabilities to adopt correct decision-making procedures, to make optimal use of new technological equipment and to overcome psychological barriers. Our system is composed of (1) a high-end simulation PC, whose main functions are execution of the main software module, rendering of 3D scenes in stereo mode, rendering of sound, and control of data transmission from/to VR devices; (2) a low-end control PC, which controls the VR simulation running on the simulation PC, manages medical emergency simulation scenarios, introduces unexpected events to the simulation and controls the simulation difficulty level; (3) a magnetic-based motion tracking device used for head and hand tracking; (4) a wireless pair of shutter glasses together with a cathode ray tube wall projector; and (5) a high-end surround sound system. The expected benefits have been verified through the design and implementation of controlled clinical trials.

While virtual crowds are becoming common in non-real-time applications, the real-time domain is still relatively unexplored. In this paper we discuss the challenges involved in creating such simulations, especially the need to efficiently manage variety. We introduce the concept of levels of variety, then present our work on crowd behaviour simulation aimed at interactive real-time applications such as computer games or virtual environments. We define a modular behavioural architecture of a multi-agent system allowing autonomous and scripted behaviour of agents supporting variety. Finally we show applications of our system in a virtual reality training system and virtual heritage reconstruction

Real-time 3D graphics are being extensively used to build interactive virtual environments for a number of different applications. In many situations, virtual objects are required to exhibit complex behaviors, and the use of a versatile behavioral programming methodology is required. This paper presents a feature modeling approach to define behavioral information of virtual objects, giving special attention to their capabilities of interaction with virtual human actors. The proposed method is based on a complete definition and representation of interactive objects, which are called smart objects. This representation is based on the description of interaction features: parts, movements, graspable sites, functionalities, purposes, etc., including a behavioral automaton based on scripts and graphical state machines

Human activity recognition in unconstrained RGB--D videos has extensive applications in surveillance, multimedia data analytics, human-computer interaction, etc, but remains a challenging problem due to the background clutter, camera motion, viewpoint changes, etc. We develop a novel RGB--D activity recognition approach that leverages the dense trajectory feature in RGB videos. By mapping the 2D positions of the dense trajectories from RGB video to the corresponding positions in the depth video, we can recover the 3D trajectory of the tracked interest points, which captures important motion information along the depth direction. To characterize the 3D trajectories, we apply motion boundary histogram (MBH) to depth direction and propose 3D trajectory shape descriptors. Our proposed 3D trajectory feature is a good complementary to dense trajectory feature extracted from RGB video only. The performance evaluation on a challenging unconstrained RGB--D activity recognition dataset, i.e., Hollywood 3D, shows that our proposed method outperforms the baseline methods (STIP-based) significantly, and achieves the state-of-the-art performance.

We propose to learn a non-intrusive quality assessment metric for enhanced speech signals. High-dimension spectro-temporal features are extracted by the Gabor filter bank for speech signals. To reduce the high-dimension features, we use PCA (Principal Component Analysis) to process these features. After obtaining the feature vector from audio signals, Support Vector Regression (SVR) is used to learn the metric for quality evaluation of enhanced speech signals. Experimental results on NOIZEUS dataset demonstrate that proposed non-intrusive quality assessment metric by using spectro-temporal features can obtain better performance for enhanced speech signals.

Trust has been extensively studied and its effectiveness demonstrated in recommender systems. Due to the lack of explicit trust information in most systems, many trust metric approaches have been proposed to infer implicit trust from user ratings. However, previous works have not compared these different approaches, and oftentimes focus only on the performance of predictive item ratings. In this paper, we first analyse five kinds of trust metrics in light of the properties of trust. We conduct an empirical study to explore the ability of trust metrics to distinguish explicit trust from implicit trust and to generate accurate predictions. Experimental results on two real-world data sets show that existing trust metrics cannot provide satisfying performance, and indicate that future metrics should be designed more carefully.

In this chapter, we present a system that provides real-time feedback about an ongoing discussion. Various speech statistics, such as speaking length, speaker turns and speaking turn duration, are computed and displayed in real time. In social monitoring, such statistics have been used to interpret and deduce talking mannerisms of people and gain insights on human social characteristics and behaviour. However, such analysis is usually conducted in an offline fashion, after the discussion has ended. In contrast, our system analyses the speakers and provides feedback to the speakers in real time during the discussion, which is a novel approach with plenty of potential applications. The proposed system consists of portable, easy to use equipment for recording the conversations. A user-friendly graphical user interface displays statistics about the ongoing discussion. Customized individual feedback to participants during conversation can be provided. Such close-loop design may help individuals to contribute effectively in the group discussion, potentially leading to more productive and perhaps shorter meetings. Here we present preliminary results on two-people face to face discussion. In the longer term, our system may prove to be useful, e.g. for coaching purposes and for facilitating business meetings.

This Chapter discusses the methods involved in the generation of large crowds of Virtual Humans in environments like cities. We focus on the geometric aspects of these methods in the different steps involved: scaler, simulator, renderer, path planner, and behaviour handler. We emphasize the application of these methods to the field of Cultural Heritage, recreating old cities with population living their life. In particular, we present examples from Pompeii and discuss the interaction between the environment and the behaviour of the Romans.

In neurofeedback, brain waves are transformed into sounds or music, graphics, and other representations, to provide real-time information on ongoing waves and patterns in the brain. Here we present various forms of neurofeedback, including sonification, sonification in combination with visualization, and at last, immersive neurofeedback, where auditory and visual feedback is provided in a multi-sided immersive environment in which participants are completely surrounded by virtual imagery and 3D sound. Neural feedback may potentially improve the user’s (or patient’s) ability to control brain activity, the diagnosis of medical conditions, and the rehabilitation of neurological or psychiatric disorders. Several psychological and medical studies have confirmed that virtual immersive activity is enjoyable, stimulating, and can have a healing effect. As an illustration, neurofeedback is generated from electroencephalograms (EEG) of Alzheimer’s disease (AD) patients and healthy subjects. The auditory, visual, and immersive representations of Alzheimer’s EEG differ substantially from healthy EEG, potentially yielding novel diagnostic tools. Moreover, such alternative representations of AD EEG are natural and intuitive, and hence easily accessible to laymen (AD patients and family members), and can provide insight into the abnormal brainwaves associated with AD.

In this paper, a human–robot interaction system based on a novel combination of sensors is proposed. It allows one person to interact with a humanoid social robot using natural body language. The robot understands the meaning of human upper body gestures and expresses itself by using a combination of body movements, facial expressions, and verbal language. A set of 12 upper body gestures is involved for communication. This set also includes gestures with human–object interactions. The gestures are characterized by head, arm, and hand posture information. The wearable Immersion CyberGlove II is employed to capture the hand posture. This information is combined with the head and arm posture captured from Microsoft Kinect. This is a new sensor solution for human-gesture capture. Based on the posture data from the CyberGlove II and Kinect, an effective and real-time human gesture recognition method is proposed. The gesture understanding approach based on an innovative combination of sensors is the main contribution of this paper. To verify the effectiveness of the proposed gesture recognition method, a human body gesture data set is built. The experimental results demonstrate that our approach can recognize the upper body gestures with high accuracy in real time. In addition, for robot motion generation and control, a novel online motion planning method is proposed. In order to generate appropriate dynamic motion, a quadratic programming (QP)-based dual-arms kinematic motion generation scheme is proposed, and a simplified recurrent neural network is employed to solve the QP problem. The integration of a handshake within the HRI system illustrates the effectiveness of the proposed online generation method.

In the past decade, networked virtual environments (NVEs) have been an increasingly active area of research, with the first commercial systems emerging recently. Graphical and behavioral representation of users within such systems is a particularly important issue that has lagged in development behind other issues such as network architectures and space structuring. In this paper, we expose the importance of using virtual humans within these systems and provide a brief overview of several virtual humans technologies used in particular for simulation of crowds. As the main technical contribution, the paper presents the integration of these technologies with the COVEN-DIVE platform, the extension of the DIVE system developed within the COVEN project. In conjunction with this, we present our contributions through the COVEN project to the MPEG-4 standard concerning the representation of virtual humans.

The precise evaluation of joint movements, using reconstructed three-dimensional models of the joint tissues, is an important issue in many medical diagnosis and surgery planning. In order to achieve appropriate results by analyzing 3D joint models, efficient and accurate evaluation of contact among the virtual tissues is critical. In this chapter, some recent works regarding contact modeling and collision detection in human joints have been covered. First, two new different fast collision detection methods, suitable for evaluating human joints, have been explained. These methods take advantage of the relative proximity and the nature of the movement in human joints, in order to discard unnecessary calculations. Later, two medical applications based on these collision detection methods, which involve finding the maximum range of motion in human joints and also evaluating joint diseases (i.e. Femoroacetabular Impingements) have been discussed. Finally, the results of an investigation of the sensitivity of the joint contact evaluation to the estimated center of rotation, again by exploiting the explained collision detection methods, have been presented. Many of the mentioned applications and investigations have been examined by using 3D models of human hip joints; however, due to the nature of the methods, they can be applied to other types of human joints too.

The creation of virtual humans capable of behaving and interacting realistically with each other requires the development of autonomous believable social agents. Standard goal-oriented approaches are not well suited to it because they don't take into account important characteristics identified by the social sciences. This paper tackles the issue of a general social reasoning mechanism, discussing its basic functional requirements using a sociological perspective, and proposing a high-level architecture based on Roles, Norms, Values and Types.

In virtual human modeling, the shoulder is usually composed of clavicular, scapular and arm segments related by rotational joints. Although the model is improved, the realistic animation of the shoulder is hardly achieved. This is due to the fact that it is difficult to coordinate the simultaneous motion of the shoulder components in a consistent way. Also, the common use of independent one-degree of freedom (DOF) joint hierarchies does not properly render the 3-D accessibility space of real joints. On the basis of former biomechanical investigations, we propose here an extended shoulder model including scapulo-thoracic constraint and joint sinus cones. As a demonstration, the model is applied, using inverse kinematics, to the animation of a 3-D anatomic muscled skeleton model.

Rendering and animating in real-time a multitude of articulated characters presents a real challenge, and few hardware systems are up to the task. Up to now, little research has been conducted to tackle the issue of real-time rendering of numerous virtual humans. This paper presents a hardware-independent technique that improves the display rate of animated characters by acting on the sole geometric and rendering information. We first review the acceleration techniques traditionally in use in computer graphics and highlight their suitability to articulated characters. We then show how impostors can be used to render virtual humans. We introduce concrete case studies that demonstrate the effectiveness of our approach. Finally, we tackle the visibility issue.

We outline the creation of an Informed Environment, dedicated to urban life simulation. We propose methods and tools for creating and providing the information necessary for animating virtual humans in a city using an Informed Environment. The Informed Environment is based on a hierarchical decomposition of an urban scene into Environment Entities providing geometrical information as well as semantic notions, thus allowing a more realistic simulation of human behaviour. In this manner, virtual humans can integrate with a certain kind of urban knowledge

We investigate how the movement measurement technologies developed for virtual reality applications can be applied to track in real-time the full body posture of a human being. The accuracy of this information is of definite importance to evaluate the feasibility of complex tasks involving human beings. We present a full body movement measurement approach. It provides a realistic conversion in real-time with a reasonable number of sensors. Associated with the hand movement measurement and correction algorithms we provide a pertinent visual and vibrotactile feedback to the performer.

Motion-capture techniques are rarely based on orientation measurements for two main reasons: (1) optical motion-capture systems are designed for tracking object position rather than their orientation (which can be deduced from several trackers), (2) known animation techniques, like inverse kinematics or geometric algorithms, require position targets constantly, but orientation inputs only occasionally. We propose a complete human motion-capture technique based essentially on orientation measurements. The position measurement is used only for recovering the global position of the performer. This method allows fast tracking of human gestures for interactive applications as well as high rate recording. Several motion-capture optimizations, including the multijoint technique, improve the posture realism. This work is well suited for magnetic-based systems that rely more on orientation registration (in our environment) than position measurements that necessitate difficult system calibration.

In this paper we present a virtual tennis game. We describe the creation and modeling of the virtual humans and body deformations, also showing the real-time animation and rendering aspects of the avatars. We focus on the animation of the virtual tennis ball and the behavior of a synthetic, autonomous referee who judges the tennis games. The networked, collaborative, virtual environment system is described with special reference to its interfaces to driver programs. We also mention the virtual reality (VR) devices that are used to merge the interactive players into the virtual tennis environment, together with the equipment and technologies employed for this exciting experience. We conclude with remarks on personal experiences during the game and on future research topics to improve parts of the presented system.

In networked virtual environments (NVEs), users are represented by their virtual embodiments. The articulated structure of these embodiments introduces a new complexity in the representation and streaming of animations, especially when the number of participants in the simulation increases. This requires real-time algorithms to decrease the networking overhead. The dead-reckoning technique is a way to reduce the required bit rate, and has been used for simple nonarticulated objects in popular NVE systems. We introduce a dead-reckoning technique for articulated virtual human figures, based on the MPEG-4 body animation specification, using Kalman filtering.

COVEN (Collaborative Virtual Environments) is a European project that seeks to develop a comprehensive approach to the issues in the development of collaborative virtual environment (CVE) technology. COVEN brings together twelve academic and industrial partners with a wide range of expertise in CSCW, networked VR, computer graphics, human factors, HCI, and telecommunications infrastructures. After two years of work, we are presenting the main features of our approach and results, our driving applications, the main components of our technical investigations, and our experimental activities. With different citizen and professional application scenarios as driving forces, COVEN is exploring the requirements and supporting techniques for collaborative interaction in scalable CVEs. Technical results are being integrated in an enriched networked VR platform based on the dVS and DIVE systems. Taking advantage of a dedicated Europe-wide ISDN and ATM network infrastructure, a large component of the project is a trial and experimentation activity that should allow a comprehensive understanding of the network requirements of these systems as well as their usability issues and human factors aspects.