Search

Purpose Surgical gesture recognition has been an essential task for providing intraoperative context-aware assistance and scheduling clinical resources. However, previous methods present limitations in catching long-range temporal information, and many of them require additional sensors. To address these challenges, we propose a symmetric dilated network, namely SD-Net, to jointly recognize surgical gestures and assess surgical skill levels only using RGB surgical video sequences. Methods We utilize symmetric 1D temporal dilated convolution layers to hierarchically capture gesture clues under different receptive fields such that features in different time span can be aggregated. In addition, a self-attention network is bridged in the middle to calculate the global frame-to-frame relativity. Results We evaluate our method on a robotic suturing task from the JIGSAWS dataset. The gesture recognition task largely outperforms the state of the arts on the frame-wise accuracy up to $$\sim $$ ∼ 6 points and the F1@50 score $$\sim $$ ∼ 8 points. We also keep the 100% predicted accuracy for the skill assessment task using LOSO validation scheme. Conclusion The results indicate that our architecture is able to obtain representative surgical video features by extensively considering the spatial, temporal and relational context from raw video input. Furthermore, the better performance in multi-task learning implies that surgical skill assessment has a complementary effects to gesture recognition task.

Brushstrokes are viewed as the artist’s “handwriting” in a painting. In many applications such as style learning and transfer, mimicking painting, and painting authentication, it is highly desired to quantitatively and accurately identify brushstroke characteristics from old masters’ pieces using computer programs. However, due to the nature of hundreds or thousands of intermingling brushstrokes in the painting, it still remains challenging. This article proposes an efficient algorithm for brush Stroke extraction based on a Deep neural network, i.e., DStroke. Compared to the state-of-the-art research, the main merit of the proposed DStroke is to automatically and rapidly extract brushstrokes from a painting without manual annotation, while accurately approximating the real brushstrokes with high reliability. Herein, recovering the faithful soft transitions between brushstrokes is often ignored by the other methods. In fact, the details of brushstrokes in a master piece of painting (e.g., shapes, colors, texture, overlaps) are highly desired by artists since they hold promise to enhance and extend the artists’ powers, just like microscopes extend biologists’ powers. To demonstrate the high efficiency of the proposed DStroke, we perform it on a set of real scans of paintings and a set of synthetic paintings, respectively. Experiments show that the proposed DStroke is noticeably faster and more accurate at identifying and extracting brushstrokes, outperforming the other methods.

Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this article, we present a new unsupervised learning framework for high-quality pixel-wise smoke detection and removal. One of the well recognized grand challenges in using convolutional neural networks (CNNs) for medical image processing is to obtain intra-operative medical imaging datasets for network training and validation, but availability and quality of these datasets are scarce. Our novel training framework does not require ground-truth image pairs. Instead, it learns purely from computer-generated simulation images. This approach opens up new avenues and bridges a substantial gap between conventional non-learning based methods and which requiring prior knowledge gained from extensive training datasets. Inspired by the Generative Adversarial Network (GAN), we have developed a novel generative-collaborative learning scheme that decomposes the de-smoke process into two separate tasks: smoke detection and smoke removal. The detection network is used as prior knowledge, and also as a loss function to maximize its support for training of the smoke removal network. Quantitative and qualitative studies show that the proposed training framework outperforms the state-of-the-art de-smoking approaches including the latest GAN framework (such as PIX2PIX). Although trained on synthetic images, experimental results on clinical images have proved the effectiveness of the proposed network for detecting and removing surgical smoke on both simulated and real-world laparoscopic images.

Even though ureteroscopic laser lithotripsy (URSL) has become the preferred treatment option for urolithiasis due to shorter operation time and a better stone-free rate, the optimum laser pulse settings for URSL with the shortest operative times remain unknown. In this chapter, two sets of design of experiments (DOE) were conducted with response surface methodology: 1) the quantitative responses of calculus ablation and retropulsion in terms of the pulse energy, pulse width, and the number of pulses of a prototype Chromium (Cr3+), Thulium (Tm3+), Holmium (Ho3+) triple doped yttrium aluminum garnet (CTH:YAG) laser system. The ablation or retropulsion is inversely proportional to the pulse width, and the pulse width has a higher impact coefficient for the ablation than for the retropulsion. The quadratic fit of the response surface for the volume of ablation has a nonlinear relationship with the pulse width and number of pulses. 2) the laser setting optimization of laser lithotripsy of a commercially available CTH: YAG laser system. The experimental setup is based on a benchtop model first introduced by Sroka’s group. Comparing to frequency, the laser pulse energy or peak power has a higher impact coefficient to stone retropulsion as compared to stone ablation in CTH: YAG laser lithotripsy. The most efficient way to curtail stone retropulsion during laser lithotripsy is to lower the laser pulse peak power.

We present a vision-based system for real-time pose tracking of the rigid object, it can not only estimate a single pose in six degrees of freedom (6DoF), but also suitable for recovering compound movements. The system is comprised of a monocular camera, and a series of 3D printed platonic solids with squared fiducial markers attached on each single face, which is easy to setup and extend, extra cameras are allowed to incorporate into the pipeline for meeting different requirements. The system realizes object tracking by estimating the pose of the fiducial platonic solid (FPS) which can be fixed onto the surface of the target object. Different sizes and shapes of the platonic solids are allowed to combine with each other to adapt to different application scenarios, this strategy provides enormous flexibility and applicability to our system. In order to track the motion of the fiducial platonic solid accurately, a robust algorithm that combines the fiducial constraint and the statistical constraint is introduced, which is able to handle illumination changes, motion blur and partial occlusion. We evaluate the performance of the proposed approach with qualitative and quantitative experiments, in addition, a couple of mixed reality (MR) applications are developed for demonstrating the effectiveness of the system.

Face image-sketch synthesis is widely applied in law enforcement and digital entertainment fields. Despite the extensive progression in face image-sketch synthesis, there are few methods focusing on generating a color face image from a sketch. The existing methods pay less attention to learning the illumination or highlight distribution on the face region. However, the illumination is the key factor that makes the generated color face image looks vivid and realistic. Moreover, existing methods tend to employ some image preprocessing technologies and facial region patching approaches to generate high-quality face images, which results in the high complexity and memory consumption in practice. In this paper, we propose a novel end-to-end generative adversarial fusion model, called GAF, which fuses two U-Net generators and a discriminator by jointly learning the content and adversarial loss functions. In particular, we propose a parametric tanh activation function to learn and control illumination highlight distribution over faces, which is integrated between the two U-Net generators by an illumination distribution layer. Additionally, we fuse the attention mechanism into the second U-Net generator of GAF to keep the identity consistency and refine the generated facial details. The qualitative and quantitative experiments on the public benchmark datasets show that the proposed GAF has better performance than existing image-sketch synthesis methods in synthesized face image quality (FSIM) and face recognition accuracy (NLDA). Meanwhile, the good generalization ability of GAF has also been verified. To further demonstrate the reliability and authenticity of face images generated using GAF, we use the generated face image to attack the well-known face recognition system. The result shows that the face images generated by GAF can maintain identity consistency and well maintain everyone's unique facial characteristics, which can be further used in the benchmark of facial spoofing. Moreover, the experiments are implemented to verify the effectiveness and rationality of the proposed parametric tanh activation function and attention mechanism in GAF.

As a promising topological superconductor, Fe(Se0.5Te0.5) recently attracts interest from researchers, while Bi2Se3 has been considered one of the most important topological insulator materials. Therefore the Bi2Se3/FeSe0.5Te0.5 bulk composite could be interesting in demonstrating novel topological properties. We investigated crystalline, morphological, magnetic, and transport properties of Bi2Se3/FeSe0.5Te0.5 composite samples synthesized by the solid-state reaction method, and found that the elevated sintering temperature at the second step during the synthesis improves the crystallinity. All samples exhibit both superconductivity and ferromagnetism. Several indices of superconductivity, such as the transition temperature, change with increasing sintering temperature.

Human motion tracking is a prevalent technique in many fields. A common difficulty encountered in motion tracking is the corrupted data is caused by detachment of markers in 3D motion data or occlusion in 2D tracking data. Most methods for missing markers problem may quickly become ineffective when gaps exist in the trajectories of multiple markers for an extended duration. In this paper, we propose the principal component eigenspace based gap filling methods that leverage a training sample set for estimation. The proposed method is especially beneficial in the scenario of motion data with less predictable or repeated movement patterns, and that of even missing entire frames within an interval of a sequence. To highlight algorithm robustness, we perform algorithms on twenty test samples for comparison. The experimental results show that our methods are numerical stable and fast to work.

The Ho:YAG laser has been the favored lithotripter for treating urinary calculus since shortly after its introduction in the 1990s because it can fragment all calculus compositions and produces less calculus migration (retropulsion) during treatment than the short-pulsed lasers. Although the lamp-pumped Ho:YAG laser has been commercialized for ureteroscopic laser lithotripsy (URS) for almost 25 years, the lamp-pumped laser rod's transient thermal behavior under an out-of-control lamp discharge has not been reported. A safety question arises: how big is the laser output under an out-of-control lamp discharge? The objective of this study is to simulate the transient thermal behavior of the lamp-pumped laser rod. The temperature profile inside the laser rod is transformed into an equivalent thermal lens, and from the time interval of the variation of the thermal lens between the lasing threshold and the maximum value of the stable region, we can estimate the level of the laser output. The simulation tool used for this study is the Ansys Fluent. The transient thermal behavior of the lamp-pumped laser rod under an out-of-control lamp discharge was reported. Optimization of the pumping pulse for the desired laser output pulse is for future study.

Since diverse and complex emotions need to be expressed by different facial deformation and appearances, facial animation has become a serious and on-going challenge for computer animation industry. Face reconstruction techniques based on 3D morphable face model and deep learning provide one effective solution to reuse existing databases and create believable animation of new characters from images or videos in seconds, which greatly reduce heavy manual operations and a lot of time. In this paper, we review the databases and state-of-the-art methods of 3D face reconstruction from a single RGB image. First, we classify 3D reconstruction methods into three categories and review each of them. These three categories are: Shape-from-Shading (SFS), 3D Morphable Face Model (3DMM), and Deep Learning (DL) based 3D face reconstruction. Next, we introduce existing 2D and 3D facial databases. After that, we review 10 methods of deep learning-based 3D face reconstruction and evaluate four representative ones among them. Finally, we draw conclusions of this paper and discuss future research directions.

Partial differential equation (PDE) based geometric modelling has a number of advantages such as fewer design variables, avoidance of stitching adjacent patches together to achieve required continuities, and physics-based nature. Although a lot of papers have investigated PDE-based shape creation, shape manipulation, surface blending and volume blending as well as surface reconstruction using implicit PDE surfaces, there is little work of investigating PDE-based shape reconstruction using explicit PDE surfaces, specially satisfying the constraints on four boundaries of a PDE surface patch. In this paper, we propose a new method of using an accurate closed form solution to a fourth-order partial differential equation to reconstruct 3D surfaces from point clouds. It includes selecting a fourth-order partial differential equation, obtaining the closed form solutions of the equation, investigating the errors of using one of the obtained closed form solutions to reconstruct PDE surfaces from differential number of 3D points.

Traditional work on bystander intervention in violent emergencies has found that the larger the group, the less the chance that any individual will intervene. Here, we tested the impact on helping behavior of the affiliation of the bystanders with respect to the participants. We recruited 40 male supporters of the U.K. Arsenal football club for a two-factor between groups study with 10 participants per group. Each participant spoke with a virtual human Arsenal supporter (V), the scenario displayed in a virtual reality system. During this conversation, another virtual character (P), not an Arsenal fan, verbally abused V for being an Arsenal fan leading eventually to physical pushing. There was a group of three virtual bystanders who were all either Arsenal supporters indicated by their shirts, or football fans wearing unbranded shirts. These bystanders either encouraged the participant to intervene or dissuaded him. We recorded the number of times that participants intervened to help V during the aggression. We found that participants were more likely to intervene when the bystanders were out-group with respect to the participant. By comparing levels of intervention with a “baseline” study (identical except for the presence of bystanders), we conclude that the presence of in-group bystanders decreases helping. We argue therefore that, other things being equal, diffusion of responsibility is more likely to be overcome when participant and victim share group membership, but bystanders do not. Our findings help to develop understanding of how diffusion of responsibility works by combining elements of both the bystander effect and the social identity approach to bystander behavior. &nbsp

Cross-section curves play an important role in many fields. Analytically representing cross-section curves can greatly reduce design variables and related storage costs and facilitate other applications. In this paper, we propose composite generalized elliptic curves to approximate open and closed cross-section curves, present their mathematical expressions, and derive the mathematical equations of surface reconstruction from composite generalized elliptic curves. The examples given in this paper demonstrate the effectiveness and high accuracy of the proposed method. Due to the analytical nature of composite generalized elliptic curves and the surfaces reconstructed from them, the proposed method can reduce design variables and storage requirements and facilitate other applications such as level of detail.

By extending the work published at ICCS 2020 [ 1 ], in this paper we propose a method to achieve cost-effective 3D printing of stiffened thin-shell objects. Our proposed method consists of three parts. The first part integrates finite element analysis, Voronoi diagram, and conformal mapping to obtain stiffener distribution. The second part combines finite element analysis with optimization calculations to determine the optimal sizes of stiffeners. And the third part introduces Monte-Carlo simulation to find a global optimum. The experiments made in this paper indicate that our proposed method is effective in minimizing 3D printing material consumption of stiffened thin-shell objects.

Once a railway accident occurs, the recovery process usually requires precise and safe operation of rescue equipment such as railway cranes. However, it is generally impractical to train crane operators in real accident sites considering the costs and human safety. This article aims to address the problem by proposing a rational VR-based crane training system, in which a visualization framework based on PhysX engine is designed to reconstruct a realistic railway accident scene for the purpose of effective virtual training. Based on that, crane operators are able to experience a realistic accident scene safely. In addition, the collision detection is used to present unexpected spatial constraints in railway accident sites. A user experiment with 10 participants demonstrated a high level of safety and smooth interactive performance of the proposed system, which is shown to be valuable for training of people who engaged in learning the proper railway accident rescue procedure.

The objective of motion capture (mocap) data completion is to recover missing measurement of the body markers from mocap. It becomes increasingly challenging as the missing ratio and duration of mocap data grow. Traditional approaches usually recast this problem as a low-rank matrix approximation problem based on the nuclear norm. However, the nuclear norm defined as the sum of all the singular values of a matrix is not a good approximation to the rank of mocap data. This paper proposes a novel approach to solve mocap data completion problem by adopting a new matrix norm, called truncated nuclear norm. An efficient iterative algorithm is designed to solve this problem based on the augmented Lagrange multiplier. The convergence of the proposed method is proved mathematically under mild conditions. To demonstrate the effectiveness of the proposed method, various comparative experiments are performed on synthetic data and mocap data. Compared to other methods, the proposed method is more efficient and accurate.

Designing and creating complex and interactive animation is still a challenge in the field of virtual reality, which has to handle various aspects of functional requirements (e.g. graphics, physics, AI, multimodal inputs and outputs, and massive data assets management). In this paper, a semantic framework is proposed to model the construction of interactive animation and promote animation assets reuse in a systematic and standardized way. As its ontological implementation, two domain specific ontologies for the hand-gesture-based interaction and animation data repository have been developed in the context of Chinese traditional shadow play art. Finally, prototype of interactive Chinese shadow play performance system using deep motion sensor device is presented as the usage example.

Multimodal interactions provide users with more natural ways to manipulate virtual 3D objects than using traditional input methods. An emerging approach is gaze modulated pointing, which enables users to perform object selection and manipulation in a virtual space conveniently through the use of a combination of gaze and other interaction techniques (e.g., mid-air gestures). As gaze modulated pointing uses different sensors to track and detect user behaviours, its performance relies on the user's perception on the exact spatial mapping between the virtual space and the physical space. An underexplored issue is, when the spatial mapping differs with the user's perception, manipulation errors (e.g., out of boundary errors, proximity errors) may occur. Therefore, in gaze modulated pointing, as gaze can introduce misalignment of the spatial mapping, it may lead to user's misperception of the virtual environment and consequently manipulation errors. This paper provides a clear definition of the problem through a thorough investigation on its causes and specifies the conditions when it occurs, which is further validated in the experiment. It also proposes three methods (Scaling, Magnet and Dual-gaze) to address the problem and examines them using a comparative study which involves 20 participants with 1040 runs. The results show that all three methods improved the manipulation performance with regard to the defined problem where Magnet and Dual-gaze delivered better performance than Scaling. This finding could be used to inform a more robust multimodal interface design supported by both eye tracking and mid-air gesture control without losing efficiency and stability.

© 2017 IOS Press and the authors. The fruit images on points cloud acquired by the current 3D scanner from field will appear a visible seams, inconvenient data acquisition or taking large space due to unorganized background. We give a SAOW method to cope with the space efficiency and realistic effects of texture synthesis on pear point model. At first, a point-quadtree is proposed to simplify the pear image division. Then, an adaptive multi-granularity morton coding scheme are presented to optimizing the memory space of pear image. At last, weighted oversampling mixing method is mainly focused on texture quality of pear surface. As shown in the experiment results, our adaptive division makes the memory space decline dramatically about 90.7% than non-division and 92.9% than general division respectively; adaptive code scheme helps to reduce the memory to 72.1% of ordinary morton code; weighted oversampling keeps the mixed texture more real and smoothly than current methods.

The segmentation of a point cloud is one of the key technologies for three-dimensional reconstruction, and the segmentation from three-dimensional views can facilitate reverse engineering. In this paper, we propose a self-adaptive segmentation algorithm, which can address challenges related to the region-growing algorithm, such as inconsistent or excessive segmentation. Our algorithm consists of two main steps: automatic selection of seed points according to extracted features and segmentation of the points using an improved region-growing algorithm. The benefits of our approach are the ability to select seed points without user intervention and the reduction of the influence of noise. We demonstrate the robustness and effectiveness of our algorithm on different point cloud models and the results show that the segmentation accuracy rate achieves 96%.

Inferring the 3D pose of a character from a drawing is a complex and under-constrained problem. Solving it may help automate various parts of an animation production pipeline such as pre-visualization. In this article, a novel way of inferring the 3D pose from a monocular 2D sketch is proposed. The proposed method does not make any external assumptions about the model, allowing it to be used on different types of characters. The inference of the 3D pose is formulated as an optimization problem and a parallel variation of the Particle Swarm Optimization algorithm called PARAC-LOAPSO is utilized for searching the minimum. Testing in isolation as well as part of a larger scene, the presented method is evaluated by posing a lamp, a horse, and a human character. The results show that this method is robust, highly scalable, and able to be extended to various types of models.

Sketch-based image retrieval is a fundamental computer vision problem. Instead of using hand-designed features to represent sketches and images, recent researches apply deep learning approaches combined with fine-grained matching to retrieve images with fine-grained details. Although these researches allow user to use hand-free sketches drawn for retrieving similar objects, the color matching is ignored which induces a low retrieval precision. To address this problem, we propose a single color sketch-based image retrieval (SCSBIR) approach using HSV color feature considering both shape matching and color matching in this paper. The SCSBIR problem is investigated using deep learning networks, in which deep features are used to represent color sketches and images. A novel ranking method considering both shape matching and color matching is also proposed. In addition, we build a SCSBIR dataset with color sketches and images, and train and test our method by using this dataset. The test results show that our method has a better retrieval performance. The research in this paper can not only promote its application in the commercial field, but also provide reference for the future research in this field.

Addressing the two bottlenecks of present robotic design, low force-to-weight ratio and low integration level, this paper presents a general bio-inspired design methodology supported using a sensor-actuator-structure integrated system fully inspired by natural muscle. The system design is centred on the reproduction of natural muscle's contraction mechanism unveiled by multi-scale researches, and the versatility and morphology of muscle are approximated concurrently through multi-objective and coupled iterative optimizations. The effectiveness of the proposed bio-inspired design methodology is verified by the experiments demonstrating the muscle-like FW ratio and mechanical behaviour of the integrated system.

The reviewers comments on the issues of criteria and sources have been used to develop the literature review.We have included a new section titled Literature review explaining the process and stages involved in searching the literature, selecting the literature, and analysing the literature.The criteria and sources used for each stage have been included in the section of Literature review. The computer animation industry has been booming and prospering in recent thirty years. One of the significant changes faced by this industry is the evolution of computer-animation data and, yet, extant literature has offered very little insights into the evolution process and management issues pertinent to computer- animation data. Hence, many questions have surfaced in the extant literature of computer- animation data management. For example, to what extent has the data content expanded in terms of quantity and quality? To what extent has the information technology used to store and process the data changed? To what extent have the user and the community groups diversified in terms of their nature and number? Knowledge pertaining to these issues can provide new research directions to academics and also insights to practitioners for more effective and innovative management of computer- animation data. This conceptual paper, therefore, takes the pioneering step to address these issues by proposing four factors prudent for examining the evolution phases associated with computer-animation data management: technology, content, users, and community. Next, this paper presents a conceptual framework illustrating the inter- dependent relationships between these four factors together with associated theoretical and managerial issues. This paper, albeit limited by its conceptual nature, advances the extant literature of computer animation, information system, and open-product model.

We propose a semi-Lagrangian method for multiphase interface tracking. In contrast to previous methods, our method maintains an explicit polygonal mesh, which is reconstructed from an unsigned distance function and an indicator function, to track the interface of arbitrary number of phases. The surface mesh is reconstructed at each step using an efficient multiphase polygonization procedure with precomputed stencils while the distance and indicator function are updated with an accurate semi-Lagrangian path tracing from the meshes of the last step. Furthermore, we provide an adaptive data structure, multiphase distance tree, to accelerate the updating of both the distance function and the indicator function. In addition, the adaptive structure also enables us to contour the distance tree accurately with simple bisection techniques. The major advantage of our method is that it can easily handle topological changes without ambiguities and preserve both the sharp features and the volume well. We will evaluate its efficiency, accuracy and robustness in the results part with several examples.

© 2016 The Author(s)Digital techniques have been used to assist narrative and storytelling, especially in many pedagogical practices. With the rapid development of HCI techniques, saturated with digital media in their daily lives, young children, demands more interactive learning methods and meaningful immersive learning experiences. In this paper, we propose a novel hand gesture-based puppetry storytelling system which provides a more intuitive and natural human computer interaction method for young children to develop narrative ability in virtual story world. Depth motion sensing and hand gestures control technology is utilized in the implementation of user-friendly interaction. Young players could intuitively use hand gestures to manipulate virtual puppet to perform story and interact with different items in virtual environment to assist narration. Based on the result of the evaluation, this novel digital storytelling system shows positive pedagogical functions on children’s narrating ability as well as the competencies of cognitive and motor coordination. The usability of the system is preliminary examined in our test, and the results which showed that young children can benefit from playing with Puppet Narrator.

Eye movements have been shown to lead hand movements in tracing tasks where subjects have to move their fingers along a predefined trace. The question remained, whether the leading relationship was similar when tracing with a pointing device, such as a mouse; more importantly, whether tasks that required more or less gaze–mouse coordination would introduce variation in this pattern of behaviour, in terms of both spatial and temporal leading of gaze position to mouse movement. A three-level gaze–mouse coordination demand paradigm was developed to address these questions. A substantial dataset of 1350 trials was collected and analysed. The linear correlation of gaze–mouse movements, the statistical distribution of the lead time, as well as the lead distance between gaze and mouse cursor positions were all considered, and we proposed a new method to quantify lead time in gaze–mouse coordination. The results supported and extended previous empirical findings that gaze often led mouse movements. We found that the gaze–mouse coordination demands of the task were positively correlated to the gaze lead, both spatially and temporally. However, the mouse movements were synchronised with or led gaze in the simple straight line condition, which demanded the least gaze–mouse coordination.

© Springer Nature Switzerland AG 2019. We present a novel integral-based Material Point Method (MPM) using state based peridynamics structure for modeling elastoplastic material and fracture animation. Previous partial derivative based MPM studies face challenges of underlying instability issues of particle distribution and the complexity of modeling discontinuities. To alleviate these problems, we integrate the strain metric in the basic elastic constitutive model by using material point truss structure, which outweighs differential-based methods in both accuracy and stability. To model plasticity, we incorporate our constitutive model with deviatoric flow theory and a simple yield function. It is straightforward to handle the problem of cracking in our hybrid framework. Our method adopts two time integration ways to update crack interface and fracture inner parts, which overcome the unnecessary grid duplication. Our work can create a wide range of material phenomenon including elasticity, plasticity, and fracture. Our framework provides an attractive method for producing elastoplastic materials and fracture with visual realism and high stability.

Virtual characters are 3D geometric models of characters. They have a lot of applications in multimedia. In this paper, we propose a new physics-based deformation method and efficient character modelling framework for creation of detailed 3D virtual character models. Our proposed physics-based deformation method uses PDE surfaces. Here PDE is the abbreviation of Partial Differential Equation, and PDE surfaces are defined as sculpting force-driven shape representations of interpolation surfaces. Interpolation surfaces are obtained by interpolating key cross-section profile curves and the sculpting force-driven shape representation uses an analytical solution to a vector-valued partial differential equation involving sculpting forces to quickly obtain deformed shapes. Our proposed character modelling framework consists of global modeling and local modeling. The global modeling is also called model building, which is a process of creating a whole character model quickly with sketch-guided and template-based modeling techniques. The local modeling produces local details efficiently to improve the realism of the created character model with four shape manipulation techniques. The sketch-guided global modeling generates a character model from three different levels of sketched profile curves called primary, secondary and key cross-section curves in three orthographic views. The template-based global modeling obtains a new character model by deforming a template model to match the three different levels of profile curves. Four shape manipulation techniques for local modeling are investigated and integrated into the new modelling framework. They include: partial differential equation-based shape manipulation, generalized elliptic curve-driven shape manipulation, sketch assisted shape manipulation, and template-based shape manipulation. These new local modeling techniques have both global and local shape control functions and are efficient in local shape manipulation. The final character models are represented with a collection of surfaces, which are modeled with two types of geometric entities: generalized elliptic curves (GECs) and partial differential equation-based surfaces. Our experiments indicate that the proposed modeling approach can build detailed and realistic character models easily and quickly.

We present an automatic approach for the semantic modeling of indoor scenes based on a single photograph, instead of relying on depth sensors. Without using handcrafted features, we guide indoor scene modeling with feature maps extracted by fully convolutional networks. Three parallel fully convolutional networks are adopted to generate object instance masks, a depth map, and an edge map of the room layout. Based on these high-level features, support relationships between indoor objects can be efficiently inferred in a data-driven manner. Constrained by the support context, a global-to-local model matching strategy is followed to retrieve the whole indoor scene. We demonstrate that the proposed method can efficiently retrieve indoor objects including situations where the objects are badly occluded. This approach enables efficient semantic-based scene editing.

Multiple visual features have been proposed and used in 3-dimensional (3D) model retrieval in recent years. Since each visual feature reflects a unique characteristic about the model, they have unequal discriminative power with respect to a specific category of 3D model, and they are complementary to each other in model representation. Thus, it would be beneficial to combine multiple visual features together in 3D model retrieval. In light of this, we propose an efficient Semi-supervised Multiple Feature Fusion (SMFF) method for view-based 3D model retrieval in this paper. Specifically, We first extract multiple visual features to describe both the local and global appearance characteristics of multiple 2D projected images that are generated from 3D models. Then, SMFF is adopted to learn a more compact and discriminative low-dimensional feature representation via multiple feature fusion using both the labeled and unlabeled 3D models. Once the low-dimensional features have been learned, many existing methods such as SVM and KNN can be used in the subsequent retrieval phase. Moreover, an out-of-sample extension of SMFF is provided to calculate the low-dimensional features for the newly added 3D models in linear time. Experiments on two public 3D model datasets demonstrate that using such a learned feature representation can significantly improve the performance of 3D model retrieval and the proposed method outperforms the other competitors.

We present an efficient interactive system for dressing a naked bird with feathers. In our system, a skeleton associated with guide feathers is used to describe the distribution of the body feathers. The special skeleton can be easily built by the user, given a 3D bird model as input. To address the problem of interpenetrations among feathers, the growth priority between the feather roots is defined, with which we obtain the growth order from a greedily constructed directed acyclic graph. Each feather is then adjusted in that order by a height field based collision resolution process. The height field not only provides an efficient way to detect the collision but also enables us to finely control the degree of collision during feather adjustments. The results show that our approach is capable of resolving the collisions among thousands of feathers in a few seconds. If model animation is desired, the feathers can be adjusted on the fly at interactive framerates. Details of our implementation are provided with several examples to demonstrate the effectiveness of our system.

Creation of detailed character models is a very challenging task in animation production. Sketch-based character model creation from a 3D template provides a promising solution. However, how to quickly find correct correspondences between user's drawn sketches and the 3D template model, how to efficiently deform the 3D template model to exactly match user's drawn sketches, and realize real-time interactive modeling is still an open topic. In this paper, we propose a new approach and develop a user interface to effectively tackle this problem. Our proposed approach includes using user's drawn sketches to retrieve a most similar 3D template model from our dataset and marrying human's perception and interactions with computer's highly efficient computing to extract occluding and silhouette contours of the 3D template model and find correct correspondences quickly. We then combine skeleton-based deformation and mesh editing to deform the 3D template model to fit user's drawn sketches and create new and detailed 3D character models. The results presented in this paper demonstrate the effectiveness and advantages of our proposed approach and usefulness of our developed user interface. Copyright © 2015 John Wiley & Sons, Ltd.

Puppetry has been a popular art form for many centuries in different cultures, which becomes a valuable and fascinating heritage assert. Traditional Chinese marionette art with over 2000years history is one of the most representative forms offering a mixture of stage performance of singing, dancing, music, poetry, opera, story narrative and action. Apart from a set of string rules, which controls the dynamics, head carving skill is another important pillar in this art form.

We present a new skin deformation method to create dynamic skin deformations in this paper. The core elements of our approach are a dynamic deformation model, an efficient data-driven finite difference solution, and a curve-based representation of 3D models. We first reconstruct skin deformation models at different poses from the taken photos of a male human arm movement to achieve real deformed skin shapes. Then, we extract curves from these reconstructed skin deformation models. A new dynamic deformation model is proposed to describe physics of dynamic curve deformations, and its finite difference solution is developed to determine shape changes of the extracted curves. In order to improve visual realism of skin deformations, we employ data-driven methods and introduce skin shapes at the initial and final poses into our proposed dynamic deformation model. Experimental examples and comparisons made in this paper indicate that our proposed dynamic skin deformation technique can create realistic deformed skin shapes efficiently with a small data size.

Under what conditions will a bystander intervene to try to stop a violent attack by one person on another? It is generally believed that the greater the size of the crowd of bystanders, the less the chance that any of them will intervene. A complementary model is that social identity is critical as an explanatory variable. For example, when the bystander shares common social identity with the victim the probability of intervention is enhanced, other things being equal. However, it is generally not possible to study such hypotheses experimentally for practical and ethical reasons. Here we show that an experiment that depicts a violent incident at life-size in immersive virtual reality lends support to the social identity explanation. 40 male supporters of Arsenal Football Club in England were recruited for a two-factor between-groups experiment: the victim was either an Arsenal supporter or not (in-group/out-group), and looked towards the participant for help or not during the confrontation. The response variables were the numbers of verbal and physical interventions by the participant during the violent argument. The number of physical interventions had a significantly greater mean in the in-group condition compared to the out-group. The more that participants perceived that the Victim was looking to them for help the greater the number of interventions in the in-group but not in the out-group. These results are supported by standard statistical analysis of variance, with more detailed findings obtained by a symbolic regression procedure based on genetic programming. Verbal interventions made during their experience, and analysis of post-experiment interview data suggest that in-group members were more prone to confrontational intervention compared to the out-group who were more prone to make statements to try to diffuse the situation.

The aim of this full-day workshop in conjunction with SIGGRAPH Asia 2017 is to bring together researchers from diverse backgrounds, such as computer graphics, computer vision, virtual reality, human computer interactions and machine learning, with a common interest in data-driven, realistic animation. Interests in this emerging topic may stem from a variety of sources, e.g. point cloud data collected from laser scanners or RGBD cameras such as KINECT, high resolution geometry reconstructed by Structure from Motion (SfM), motion capture devices including high frame-rate optical trackers and IMUs, GPS data from millions of mobile devices etc. Despite the high dimensionality and the huge volume of the dataset, the boost in machine learning and big data technologies are allowing researchers to extract important features from them for which can be applied for data analysis, synthesis and editing.

Surface blending with tangential continuity is most widely applied in computer-aided design, manufacturing systems, and geometric modeling. In this paper, we propose a new blending method to effectively control the shape of blending surfaces, which can also satisfy the blending constraints of tangent continuity exactly. This new blending method is based on the concept of swept surfaces controlled by a vector-valued fourth order ordinary differential equation (ODE). It creates blending surfaces by sweeping a generator along two trimlines and making the generator exactly satisfy the tangential constraints at the trimlines. The shape of blending surfaces is controlled by manipulating the generator with the solution to a vector-valued fourth order ODE. This new blending methods have the following advantages: (1) exact satisfaction of $$C^1$$ C 1 continuous blending boundary constraints, (2) effective shape control of blending surfaces, (3) high computing efficiency due to explicit mathematical representation of blending surfaces, and (4) ability to blend multiple (more than two) primary surfaces.

Character motion synthesis is the process of artificially generating natural motion for a virtual character. In film, motion synthesis can be used to generate difficult or dangerous stunts without putting performers at risk. In computer games and virtual reality, motion synthesis enriches the player or participant experience by allowing for unscripted and emergent character behavior. In each of these applications the ability to adapt to changes to environmental conditions or to the character in a smooth and natural manner, while still conforming with user-specified constraints, determines the utility of a method to animators and industry practitioners. This focus on adaptation capability distinguishes our survey from other reviews which focus on general technology developments. Three main methodologies (example-based; simulation-based and hybrid) are summarised and evaluated using compound metrics: adaptivity, naturalness and controllability. By assessing existing techniques according to this classification we are able to determine how well a method corresponds to users' expectations. We discuss optimization strategies commonly used in motion synthesis literature, and also contemporary perspectives from biology which give us a deeper insight into this problem. We also present observations and reflections from industry practitioners to reveal the operational constraints of character motion synthesis techniques. Our discussion and review presents a unique insight into the subject, and provide essential guidance when selecting appropriate methods to design an adaptive motion controller.

Background Virtual-reality (VR) based simulation techniques offer an efficient and low cost alternative to conventional surgery training. This article describes a VR training and assessment system in laparoscopic rectum surgery. Methods To give a realistic visual performance of interaction between membrane tissue and surgery tools, a generalized cylinder based collision detection and a multi-layer mass–spring model are presented. A dynamic assessment model is also designed for hierarchy training evaluation. Results With this simulator, trainees can operate on the virtual rectum with both visual and haptic sensation feedback simultaneously. The system also offers surgeons instructions in real time when improper manipulation happens. The simulator has been tested and evaluated by ten subjects. Conclusions This prototype system has been verified by colorectal surgeons through a pilot study. They believe the visual performance and the tactile feedback are realistic. It exhibits the potential to effectively improve the surgical skills of trainee surgeons and significantly shorten their learning curve. Copyright © 2014 John Wiley & Sons, Ltd.

FeSex/Bi2Se3 bilayer thin films were grown by RF magnetron sputtering on silicon substrates with different thicknesses of Bi2Se3 and the structural, morphological and magnetic properties were investigated. FeSex/Bi2Se3 bilayer films had Bi2Se3 crystallites oriented with c-axis perpendicular to the film plane, and exhibited weak ferromagnetism at low temperature due to the ferromagnetic FeSe2. The thickness of Bi2Se3 layer affected both crystalline structure of Fe–Se layer and the magnetic property.

We present a novel framework to compute geodesics on implicit surfaces and point clouds. Our framework consists of three parts, particle based approximate geodesics on implicit surfaces, Cartesian grid based approximate geodesics on point clouds, and geodesic correction. The first two parts can effectively generate approximate geodesics on implicit surfaces and point clouds, respectively. By introducing the geodesic curvature flow, the third part produces smooth and accurate geodesic solutions. Differing from most of the existing methods, our algorithms can converge to a given tolerance. The presented computational framework is suitable for arbitrary implicit hypersurfaces or point clouds with high genus or high curvature.

In this paper, we develop a new approach to animate skin deformation of character models. It aims to combine the strengths of joint-based approaches, physics-based algorithms and curve-based surface modeling methods together for efficient and realistic animation of skin deformation. We first transform the deformation of skin surfaces of character models into that of the curves defining the skin surfaces, and introduce a mathematical model consisting of a vector-valued fourth order ordinary differential equation and boundary conditions to describe the curve deformation. In order to achieve capacity and high animation efficiency, we propose an efficient finite difference solution of the mathematical model, and apply our proposed solution to animate skin deformation of character models. The application examples demonstrate that our proposed approach can create realistic skin deformations for real-time character animation.

Background Previous studies showed improvement in heart function by injecting bone marrow mesenchymal stem cells (BMSCs) after AMI. Emerging evidence suggested that both the number and function of BMSCs decline with ageing. We designed a randomized, controlled trial to further investigate the safety and efficacy of this treatment. Methods Patients with ST-elevation AMI undergoing successful reperfusion treatment within 12hours were randomly assigned to receive an intracoronary infusion of BMSCs (n=21) or standard medical treatment (n=22) (the numbers of patients were limited because of the complication of coronary artery obstruction). Results There is a closely positive correlation of the number and function of BMSCs vs. the cardiac function reflected by LVEF at baseline (r=0.679, P=0.001) and at 12-month follow-up (r=0.477, P=0.039). Six months after cell administration, myocardial viability within the infarct area by 18-FDG SPECT was improved in both groups compared with baseline, but no significant difference in the BMSCs compared with control groups (4.0±0.4% 95%CI 3.1–4.9 vs. 3.2±0.5% 95%CI 2.1–4.3, P=0.237). 99mTc-sestamibi SPECT demonstrated that myocardial perfusion within the infarct area in the BMSCs did not differ from the control group (4.4±0.5% 95%CI 3.2–5.5 vs. 3.9±0.6% 95%CI 2.6–5.2, P=0.594). Similarly, LVEF after 12 and 24months follow-up did not show any difference between the two groups. In the BMSCs group, one patient suffered a serious complication of coronary artery occlusion during the BMSCs injection procedure. Conclusions The clinical benefits of intracoronary injection of autologous BMSCs in acute STEMI patients need further investigation and reevaluation.