Search

The current researches trend to adopt a low-resolution hot spot map to restore the original high-resolution representation to save computing cost, resulting in unsatisfactory detection performance, especially in human body recognition with a highly complex background. Aiming at this problem, we proposed a model of parallel connection of multiple sub-networks with different resolution levels on a high-resolution main network. It can maintain the network structure of a high-resolution hot spot map in the whole operation process. By using this structure in the human key point vector field network, the accuracy of human posture recognition has been improved with high-speed operation. To validate the proposed model’s effectiveness, two common benchmark data sets of COCO key point data set and MPII human posture data set are used for evaluation. Experimental results show that our network achieves the accuracy of 72.3% AP and 92.2% AP in the two data sets, respectively, which is 3%-4% higher than those of the existing mainstream researches. In our test, only the accuracy of backbone’s SimpleBaseline with ResNet-152 is close to ours, yet our network realized a much lower computing cost.

Remote patient monitoring is one of the most reliable choices for the availability of health care services for the elderly and/or chronically ill. Rehabilitation requires the exact and medically correct completion of physiotherapy activities. This paper presents BiomacVR, a virtual reality (VR)-based rehabilitation system that combines a VR physical training monitoring environment with upper limb rehabilitation technology for accurate interaction and increasing patients’ engagement in rehabilitation training. The system utilises a deep learning motion identification model called Convolutional Pose Machine (CPM) that uses a stacked hourglass network. The model is trained to precisely locate critical places in the human body using image sequences collected by depth sensors to identify correct and wrong human motions and to assess the effectiveness of physical training based on the scenarios presented. This paper presents the findings of the eight most-frequently used physical training exercise situations from post-stroke rehabilitation methodology. Depth sensors were able to accurately identify key parameters of the posture of a person performing different rehabilitation exercises. The average response time was 23 ms, which allows the system to be used in real-time applications. Furthermore, the skeleton features obtained by the system are useful for discriminating between healthy (normal) subjects and subjects suffering from lower back pain. Our results confirm that the proposed system with motion recognition methodology can be used to evaluate the quality of the physiotherapy exercises of the patient and monitor the progress of rehabilitation and assess its effectiveness.

This paper provides an in-depth study and analysis of human artistic poses through intelligently enhanced multimodal artistic pose recognition. A complementary network model architecture of multimodal information based on motion energy proposed. The network exploits both the rich information of appearance features provided by RGB data and the depth information provided by depth data as well as the characteristics of robustness to luminance and observation angle. The multimodal fusion is accomplished by the complementary information characteristics of the two modalities. Moreover, to better model the long-range temporal structure while considering action classes with sub-action sharing phenomena, an energy-guided video segmentation method is employed. And in the feature fusion stage, a cross-modal cross-fusion approach is proposed, which enables the convolutional network to share local features of two modalities not only in the shallow layer but also to obtain the fusion of global features in the deep convolutional layer by connecting the feature maps of multiple convolutional layers. Firstly, the Kinect camera is used to acquire the color image data of the human body, the depth image data, and the 3D coordinate data of the skeletal points using the Open pose open-source framework. Then, the action automatically extracted from keyframes based on the distance between the hand and the head, and the relative distance features are extracted from the keyframes to describe the action, the local occupancy pattern features and HSV color space features are extracted to describe the object, and finally, the feature fusion is performed and the complex action recognition task is completed. To solve the consistency problem of virtual-reality fusion, the mapping relationship between hand joint point coordinates and the virtual scene is determined in the augmented reality scene, and the coordinate consistency model of natural hand and virtual model is established; finally, the real-time interaction between hand gesture and virtual model is realized, and the average correct rate of its hand gesture reaches 99.04%, which improves the robustness and real-time interaction of hand gesture recognition.

Nowadays, wearable devices are becoming more and more widely used. The development of interfaces for wearable devices is a challenge due to their small size on the one hand and on the other hand - their increasing capabilities (respectively functions) that must be managed by their interface. The purpose of the presented study is to propose a method for recognizing the pose of the user (lying, sitting, standing) when waking a wearable device from sleep mode. The experimental study and results are given.

The desire to remain living in one&rsquo, s own home rather than a care home by those in need of 24/7 care is one that requires a level of understanding for the actions of an environment&rsquo, s inhabitants. This can potentially be accomplished with the ability to recognise Activities of Daily Living (ADLs), however, this research focuses first on producing an unobtrusive solution for pose recognition where the preservation of privacy is a primary aim. With an accurate manner of predicting an inhabitant&rsquo, s poses, their interactions with objects within the environment and, therefore, the activities they are performing, can begin to be understood. This research implements a Convolutional Neural Network (CNN), which has been designed with an original architecture derived from the popular AlexNet, to predict poses from thermal imagery that have been captured using thermopile infrared sensors (TISs). Five TISs have been deployed within the smart kitchen in Ulster University where each provides input to a corresponding trained CNN. The approach is evaluated using an original dataset and an F1-score of 0.9920 was achieved with all five TISs. The limitations of utilising a ceiling-based TIS are investigated and each possible permutation of corner-based TISs is evaluated to satisfy a trade-off between the number of TISs, the total sensor cost and the performances. These tests are also promising as F1-scores of 0.9266, 0.9149 and 0.8468 were achieved with the isolated use of four, three, and two corner TISs, respectively.

Continuous monitoring of frail individuals for detecting dangerous situations during their daily living at home can be a powerful tool toward their inclusion in the society by allowing living independently while safely. To this goal we developed a pose recognition system tailored to disabled students living in college dorms and based on skeleton tracking through four Kinect One devices independently recording the inhabitant with different viewpoints, while preserving the individual’s privacy. The system is intended to classify each data frame and provide the classification result to a further decision-making algorithm, which may trigger an alarm based on the classified pose and the location of the subject with respect to the furniture in the room. An extensive dataset was recorded on 12 individuals moving in a mockup room and undertaking four poses to be recognized: standing, sitting, lying down, and “dangerous sitting.” The latter consists of the subject slumped in a chair with his/her head lying forward or backward as if unconscious. Each skeleton frame was labeled and represented using 10 discriminative features: three skeletal joint vertical coordinates and seven relative and absolute angles describing articular joint positions and body segment orientation. In order to classify the pose of the subject in each skeleton frame we built a two hidden layers multi-layer perceptron neural network with a “SoftMax” output layer, which we trained on the data from 10 of the 12 subjects (495,728 frames), with the data from the two remaining subjects representing the test set (106,802 frames). The system achieved very promising results, with an average accuracy of 83.9% (ranging 82.7 and 94.3% in each of the four classes). Our work proves the usefulness of human pose recognition based on machine learning in the field of safety monitoring in assisted living conditions.

A video-based fall detection system was presented; which consists of data acquisition, image processing, feature extraction, feature selection, classification and finite state machine. A two-dimensional human posture image was represented by 12 features extracted from the generalisation of a silhouette shape to a quadrilateral. The corresponding feature vectors for three groups of human pose were statistically analysed by using a nonparametric Kruskal Wallis test to assess the different significance level between them. From the statistical test, non-significant features were discarded. Four selected kernel-based Support Vector Machine: linear, quadratics, cubic and Radial Basis Function classifiers were trained to classify three human posture groups. Among four classifiers, the last one performed the best in terms of performance matric on testing set. The classifier outperformed others with high achievement ofaverage sensitivity, precision and F-score of 99.19%, 99.25% and 99.22%, respectively. Such pose classification model output was further used in a simple finite state machine to trigger the falling event alarms. The fall detection system was tested on different fall video sets and able to detect the presence offalling events in a frame sequence of videos with accuracy of 97.32% and low computional time.

In recent years, virtual reality (VR) technologies have been increasingly used for teaching motion skills to learners. In this paper, the authors employed a VR assistive system for teaching motion skills to learners by the use of an inertial sensor-embedded head-mount-display (HMD). As a step of the development, we studied a motion instruction method using “Head Motion-Associated virtual stereo Rearview (HMAR in short)”, and conducted a study on pose-recognition under a time-consuming vision-restricted condition. Under this condition, subjects were to ensure their remembrance only by vision and taking enough time, and not by using proprioception. The time consuming condition is considered to be antithetical to the instantaneous less time consuming condition, and is expected to contribute to deepening the understanding of the effect of the HMAR. In the experiment, reference poses are displayed to learners with the use of a VR system. In the system, the learners observe the virtual stereo rearview via HMD, and perceive and reproduce the displayed reference poses. Here, the virtual stereo camera that is assumed to observe the reference avatar is associated with the learner’s head motion. The virtual stereo camera is moved around the reference avatar away from the back of the avatar’s head in accordance with the head-rotating motion. As the HMAR was compared with two representative ordinary methods, i.e., a key-switched rearview (KSR) and a mouse-associated rearview (MAR), the elapsed time of the HMAR showed significantly smaller variance, although did not show any significant difference in the mean.

When we see a human sitting down, standing up, or walking, we can recognize one of these poses independently of the individual, or we can recognize the individual person, independently of the pose. The same issues arise for deforming objects. For example, if we see a flag deformed by the wind, either blowing out or hanging languidly, we can usually recognize the flag, independently of its deformation; or we can recognize the deformation independently of the identity of the flag. We hypothesize that these types of recognition can be implemented by the primate visual system using temporo-spatial continuity as objects transform as a learning principle. In particular, we hypothesize that pose or deformation can be learned under conditions in which large numbers of different people are successively seen in the same pose, or objects in the same deformation. We also hypothesize that person-specific representations that are independent of pose, and object-specific representations that are independent of deformation and view, could be built, when individual people or objects are observed successively transforming from one pose or deformation and view to another. These hypotheses were tested in a simulation of the ventral visual system, VisNet, that uses temporal continuity, implemented in a synaptic learning rule with a short-term memory trace of previous neuronal activity, to learn invariant representations. It was found that depending on the statistics of the visual input, either pose-specific or deformation-specific representations could be built that were invariant with respect to individual and view; or that identity-specific representations could be built that were invariant with respect to pose or deformation and view. We propose that this is how pose-specific and pose-invariant, and deformation-specific and deformation-invariant, perceptual representations are built in the brain.

The goal of this thesis is to research detection and tracking of a single person with just one camera and recognize and track human poses in order to improve the performance of an interactive spatial game that uses the horizontal position of the person and the performed poses for its control. Of course the results achieved with this research are not attached to this application only but to any application that can be imagined for using human poses detection as input. The use of one single camera restricts considerably the depth information and it is more challenging for detection and tracking when self occlusions occur but, on the other hand, it has the advantage that you only need a simple camera to run applications of this kind so it is interesting for commercial use on laptops or personal computers with a web camera. These types of games controlled by human poses can develop physical aptitudes and coordination while having fun playing with them and also re lieves you from the load of holding a mouse or stick; it is just you and the screen.

International audience; Human gesture recognition is an active topic of vision research which has applications in diverse fields such as collaborative virtual environments and robot tele-operation. We propose a novel method for the recognition of hand gestures, used by air marshals for steering aircraft on the runway, using the Radon transform. Various aspects of the algorithm, including acquisition, segmentation, labeling and recognition using the parametric Radon transform are addressed in this paper. A binary skeleton representation of the human subject is computed. The Radon transform is used to generate maxima corresponding to specific orientations of the skeletal representation. Feature vectors are extracted from the transform space by computing the normalized cumulative projections of the Radon transform on the angle axis. Kmeans clustering is then applied to recognize static gestures from the extracted features. This technique has the potential to provide information about the exact orientation of gesture segments and can find use in ground control of unmanned air vehicles. Experiments with image data corresponding to the various ground air traffic control gestures used in directing aircrafts, highlight the potential application of this approach.