Search

BCI (brain computer interface) is one of the important research methods in the fields of brain cognition, brain medicine and brain-like research, where the precise implantation of microelectrodes is a necessary prerequisite and important guarantee. With the rapid development of robotics, machine vision and artificial intelligence, surgical robots are gradually used in brain-computer interface implantation surgery. To meet the demand for precise implantation of microelectrodes in the somatosensory and cerebral motor cortex of SD (Sprague-Dawley) rats, a vision-guided precision implantation system for brain-computer interface microelectrodes was presented. Based on the method of machine vision to identify the key points of the rat skull, the coordinate system was established to obtain the point cloud information of the rat skull, to realize the high-precision identification and localization of the target points, and to guide the actuator to complete the electrode implantation operation. Through model analysis and animal experiments, it has been demonstrated that the implantation system can accurately identify surgical targets on the subjects, guide the actuator to swiftly penetrate the skull, and accurately and stably implant the electrodes into the target area, which effectively improves the accuracy of microelectrode implantation. [ABSTRACT FROM AUTHOR]

This paper aims to use deep learning algorithms to identify and study the stability of breakwater armor blocks. It introduces a posture identification model for fender blocks using a Mask Region-based Convolutional Neural Network (R-CNN), which has been enhanced by considering factors affecting breakwater fender blocks. Furthermore, a wave prediction model for breakwaters is developed by integrating Bidirectional Encoder Representations from Transformers (BERTs) with Bidirectional Long Short-Term Memory (BiLSTM). The performance of these models is evaluated. The results show that the accuracy of the Mask R-CNN and its comparison algorithms initially increases and then decreases with higher Intersection Over Union (IOU) thresholds, peaking at 95.16% accuracy at an IOU threshold of 0.5. The BERT-BiLSTM wave prediction model maintains a loss value around 0.01 and an accuracy of approximately 90.00%. These results suggest that the proposed models offer more accurate stability assessments of breakwater armor blocks. By combining the random forest prediction model with BiLSTM, the wave characteristics and fender posture can be predicted better, offering reliable decision support for breakwater engineering. [ABSTRACT FROM AUTHOR]

With the continuous advancement of autonomous driving and monitoring technologies, there is increasing attention on non-intrusive target monitoring and recognition. This paper proposes an ArcFace SE-attention model-agnostic meta-learning approach (AS-MAML) by integrating attention mechanisms into residual networks for pedestrian gait recognition using frequency-modulated continuous-wave (FMCW) millimeter-wave radar through meta-learning. We enhance the feature extraction capability of the base network using channel attention mechanisms and integrate the additive angular margin loss function (ArcFace loss) into the inner loop of MAML to constrain inner loop optimization and improve radar discrimination. Then, this network is used to classify small-sample micro-Doppler images obtained from millimeter-wave radar as the data source for pose recognition. Experimental tests were conducted on pose estimation and image classification tasks. The results demonstrate significant detection and recognition performance, with an accuracy of 94.5%, accompanied by a 95% confidence interval. Additionally, on the open-source dataset DIAT-μRadHAR, which is specially processed to increase classification difficulty, the network achieves a classification accuracy of 85.9%. [ABSTRACT FROM AUTHOR]

Manual picking cannot fully meet the large-scale production in China at present. Robotic picking has been an inevitable trend, particularly with the shortage of labor resources and the rapid development of mechanical automation. It is very necessary to accurately identify and position the apples in the complex environments. Fruit attitude fusion acquisition can be synchronously realized and then classified the apple information. Sometimes, only a small portion of target fruit is covered from the orchard environment, including the leaves, branches, and fruits. There are the small differences among the fruit growth patterns. The convolutional neural network is easy to cause the deep feature map, and then lose the key information of fruit covering parts after multiple convolution operations, resulting in the misrecognition of the fruit growth pattern. At the same time, the detection network can easily identify two apples as one for the overlapping fruits in the natural environment, thus causing the omission of the occluded fruits. In this study, an improved YOLOv7 model was proposed to recognize the apple posture from the growth morphologies. Firstly, the multi-scale feature fusion network was improved to add a 160×160 feature scale layer in the backbone network. The sensitivity of the model was enhanced to identify the tiny local features; Secondly, CBAM attention mechanism was introduced to improve the target region of interest; Finally, the Soft-NMS was used to effectively avoid the high-density overlapping targets being suppressed at one time, thus reducing the missed detection. The experimental results show that the recognition accuracy, recall and average recognition precision of DCS-YOLOv7 were 86.9%, 80.5% and 87.1%, respectively, which were 4.2%, 2.2% and 3.7% higher than the original YOLOv7 model. The average accuracy and speed were greatly improved to fully meet the requirements of picking robot. In addition, an apple gesture recognition was proposed using semantic segmentation and the minimum outer join features. Firstly, comparison tests showed that the Unet model exhibited the best performance in apple image segmentation. The average pixel accuracies were 0.7 and 0.2 percentage points higher than those of DeepLabv3+ and PSPNet. The average intersection and merger ratios were 1.6 and 1.1 percentage points higher as well. The average speed of segmentation also outperformed the rest. As such, the UNet instance segmentation network was chosen as the apple segmentation model. The apple image was segmented using UNet semantic segmentation network. The apple and calyx contour features were obtained by the contour extraction, and then the pose of unobstructed apple was obtained using the apple minimum external feature. The accuracy was 94% to detect the apple pose. The average processing time for each image was 15.7ms, indicating the better acquisition for the pose of apple target. The validity and correctness of recognition model were verified with the high detection accuracy to integrate the recognition of fruit growth pattern and posture. The recognition of fruit posture was considered to classify the growth pattern of apples. The end effector can rapidly and accurately pick the fruits in a suitable way. The finding can lay the foundation for the non-destructive and efficient picking of apples. [ABSTRACT FROM AUTHOR]

Users watch videos for a long time when they are learning or entertaining. They may inevitably be tired, doze off or leave temporarily. However, the videos continue to play. When the users want to watch the video again, they must return to find the appropriate restarting position, causing inconvenience. The ultimate need of this study is to implement an effective video playback control system to automatically pause video playback when a user leaves the seat or falls asleep, while the system continues to play videos when the user sits in front of the computer and is in good condition. The proposed system recognizes human body movements and the opening/closing of the eyes (OCE). First, the user's image is captured through a web camera. Then the OpenPose deep-learning neural network recognizes the human pose. The recognized results are used to determine whether the user leaves or lies on her/his stomach. Therefore, the video can be paused if the user falls asleep while sitting with his eyes closed. The novelty of this study is that the proposed playback control system automatically pauses the video when the user is not in good condition. Accordingly, the user is free from wasting time searching for a proper playback position when the user wants to continue watching the video. The experimental results show that the accuracy rates of body motion recognition can reach 99.5%, and the accuracy rate of eyes closed recognition can reach 99.58%. Consequently, the proposed system can effectively control video playback in practice. [ABSTRACT FROM AUTHOR]

Given the limitations of traditional methods in automatic feature extraction, the study aims to improve the accuracy of posture recognition in adolescent running training through deep learning techniques. This study first constructed a human pose motion model to capture complex pose changes during the motion process. Furthermore, an algorithm framework combining convolutional neural networks and long short-term memory networks was proposed, and optimized by integrating attention modules to enhance the model’s ability to capture key action details and improve recognition accuracy. The outcomes indicated that the recognition accuracy of the raw data processed using the convolutional neural network algorithm is 0.93, which is improved to 0.96 after preprocessing, while the accuracy of the long-short-term memory network algorithm is 0.91 on the raw data and 0.95 after preprocessing. The Inception combined with long-short-term memory network algorithm optimised by incorporating the attention module has the highest accuracy of 0.98 on the preprocessed data, while that on the 0.95 on the raw data, and 1.0 on specific actions such as running and riding. It can be found that the attention module significantly improves the recognition accuracy of the algorithm and its ability to recognise complex actions. This study not only improves the way that teenagers train to run, but it also creates new opportunities for the use of deep learning methods in the context of sports-assisted training.

Manual laborers are often at an increased risk for work-related musculoskeletal disorders (WMSDs) due to improper work postures or the lifting of excessively heavy objects. Therefore, conducting an effective ergonomic assessment of workers is crucial for enhancing productivity and minimizing the occurrence of WMSDs. The Ergonomic Posture Risk Assessment (EPRA) is a widely used method for this assessment. However, traditional EPRA methods rely on human or sensor input, leading to subjective bias, instability, and reduced accuracy. This study addresses these issues by proposing an objective machine-learning approach. It employs a non-intrusive computer vision technique for posture capture, enabling rapid analysis of the worker’s activities through Random Forest analysis. The dataset for risk assessment is generated from the worker’s skeletal joint posture data, collected using Movenet Thunder in conjunction with an inertial motion capture device and the Rapid Entire Body Assessment (REBA) standard. The Particle Swarm Optimization Random Forest (PSO-RF) model is then utilized to predict risk scores for various postures, incorporating limb length ratios to tackle challenges associated with observing torsional joints. The model’s effectiveness in detecting poor posture is subsequently evaluated. The findings indicate that the PSO-RF model successfully identifies poor postures and computes the corresponding REBA scores with 89% accuracy, 93% precision, 91% F1 score, 89% recall, and a kappa value of 82%. This research demonstrates that the machine learning approach, utilizing computer vision and Random Forest, can effectively conduct EPRA to prevent musculoskeletal injuries, providing a data-driven and accurate method for enhancing workplace safety and health.

Abstract Three-dimensional (3D) human pose recognition techniques based on spatial data have gained attention. However, existing models and algorithms fail to achieve desired precision. We propose a 3D human motion pose recognition method using deep contrastive learning and an improved Transformer. The improved Transformer removes noise between human motion RGB and depth images, addressing orientation correlation in 3D models. Two-dimensional (2D) pose features are extracted from de-noised RGB images using a kernel generation module in a graph convolutional network (GCN). Depth features are extracted from de-noised depth images. The 2D pose features and depth features are fused using a regression module in the GCN to obtain 3D pose recognition results. The results demonstrate that the proposed method captures RGB and depth images, achieving high recognition accuracy and fast speed. The proposed method demonstrates good accuracy in 3D human motion pose recognition.

Badminton is a very popular subject in Physical Education (PE). Many students enroll badminton courses in every semester which pose a tremendous teaching load to the instructors. The one-on-one guiding/feedback time provided by the instructor to each student is also greatly reduced. To overcome this challenge, some studies have tried to adopt pose recognition technique in teaching badminton. However, the lack of mobility and recognition accuracy problems hinder its applicability. To address this issue, a new pose recognition technique, OpenPose, was employed to develop a real-time pose recognition badminton teaching APP in this study. The APP was then installed on a mobile device to enhance badminton smash skill learning performance. The scores distribution of the experimental group shows majority of students achieved satisfactory performance, this implies the badminton teaching APP is helpful to all students no matter what their initial skill levels are. [ABSTRACT FROM AUTHOR]

Linear conveyors, traditional tools for cargo transportation, have faced criticism due to their directional constraints, inability to adjust poses, and single-item conveyance, making them unsuitable for modern flexible logistics demands. This paper introduces a platform designed to convey and adjust cargo boxes according to their spatial positions and orientations. Additionally, a cargo pose recognition algorithm that integrates image and point cloud data are presented. By aligning depth camera data, the axis-aligned bounding box (AABB) point serves as the image's region of interest (ROI). Peaks extracted from the image's Hough transform are refined using RANSAC-based point cloud linear fitting, then integrated with the point cloud's oriented bounding box (OBB). Notably, the algorithm eliminates the need for deep learning and registration, enabling its use in rectangular cargo boxes of various sizes. A comparative experiment using accelerometer sensors for pose acquisition revealed a deviation of <0.7° between the two processes. Throughout the real-time adjustments controlled by the experimental platform, cargo angles consistently remained stable. The proposed two-dimensional conveyance platform, compared to existing methods, exhibits simplicity, accurate recognition, enhanced flexibility, and wide applicability. [ABSTRACT FROM AUTHOR]

The emotional context of a given environment can profoundly influence an individual’s feelings and responses. Nonetheless, current emotion recognition methodologies primarily concentrate on analyzing the target subject’s features and inadequately integrate these features with the contextual information of the scene. To tackle this challenge, we introduce a novel emotion recognition model that employs three independent and prioritized deep convolutional neural networks, alongside a feature fusion enhancement technique, to effectively merge facial information, body pose information, and subject features within the overall image. By amalgamating the performance of object detection models and deep convolutional network models, our framework capitalizes on the strengths of multiple approaches. Experiments with the Emotic dataset validate that our proposed model is technically innovative and surpasses existing methods and benchmark models in terms of feature fusion performance. Moreover, our evaluation of the proposed method on the Emotic dataset underscores the significance of environmental contextual information in shaping human emotions.

Pose recognition in character animations is an important avenue of research in computer graphics. However, the current use of traditional artificial intelligence algorithms to recognize animation gestures faces hurdles such as low accuracy and speed. Therefore, to overcome the above problems, this paper proposes a real-time 3D pose recognition system, which includes both facial and body poses, based on deep convolutional neural networks and further designs a single-purpose 3D pose estimation system. First, we transformed the human pose extracted from the input image to an abstract pose data structure. Subsequently, we generated the required character animation at runtime based on the transformed dataset. This challenges the conventional concept of monocular 3D pose estimation, which is extremely difficult to achieve. It can also achieve real-time running speed at a resolution of 384 fps. The proposed method was used to identify multiple-character animation using multiple datasets (Microsoft COCO 2014, CMU Panoptic, Human3.6M, and JTA). The results indicated that the improved algorithm improved the recognition accuracy and performance by approximately 3.5% and 8–10 times, respectively, which is significantly superior to other classic algorithms. Furthermore, we tested the proposed system on multiple pose-recognition datasets. The 3D attitude estimation system speed can reach 24 fps with an error of 100 mm, which is considerably less than that of the 2D attitude estimation system with a speed of 60 fps. The pose recognition based on deep learning proposed in this study yielded surprisingly superior performance, proving that the use of deep-learning technology for image recognition has great potential. [ABSTRACT FROM AUTHOR]

Copyright of Machine Tool & Hydraulics is the property of Guangzhou Mechanical Engineering Research Institute (GMERI) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Remote patient monitoring is one of the most reliable choices for the availability of health care services for the elderly and/or chronically ill. Rehabilitation requires the exact and medically correct completion of physiotherapy activities. This paper presents BiomacVR, a virtual reality (VR)-based rehabilitation system that combines a VR physical training monitoring environment with upper limb rehabilitation technology for accurate interaction and increasing patients' engagement in rehabilitation training. The system utilises a deep learning motion identification model called Convolutional Pose Machine (CPM) that uses a stacked hourglass network. The model is trained to precisely locate critical places in the human body using image sequences collected by depth sensors to identify correct and wrong human motions and to assess the effectiveness of physical training based on the scenarios presented. This paper presents the findings of the eight most-frequently used physical training exercise situations from post-stroke rehabilitation methodology. Depth sensors were able to accurately identify key parameters of the posture of a person performing different rehabilitation exercises. The average response time was 23 ms, which allows the system to be used in real-time applications. Furthermore, the skeleton features obtained by the system are useful for discriminating between healthy (normal) subjects and subjects suffering from lower back pain. Our results confirm that the proposed system with motion recognition methodology can be used to evaluate the quality of the physiotherapy exercises of the patient and monitor the progress of rehabilitation and assess its effectiveness. [ABSTRACT FROM AUTHOR]

The 6-axis external fixation mechanism with Gough-Stewart configuration has been widely applied to the correction of long bone deformities in orthopedics. Pose recognition of the mechanism is essential for trajectory planning of bone correction, but is usually implemented by the surgeons' experience, resulting in a relatively low level of correction accuracy. This paper proposes a pose recognition method based on novel image markers, and implements accuracy analysis. Firstly, a pose description of the mechanism is established with several freely installed markers, and the layout of the markers is also parametrically described. Then, a pose recognition method is presented by identifying the orientation and position parameters using the markers. The recognition method is general in that it encompasses all possible marker layouts, and the recognition accuracy is investigated by analyzing variations in the marker layout. On this basis, layout principles for markers that achieve a desired recognition accuracy are established, and an error compensation strategy for precision improvement is provided. Finally, experiments were conducted. The results show that volume errors of pose recognition were 0.368 ± 0.130 mm and 0.151 ± 0.045°, and the correction accuracy of the fracture model after taking compensation was 0.214 ± 0.573 mm and −0.031 ± 0.161°, validating the feasibility and accuracy of the proposed methods. [ABSTRACT FROM AUTHOR]

As the world's population is aging and there is a shortage of sufficient caring manpower, the development of intelligent care robots is a feasible solution. At present, plenty of care robots have been developed, but humanized care robots that can suitably respond to the individual behaviors of elderly people, such as pose, expression, gaze, and speech are generally lacking. To achieve the interaction, the main objectives of this study are: (1) conducting a literature review and analyzing the status quo on the following four core tasks of image and speech recognition technology: human pose recognition, human facial expression recognition, eye gazing recognition, and Chinese speech recognition; (2) proposing improvement strategies for these tasks based on the results of the literature review. The results of the study on these improvement strategies will provide the basis for using human facial expression robots in elderly care. [ABSTRACT FROM AUTHOR]

The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experimental operation. In this paper, a control system for desktop experimental manipulators based on an audio-visual information fusion algorithm was designed. The robot could replace the operator to complete some tedious and dangerous experimental work by teaching it the arm movement skills. The system is divided into two parts: skill acquisition and movement control. For the former, the visual signal was obtained through two algorithms of motion detection, which were realized by an improved two-stream convolutional network; the audio signal was extracted by Voice AI with regular expressions. Then, we combined the audio and visual information to obtain high coincidence motor skills. The accuracy of skill acquisition can reach more than 81%. The latter employed motor control and grasping pose recognition, which achieved precise controlling and grasping. The system can be used for the teaching and control work of chemical experiments with specific processes. It can replace the operator to complete the chemical experiment work while greatly reducing the programming threshold and improving the efficiency. [ABSTRACT FROM AUTHOR]

Pose recognition in character animations is an important avenue of research in computer graphics. However, the current use of traditional artificial intelligence algorithms to recognize animation gestures faces hurdles such as low accuracy and speed. Therefore, to overcome the above problems, this paper proposes a real-time 3D pose recognition system, which includes both facial and body poses, based on deep convolutional neural networks and further designs a single-purpose 3D pose estimation system. First, we transformed the human pose extracted from the input image to an abstract pose data structure. Subsequently, we generated the required character animation at runtime based on the transformed dataset. This challenges the conventional concept of monocular 3D pose estimation, which is extremely difficult to achieve. It can also achieve real-time running speed at a resolution of 384 fps. The proposed method was used to identify multiple-character animation using multiple datasets (Microsoft COCO 2014, CMU Panoptic, Human3.6M, and JTA). The results indicated that the improved algorithm improved the recognition accuracy and performance by approximately 3.5% and 8–10 times, respectively, which is significantly superior to other classic algorithms. Furthermore, we tested the proposed system on multiple pose-recognition datasets. The 3D attitude estimation system speed can reach 24 fps with an error of 100 mm, which is considerably less than that of the 2D attitude estimation system with a speed of 60 fps. The pose recognition based on deep learning proposed in this study yielded surprisingly superior performance, proving that the use of deep-learning technology for image recognition has great potential.

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The current researches trend to adopt a low-resolution hot spot map to restore the original high-resolution representation to save computing cost, resulting in unsatisfactory detection performance, especially in human body recognition with a highly complex background. Aiming at this problem, we proposed a model of parallel connection of multiple sub-networks with different resolution levels on a high-resolution main network. It can maintain the network structure of a high-resolution hot spot map in the whole operation process. By using this structure in the human key point vector field network, the accuracy of human posture recognition has been improved with high-speed operation. To validate the proposed model’s effectiveness, two common benchmark data sets of COCO key point data set and MPII human posture data set are used for evaluation. Experimental results show that our network achieves the accuracy of 72.3% AP and 92.2% AP in the two data sets, respectively, which is 3%-4% higher than those of the existing mainstream researches. In our test, only the accuracy of backbone’s SimpleBaseline with ResNet-152 is close to ours, yet our network realized a much lower computing cost.

Advances in artificial intelligence-based autonomous applications have led to the advent of domestic robots for smart elderly care; the preliminary critical step for such robots involves increasing the comprehension of robotic visualizing of human activity recognition. In this paper, discrete hidden Markov models (D-HMMs) are used to investigate human activity recognition. Eleven daily home activities are recorded using a video camera with an RGB-D sensor to collect a dataset composed of 25 skeleton joints in a frame, wherein only 10 skeleton joints are utilized to efficiently perform human activity recognition. Features of the chosen ten skeleton joints are sequentially extracted in terms of pose sequences for a specific human activity, and then, processed through coordination transformation and vectorization into a codebook prior to the D-HMM for estimating the maximal posterior probability to predict the target. In the experiments, the confusion matrix is evaluated based on eleven human activities; furthermore, the extension criterion of the confusion matrix is also examined to verify the robustness of the proposed work. The novelty indicated D-HMM theory is not only promising in terms of speech signal processing but also is applicable to visual signal processing and applications. [ABSTRACT FROM AUTHOR]

This paper presents a framework of a marker-less human pose recognition system by identifying key body extremity parts through a network of calibrated low-cost depth sensors. The usage of depth sensors overcomes challenges related to low illuminations which usually compromises the information from the RGB cameras. Furthermore, the addition of multiple depth sensors complements the existing information with more visibility and less self-occlusion. A simple algorithm was applied which finds the connections between aligned and updated meshes produced from multiple sensors. These connections help to fuse the meshes into one large geodesic graph network. On this graph, a novel algorithm is applied to identify key body extremities such as head, hands, and feet of a human subject. A geodesic mapping is applied to the fused point cloud to produce a set of distinct topological clusters of 3D points. These clusters generate a hierarchical skeleton tree graph (Reeb graph) and produce a set of features for semantic identification of key body extremities. The combination of both the shape model and semantic classification finally leads to pose recognition. The paper presents the assessment of the proposed framework and its comparison with another available technique in a succession of experimental configurations. [ABSTRACT FROM AUTHOR]

The 6-axis external fixation mechanism with Gough-Stewart configuration has been widely applied to the correction of long bone deformities in orthopedics. Pose recognition of the mechanism is essential for trajectory planning of bone correction, but is usually implemented by the surgeons’ experience, resulting in a relatively low level of correction accuracy. This paper proposes a pose recognition method based on novel image markers, and implements accuracy analysis. Firstly, a pose description of the mechanism is established with several freely installed markers, and the layout of the markers is also parametrically described. Then, a pose recognition method is presented by identifying the orientation and position parameters using the markers. The recognition method is general in that it encompasses all possible marker layouts, and the recognition accuracy is investigated by analyzing variations in the marker layout. On this basis, layout principles for markers that achieve a desired recognition accuracy are established, and an error compensation strategy for precision improvement is provided. Finally, experiments were conducted. The results show that volume errors of pose recognition were 0.368 ± 0.130 mm and 0.151 ± 0.045°, and the correction accuracy of the fracture model after taking compensation was 0.214 ± 0.573 mm and −0.031 ± 0.161°, validating the feasibility and accuracy of the proposed methods.

This paper provides an in-depth study and analysis of human artistic poses through intelligently enhanced multimodal artistic pose recognition. A complementary network model architecture of multimodal information based on motion energy proposed. The network exploits both the rich information of appearance features provided by RGB data and the depth information provided by depth data as well as the characteristics of robustness to luminance and observation angle. The multimodal fusion is accomplished by the complementary information characteristics of the two modalities. Moreover, to better model the long-range temporal structure while considering action classes with sub-action sharing phenomena, an energy-guided video segmentation method is employed. And in the feature fusion stage, a cross-modal cross-fusion approach is proposed, which enables the convolutional network to share local features of two modalities not only in the shallow layer but also to obtain the fusion of global features in the deep convolutional layer by connecting the feature maps of multiple convolutional layers. Firstly, the Kinect camera is used to acquire the color image data of the human body, the depth image data, and the 3D coordinate data of the skeletal points using the Open pose open-source framework. Then, the action automatically extracted from keyframes based on the distance between the hand and the head, and the relative distance features are extracted from the keyframes to describe the action, the local occupancy pattern features and HSV color space features are extracted to describe the object, and finally, the feature fusion is performed and the complex action recognition task is completed. To solve the consistency problem of virtual-reality fusion, the mapping relationship between hand joint point coordinates and the virtual scene is determined in the augmented reality scene, and the coordinate consistency model of natural hand and virtual model is established; finally, the real-time interaction between hand gesture and virtual model is realized, and the average correct rate of its hand gesture reaches 99.04%, which improves the robustness and real-time interaction of hand gesture recognition.

This paper provides an in-depth study and analysis of human artistic poses through intelligently enhanced multimodal artistic pose recognition. A complementary network model architecture of multimodal information based on motion energy proposed. The network exploits both the rich information of appearance features provided by RGB data and the depth information provided by depth data as well as the characteristics of robustness to luminance and observation angle. The multimodal fusion is accomplished by the complementary information characteristics of the two modalities. Moreover, to better model the long-range temporal structure while considering action classes with sub-action sharing phenomena, an energy-guided video segmentation method is employed. And in the feature fusion stage, a cross-modal cross-fusion approach is proposed, which enables the convolutional network to share local features of two modalities not only in the shallow layer but also to obtain the fusion of global features in the deep convolutional layer by connecting the feature maps of multiple convolutional layers. Firstly, the Kinect camera is used to acquire the color image data of the human body, the depth image data, and the 3D coordinate data of the skeletal points using the Open pose open-source framework. Then, the action automatically extracted from keyframes based on the distance between the hand and the head, and the relative distance features are extracted from the keyframes to describe the action, the local occupancy pattern features and HSV color space features are extracted to describe the object, and finally, the feature fusion is performed and the complex action recognition task is completed. To solve the consistency problem of virtual-reality fusion, the mapping relationship between hand joint point coordinates and the virtual scene is determined in the augmented reality scene, and the coordinate consistency model of natural hand and virtual model is established; finally, the real-time interaction between hand gesture and virtual model is realized, and the average correct rate of its hand gesture reaches 99.04%, which improves the robustness and real-time interaction of hand gesture recognition. [ABSTRACT FROM AUTHOR]

In the task of three-dimensional human motion posture recognition, there are problems such as target loss, inaccurate target positioning, and high computational complexity. This article designs a recognition evaluation algorithm to address these issues. Design a LiteHRNet model for extracting skeleton sequences from action videos, and propose a graph convolutional structure that combines residual networks and attention mechanisms. This network can effectively enhance the expression ability of node key features. Introducing second-order velocity information and spatial position information of joint points to improve positioning accuracy. Improve the TCN and Transformer network models to simultaneously extract local and long-term features throughout the entire model, and more accurately model the temporal correlation between nodes in the entire action sequence. The fusion of Transformer networks can reduce the computational complexity of the model while ensuring its accuracy. The experiment shows that the model has good evaluation performance on multiple datasets. • This paper proposes a technique for 3D human motion pose recognition algorithm. • This method improves the efficiency of human pose estimation models based on GCN. • This method solves the problems of low accuracy in action recognition models. • This method solves the problems of high resource consumption. [ABSTRACT FROM AUTHOR]

To control industry robots and make sure they are working in a correct status, an efficient way to judge the motion of the robot is important. In this article, an industry robotic motion and pose recognition method based on camera pose estimation and neural network are proposed. Firstly, industry robotic motion recognition based on the neural network has been developed to estimate and optimize motion of the robotics only by a monoscope camera. Secondly, the motion recognition including key flames recording and pose adjustment has been proposed and analyzed to restore the pose of the robotics more accurately. Finally, a KUKA industry robot has been used to test the proposed method, and the test results have demonstrated that the motion and pose recognition method can recognize the industry robotic pose accurately and efficiently without inertial measurement unit (IMU) and other censers. Below in the same algorithm, the error of the method introduced in this article is better than the traditional method using IMU and has a better merit of reducing cumulative error. [ABSTRACT FROM AUTHOR]

Continuous monitoring of frail individuals for detecting dangerous situations during their daily living at home can be a powerful tool toward their inclusion in the society by allowing living independently while safely. To this goal we developed a pose recognition system tailored to disabled students living in college dorms and based on skeleton tracking through four Kinect One devices independently recording the inhabitant with different viewpoints, while preserving the individual’s privacy. The system is intended to classify each data frame and provide the classification result to a further decision-making algorithm, which may trigger an alarm based on the classified pose and the location of the subject with respect to the furniture in the room. An extensive dataset was recorded on 12 individuals moving in a mockup room and undertaking four poses to be recognized: standing, sitting, lying down, and “dangerous sitting.” The latter consists of the subject slumped in a chair with his/her head lying forward or backward as if unconscious. Each skeleton frame was labeled and represented using 10 discriminative features: three skeletal joint vertical coordinates and seven relative and absolute angles describing articular joint positions and body segment orientation. In order to classify the pose of the subject in each skeleton frame we built a two hidden layers multi-layer perceptron neural network with a “SoftMax” output layer, which we trained on the data from 10 of the 12 subjects (495,728 frames), with the data from the two remaining subjects representing the test set (106,802 frames). The system achieved very promising results, with an average accuracy of 83.9% (ranging 82.7 and 94.3% in each of the four classes). Our work proves the usefulness of human pose recognition based on machine learning in the field of safety monitoring in assisted living conditions.

Advances in artificial intelligence-based autonomous applications have led to the advent of domestic robots for smart elderly care; the preliminary critical step for such robots involves increasing the comprehension of robotic visualizing of human activity recognition. In this paper, discrete hidden Markov models (D-HMMs) are used to investigate human activity recognition. Eleven daily home activities are recorded using a video camera with an RGB-D sensor to collect a dataset composed of 25 skeleton joints in a frame, wherein only 10 skeleton joints are utilized to efficiently perform human activity recognition. Features of the chosen ten skeleton joints are sequentially extracted in terms of pose sequences for a specific human activity, and then, processed through coordination transformation and vectorization into a codebook prior to the D-HMM for estimating the maximal posterior probability to predict the target. In the experiments, the confusion matrix is evaluated based on eleven human activities; furthermore, the extension criterion of the confusion matrix is also examined to verify the robustness of the proposed work. The novelty indicated D-HMM theory is not only promising in terms of speech signal processing but also is applicable to visual signal processing and applications.