Search

In the action recognition field based on the characteristics of human skeleton joint points, the selection of keyframes in the skeleton sequence is a significant issue, which directly affects the action recognition accuracy. In order to improve the effectiveness of keyframes selection, this paper proposes inflection point frames, and transforms keyframes selection into a multi-objective optimization problem based on it. First, the pose features are extracted from the input skeleton joint point data, which used to construct the pose feature vector of each frame in time sequence; then, the inflection point frames in the sequence are determined according to the flow of momentum of each body part. Next, the pose feature vectors are input into the keyframes multi-objective optimization model, with the fusion of domain information and the number of keyframes; finally, the output keyframes are input to the action classifier. To verify the effectiveness of the method, the MSR-Action3D, the UTKinect-Action and Florence3D-Action, and the 3 public datasets, are chosen for simulation experiments and the results show that the keyframes sequence obtained by this method can significantly improve the accuracy of multiple action classifiers, and the average recognition accuracy of the three data sets can reach 94.6%, 97.6% and 94.2% respectively. Besides, combining the optimized keyframes with deep learning classifier on the NTU RGB + D dataset can make the accuracies reaching 83.2% and 93.7%. [ABSTRACT FROM AUTHOR]

Abstract In the action recognition field based on the characteristics of human skeleton joint points, the selection of keyframes in the skeleton sequence is a significant issue, which directly affects the action recognition accuracy. In order to improve the effectiveness of keyframes selection, this paper proposes inflection point frames, and transforms keyframes selection into a multi-objective optimization problem based on it. First, the pose features are extracted from the input skeleton joint point data, which used to construct the pose feature vector of each frame in time sequence; then, the inflection point frames in the sequence are determined according to the flow of momentum of each body part. Next, the pose feature vectors are input into the keyframes multi-objective optimization model, with the fusion of domain information and the number of keyframes; finally, the output keyframes are input to the action classifier. To verify the effectiveness of the method, the MSR-Action3D, the UTKinect-Action and Florence3D-Action, and the 3 public datasets, are chosen for simulation experiments and the results show that the keyframes sequence obtained by this method can significantly improve the accuracy of multiple action classifiers, and the average recognition accuracy of the three data sets can reach 94.6%, 97.6% and 94.2% respectively. Besides, combining the optimized keyframes with deep learning classifier on the NTU RGB + D dataset can make the accuracies reaching 83.2% and 93.7%.

To improve the robustness of UAVs in rotating and motion scenes, we propose a stereo simultaneous localization and mapping (SLAM) for UAVs with a new keyframe strategy that differs from current SLAM systems. Moreover, it has two strategies. 1) In the dominant strategy based on image quality, we filter and retain the strong feature points in each image frame by our own defined rules; this leads to longer survival time and attribute invariance in image tracking, and we save the image frames containing more strong feature points as keyframes. 2) In the secondary strategy based on motion state, we quantify the motion state during the UAV’s motion (called the compound rotation amount), and characterize the intensity of the motion. Since we want the UAV to have better robustness in the rotating scene, this strategy generates keyframes when the compound rotation amount meets the threshold. These two strategies are used to cope with gentle motion scenes and rotational motion scenes, respectively. Thus, the insertion of our keyframes is determined by the motion state; our SLAM system is proposed based on the motion state (MS-SLAM). In the back-end part of the system we construct a new weighted cost function to optimize the pose. Finally, through comparison experiments on the public dataset EuRoc, we demonstrate that our algorithm is more advantageous than some current mainstream algorithms. In difficult sequences, our algorithm compares the absolute trajectory error with ORB-SLAM2, SVO+gtsam, and VINS-Mono, the absolute trajectory error of our algorithm can be reduced by 87%.

In recent years, with the growth of digital media and modern imaging equipment, the use of video processing algorithms and semantic film and image management has expanded. The usage of different video datasets in training artificial intelligence algorithms is also rapidly expanding in various fields. Due to the high volume of information in a video, its processing is still expensive for most hardware systems, mainly in terms of its required runtime and memory. Hence, the optimal selection of keyframes to minimize redundant information in video processing systems has become noteworthy in facilitating this problem. Eliminating some frames can simultaneously reduce the required computational load, hardware cost, memory and processing time of intelligent video-based systems. Based on the aforementioned reasons, this research proposes a method for selecting keyframes and adaptive cropping input video for human action recognition (HAR) systems. The proposed method combines edge detection, simple difference, adaptive thresholding and 1D and 2D average filter algorithms in a hierarchical method. Some HAR methods are trained with videos processed by the proposed method to assess its efficiency. The results demonstrate that the application of the proposed method increases the accuracy of the HAR system by up to 3% compared to random image selection and cropping methods. Additionally, for most cases, the proposed method reduces the training time of the used machine learning algorithm. [ABSTRACT FROM AUTHOR]

In recent years, with the growth of digital media and modern imaging equipment, the use of video processing algorithms and semantic film and image management has expanded. The usage of different video datasets in training artificial intelligence algorithms is also rapidly expanding in various fields. Due to the high volume of information in a video, its processing is still expensive for most hardware systems, mainly in terms of its required runtime and memory. Hence, the optimal selection of keyframes to minimize redundant information in video processing systems has become noteworthy in facilitating this problem. Eliminating some frames can simultaneously reduce the required computational load, hardware cost, memory and processing time of intelligent video-based systems. Based on the aforementioned reasons, this research proposes a method for selecting keyframes and adaptive cropping input video for human action recognition (HAR) systems. The proposed method combines edge detection, simple difference, adaptive thresholding and 1D and 2D average filter algorithms in a hierarchical method. Some HAR methods are trained with videos processed by the proposed method to assess its efficiency. The results demonstrate that the application of the proposed method increases the accuracy of the HAR system by up to 3% compared to random image selection and cropping methods. Additionally, for most cases, the proposed method reduces the training time of the used machine learning algorithm.

Video abstraction is a basic step for intelligent access to video and multimedia databases which facilitates content-based video indexing, retrieving and browsing. This paper presents a new video abstraction scheme. The proposed method relies on two stages. First, video is divided into short segments. Second, keyframes in each segment are selected using particle swarm optimisation. A group of experiments show that the proposed technique is promising in regards to selecting the most significant keyframes despite a sustainment in overhead processing.

Abstract: Video abstraction is a basic step for intelligent access to video and multimedia databases which facilitates content-based video indexing, retrieving and browsing. This paper presents a new video abstraction scheme. The proposed method relies on two stages. First, video is divided into short segments. Second, keyframes in each segment are selected using particle swarm optimisation. A group of experiments show that the proposed technique is promising in regards to selecting the most significant keyframes despite a sustainment in overhead processing. [Copyright &y& Elsevier]