Search

Abstract Stroke is the prevalent cerebrovascular disease characterized by significant incidence and disability rates. To enhance the early perceive and detection of potential stroke patients, the early stroke behavior detection based on improved Video Masked Autoencoders (VideoMAE) for potential patients (EPBR-PS) is proposed. The proposed method begins with novel time interval-based sampling strategy, capturing video frame sequences enriched with sparse motion features. On the basis of establishing the masking mechanism for adjacent frames and pixel blocks within these sequences, The EPBR-PS employes pipeline mask strategy to extract spatiotemporal features effectively. Then, the local convolution attention mechanism is designed to capture local dynamic feature information, and central to the EPBR-PS is the integration of local convolutional attention mechanism with VideoMAE's multi-head attention mechanism. This integration facilitates the simultaneous leveraging of global high-level semantics and local dynamic feature information. Dual attention mechanism-based method for the fusion of these global and local features is proposed. After that, the optimal parameters of EPBR-PS were determined through the experiment of learning rate and fusion weights of different features. On the NTU-ST dataset, comparative analysis with eight models demonstrated the superiority of EPBR-PS, evidenced by the average recognition accuracy of 89.61%, surpassing that 1.67% over the benchmark VideoMAE. On the HMDB51 dataset, EPBR-PS has Top1 of 71.31%, which is 0.73% higher than that of the VideoMAE, providing the viable behavior detection for perception early signs of potential stroke in the home environment. This code is available at https://github.com/wang-325/EPBR-PS/ .

Abstract Keyframe extraction can effectively help users quickly understand video content. Generally, keyframes should be representative of the video content and simultaneously be diverse to reduce redundancy. Aiming to find the features of frames and filter out representative frames of the video, we propose a method of keyframe recommendation based on feature intercross and fusion (KFRFIF). The method is inspired by the implied relations between keyframe-extraction problem and recommendation problem. First, we investigate the application of a recommendation framework to the keyframe extraction problem. Second, the architecture of the proposed KFRFIF is put forward. Then, an algorithm for extracting intra-frame image features based on the combination of multiple image descriptors is proposed. An algorithm for extracting inter-frame distance features based on the combination of multiple distance calculation methods is designed. Moreover, A recommendation model based on feature intercross and fusion is put forward. An ablation study is further performed to verify the effectiveness of the submodule. Ultimately, the experimental results on four datasets with five outstanding approaches indicate the superior performance of our approach.

Abstract Although social robots equipped with visual devices may leak user information, countermeasures for ensuring privacy are not readily available, making visual privacy protection problematic. In this article, a semi‐supervised learning algorithm is proposed for visual privacy behaviour recognition based on an improved generative adversarial network for social robots; it is called PBR‐GAN. A 9‐layer residual generator network enhances the data quality, and a 10‐layer discriminator network strengthens the feature extraction. A tailored objective function, loss function, and strategy are proposed to dynamically adjust the learning rate to guarantee high performance. A social robot platform and architecture for visual privacy recognition and protection are implemented. The recognition accuracy of the proposed PBR‐GAN is compared with Inception_v3, SS‐GAN, and SF‐GAN. The average recognition accuracy of the proposed PBR‐GAN is 85.91%, which is improved by 3.93%, 9.91%, and 1.73% compared with the performance of Inception_v3, SS‐GAN, and SF‐GAN respectively. Through a case study, seven situations are considered related to privacy at home, and develop training and test datasets with 8,720 and 1,280 images, respectively, are developed. The proposed PBR‐GAN recognises the designed visual privacy information with an average accuracy of 89.91%.

Abstract Non‐suicide self‐injury (NSSI) can be dangerous and difficult for guardians or caregivers to detect in time. NSSI refers to when people hurt themselves even though they have no wish to cause critical or long‐lasting hurt. To timely identify and effectively prevent NSSI in order to reduce the suicide rates of patients with a potential suicide risk, the detection of NSSI based on the spatiotemporal features of indoor activities is proposed. Firstly, an NSSI behaviour dataset is provided, and it includes four categories that can be used for scientific research on NSSI evaluation. Secondly, an NSSI detection algorithm based on the spatiotemporal features of indoor activities (NssiDetection) is proposed. NssiDetection calculates the human bounding box by using an object detection model and employs a behaviour detection model to extract the temporal and spatial features of NSSI behaviour. Thirdly, the optimal combination schemes of NssiDetection is investigated by checking its performance with different behaviour detection methods and training strategies. Lastly, a case study is performed by implementing an NSSI behaviour detection prototype system. The prototype system has a recognition accuracy of 84.18% for NSSI actions with new backgrounds, persons, or camera angles.

The stochastic configuration network (SCN) randomly configures the input weights and biases of hidden layers under a set of inequality constraints to guarantee its universal approximation property. The SCN has demonstrated great potential for fast and efficient data modeling. However, the prediction accuracy and convergence rate of SCN are frequently impacted by the parameter settings of the model. The weighted mean of vectors (INFO) is an innovative swarm intelligence optimization algorithm, with an optimization procedure consisting of three phases: updating rule, vector combining, and a local search. This article aimed at establishing a new regularized SCN based on the weighted mean of vectors (RSCN-INFO) to optimize its parameter selection and network structure. The regularization term that combines the ridge method with the residual error feedback was introduced into the objective function in order to dynamically adjust the training parameters. Meanwhile, INFO was employed to automatically explore an appropriate four-dimensional parameter vector for RSCN. The selected parameters may lead to a compact network architecture with a faster reduction of the network residual error. Simulation results over some benchmark datasets demonstrated that the proposed RSCN-INFO showed superior performance with respect to parameter setting, fast convergence, and network compactness compared with other contrast algorithms.

As the problem of an aging population becomes more and more serious, social robots have an increasingly significant influence on human life. By employing regular question-and-answer conversations or wearable devices, some social robotics products can establish personal health archives. But those robots are unable to collect diet information automatically through robot vision or audition. A healthy diet can reduce a person's risk of developing cancer, diabetes, heart disease, and other age-related diseases. In order to automatically perceive the dietary composition of the elderly by listening to people's chatting, this paper proposed a chat-based automatic dietary composition perception algorithm (DCPA). DCPA uses social robot audition to understand the semantic information and percept dietary composition for Mandarin Chinese. Firstly, based on the Mel-frequency cepstrum coefficient and convolutional neural network, a speaker recognition method is designed to identify speech data. Based on speech segmentation and speaker recognition algorithm, an audio segment classification method is proposed to distinguish different speakers, store their identity information and the sequence of expression in a speech conversation. Secondly, a dietetic lexicon is established, and two kinds of dietary composition semantic understanding algorithms are proposed to understand the eating semantics and sensor dietary composition information. To evaluate the performance of the proposed DCPA algorithm, we implemented the proposed DCPA in our social robot platform. Then we established two categories of test datasets relating to a one-person and a multi-person chat. The test results show that DCPA is capable of understanding users' dietary compositions, with an F1 score of 0.9505, 0.8940 and 0.8768 for one-person talking, a two-person chat and a three-person chat, respectively. DCPA has good robustness for obtaining dietary information.

We adopted actual intelligent production requirements and proposed a tiny part defect detection method to obtain a stable and accurate real-time tiny part defect detection system and solve the problems of manually setting conveyor speed and industrial camera parameters in defect detection for factory products. First, we considered the important influences of the properties of tiny parts and the environmental parameters of a defect detection system on its stability. Second, we established a correlation model between the detection capability coefficient of the part system and the moving speed of the conveyor. Third, we proposed a defect detection algorithm for tiny parts that are based on a single short detector network (SSD) and deep learning. Finally, we combined an industrial real-time detection platform with the missed detection algorithm for mechanical parts based on intermediate variables to address the problem of missed detections. We used a 0.8 cm darning needle as the experimental object. The system defect detection accuracy was the highest when the speed of the conveyor belt was 7.67 m/min.

In order to automatically perceive the user’s dietary nutritional information in the smart home environment, this paper proposes a dietary nutritional information autonomous perception method based on machine vision in smart homes. Firstly, we proposed a food-recognition algorithm based on YOLOv5 to monitor the user’s dietary intake using the social robot. Secondly, in order to obtain the nutritional composition of the user’s dietary intake, we calibrated the weight of food ingredients and designed the method for the calculation of food nutritional composition; then, we proposed a dietary nutritional information autonomous perception method based on machine vision (DNPM) that supports the quantitative analysis of nutritional composition. Finally, the proposed algorithm was tested on the self-expanded dataset CFNet-34 based on the Chinese food dataset ChineseFoodNet. The test results show that the average recognition accuracy of the food-recognition algorithm based on YOLOv5 is 89.7%, showing good accuracy and robustness. According to the performance test results of the dietary nutritional information autonomous perception system in smart homes, the average nutritional composition perception accuracy of the system was 90.1%, the response time was less than 6 ms, and the speed was higher than 18 fps, showing excellent robustness and nutritional composition perception performance.

Game theory has been widely recognized as an important tool in many fields, which provides general mathematical techniques for analyzing situations in which two or more individuals make decisions that will influence one another’s welfare. This paper presents a game-theoretic evolutionary algorithm based on behavioral expectation, which is a type of optimization approach based on game theory. A formulation to estimate the payoffs expectation is given, which is a mechanism of trying to master the player’s information so as to facilitate the player becoming the rational decision maker. GameEA has one population (players set), and generates new offspring only by the imitation operator and the belief learning operator. The imitation operator is used to learn strategies and actions from other players to improve its competitiveness and applies it into the future game, namely that one player updates its chromosome by strategically copying some segments of gene sequences from the competitor. Belief learning refers to models in which a player adjusts its own strategies, behavior or chromosome by analyzing current history information with respect to an improvement of solution quality. The experimental results on various classes of problems using real-valued representation show that GameEA outperforms not only the standard genetic algorithm (GA) but also other GAs having additional mechanisms of accuracy enhancement. Finally, we compare the convergence of GameEA with different numbers of players to determine whether this parameter has a significant effect on convergence. The statistical results show that at the 0.05 significance level, the number of players has a crucial impact on GameEA's performance. The results suggest that 50 or 100 players will provide good results with unimodal functions, while 200 players will provide good results for multimodal functions.

State-of-the-art semantic segmentation methods rely too much on complicated deep networks and thus cannot train efficiently. This paper introduces a novel Circle-U-Net architecture that exceeds the original U-Net on several standards. The proposed model includes circle connect layers, which is the backbone of ResUNet-a architecture. The model possesses a contracting part with residual bottleneck and circle connect layers that capture context and expanding paths, with sampling layers and merging layers for a pixel-wise localization. The results of the experiment show that the proposed Circle-U-Net achieves an improved accuracy of 5.6676%, 2.1587% IoU (Intersection of union, IoU) and can detect 67% classes greater than U-Net, which is better than current results.

In order to realize fast real-time positioning after a mobile robot starts, this paper proposes an improved ORB-SLAM2 algorithm. Firstly, we proposed a binary vocabulary storage method and vocabulary training algorithm based on an improved Oriented FAST and Rotated BRIEF (ORB) operator to reduce the vocabulary size and improve the loading speed of the vocabulary and tracking accuracy. Secondly, we proposed an offline map construction algorithm based on the map element and keyframe database; then, we designed a fast reposition method of the mobile robot based on the offline map. Finally, we presented an offline visualization method for map elements and mapping trajectories. In order to check the performance of the algorithm in this paper, we built a mobile robot platform based on the EAI-B1 mobile chassis, and we implemented the rapid relocation method of the mobile robot based on improved ORB SLAM2 algorithm by using C++ programming language. The experimental results showed that the improved ORB SLAM2 system outperforms the original system regarding start-up speed, tracking and positioning accuracy, and human–computer interaction. The improved system was able to build and load offline maps, as well as perform rapid relocation and global positioning tracking. In addition, our experiment also shows that the improved system is robust against a dynamic environment.

Tensioning is an important process for producing prestressed concrete beams and directly affects bridge performance and driving safety. Active sensing and management of tensioning process data can improve the efficiency of quality monitoring and level of prestressed concrete beams. To realize remote collection and quality monitoring of tensioning process data, a framework for data sensing and processing of tensioning system based on the Internet of Things (IoT) is proposed in this study. Firstly, we investigate the technical framework and techniques of the system and designs a work flow of sensing, transport, and application service layers. The architecture of the tensioning system is presented. Then we propose a data acquisition and preprocessing method for the sensing layer, put forwards the data-pushing mechanism of the transport layer, and designs the function and work flow of the application service layer. After that, .NET platform and Android Studio are used to implement the tensioning system based on Browser/Server (B/S) architecture and mobile terminals. Finally, the case results of the system in seven precast beam fields in the Hubei section of Zhengzhou–Wanzhou high-speed Railway are given, which show that the developed system realizes collection, active pushing, and remote monitoring of tensioning process data.

Prognostics, such as remaining useful life (RUL) prediction, is a crucial task in condition-based maintenance. A major challenge in data-driven prognostics is the difficulty of obtaining a sufficient number of samples of failure progression. However, for traditional machine learning methods and deep neural networks, enough training data is a prerequisite to train good prediction models. In this work, we proposed a transfer learning algorithm based on Bi-directional Long Short-Term Memory (BLSTM) recurrent neural networks for RUL estimation, in which the models can be first trained on different but related datasets and then fine-tuned by the target dataset. Extensive experimental results show that transfer learning can in general improve the prediction models on the dataset with a small number of samples. There is one exception that when transferring from multi-type operating conditions to single operating conditions, transfer learning led to a worse result.

This study proposes a modified convolutional neural network (CNN) algorithm that is based on dropout and the stochastic gradient descent (SGD) optimizer (MCNN-DS), after analyzing the problems of CNNs in extracting the convolution features, to improve the feature recognition rate and reduce the time-cost of CNNs. The MCNN-DS has a quadratic CNN structure and adopts the rectified linear unit as the activation function to avoid the gradient problem and accelerate convergence. To address the overfitting problem, the algorithm uses an SGD optimizer, which is implemented by inserting a dropout layer into the all-connected and output layers, to minimize cross entropy. This study used the datasets MNIST, HCL2000, and EnglishHand as the benchmark data, analyzed the performance of the SGD optimizer under different learning parameters, and found that the proposed algorithm exhibited good recognition performance when the learning rate was set to [0.05, 0.07]. The performances of WCNN, MLP-CNN, SVM-ELM, and MCNN-DS were compared. Statistical results showed the following: (1) For the benchmark MNIST, the MCNN-DS exhibited a high recognition rate of 99.97%, and the time-cost of the proposed algorithm was merely 21.95% of MLP-CNN, and 10.02% of SVM-ELM; (2) Compared with SVM-ELM, the average improvement in the recognition rate of MCNN-DS was 2.35% for the benchmark HCL2000, and the time-cost of MCNN-DS was only 15.41%; (3) For the EnglishHand test set, the lowest recognition rate of the algorithm was 84.93%, the highest recognition rate was 95.29%, and the average recognition rate was 89.77%.

Many text mining tasks such as text retrieval, text summarization, and text comparisons depend on the extraction of representative keywords from the main text. Most existing keyword extraction algorithms are based on discrete bag-of-words type of word representation of the text. In this paper, we propose a patent keyword extraction algorithm (PKEA) based on the distributed Skip-gram model for patent classification. We also develop a set of quantitative performance measures for keyword extraction evaluation based on information gain and cross-validation, based on Support Vector Machine (SVM) classification, which are valuable when human-annotated keywords are not available. We used a standard benchmark dataset and a homemade patent dataset to evaluate the performance of PKEA. Our patent dataset includes 2500 patents from five distinct technological fields related to autonomous cars (GPS systems, lidar systems, object recognition systems, radar systems, and vehicle control systems). We compared our method with Frequency, Term Frequency-Inverse Document Frequency (TF-IDF), TextRank and Rapid Automatic Keyword Extraction (RAKE). The experimental results show that our proposed algorithm provides a promising way to extract keywords from patent texts for patent classification.

Many approaches that model specific intelligent behaviors perform excellently in solving complex optimization problems. Game theory is widely recognized as an important tool in many fields. This paper introduces a game theory-inspired evolutionary algorithm for global optimization (GameEA). A formulation to estimate payoff expectations is provided, which is a mechanism to make a player become a rational decision-maker. GameEA has one population (i.e., set of players) and generates new offspring only through an imitation operator and a belief-learning operator. An imitation operator adopts learning strategies and actions from other players to improve its competitiveness and applies these strategies to future games where one player updates its chromosome by strategically copying segments of gene sequences from a competitor. Belief learning refers to models in which a player adjusts his/her strategies, behavior or chromosomes by analyzing the current history information to improve solution quality. Experimental results on various classes of problems show that GameEA outperforms the other four algorithms on stability, robustness, and accuracy.