Search

This article presents a structure constraint matrix factorization framework for different behavior segmentation of the human behavior sequential data. This framework is based on the structural information of the behavior continuity and the high similarity between neighboring frames. Due to the high similarity and high dimensionality of human behavior data, the high-precision segmentation of human behavior is hard to achieve from the perspective of application and academia. By making the behavior continuity hypothesis, first, the effective constraint regular terms are constructed. Subsequently, the clustering framework based on constrained non-negative matrix factorization is established. Finally, the segmentation result can be obtained by using the spectral clustering and graph segmentation algorithm. For illustration, the proposed framework is applied to the Weiz dataset, Keck dataset, mo_86 dataset, and mo_86_9 dataset. Empirical experiments on several public human behavior datasets demonstrate that the structure constraint matrix factorization framework can automatically segment human behavior sequences. Compared to the classical algorithm, the proposed framework can ensure consistent segmentation of sequential points within behavior actions and provide better performance in accuracy. [ABSTRACT FROM AUTHOR]

This special issue of the IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (T-ASE) focuses on how the state-of-the-art achievements and applications in the general area of artificial intelligence in automation for autonomous unmanned systems applications. As Guest Editors, we are very pleased to present the selected 16 articles, whose topics are specifically related to artificial intelligence real-time object detection, recognition, localization, control optimization, motion planning, formation control, adaptive control, and autonomous decision-making. [ABSTRACT FROM AUTHOR]

Radar and camera information fusion sensing methods are used to solve the inherent shortcomings of the single sensor in severe weather. Our fusion scheme uses radar as the main hardware and camera as the auxiliary hardware framework. At the same time, the Mahalanobis distance is used to match the observed values of the target sequence. Data fusion based on the joint probability function method. Moreover, the algorithm was tested using actual sensor data collected from a vehicle, performing real-time environment perception. The test results show that radar and camera fusion algorithms perform better than single sensor environmental perception in severe weather, which can effectively reduce the missed detection rate of autonomous vehicle environment perception in severe weather. The fusion algorithm improves the robustness of the environment perception system and provides accurate environment perception information for the decision-making system and control system of autonomous vehicles. [ABSTRACT FROM AUTHOR]

This article presents a trajectory following control solution for the lateral motion of an unmanned vehicle. The proposed solution is based on model predictive lateral control. The lateral motion is hard to control since it is nonlinear with large dynamics and uncertainties. By making a small angle approximation, the dynamic model can be linearized. A new bounded equivalent function based on the vehicle kinematic model and the Taylor series expansion is presented for trajectory following control solution for the lateral motion problem. The model predictive lateral control is used to ensure both strong robustness and control accuracy. Experiment results in a real environment are presented to show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This article presents a simple control method for the fully automatic parking of an unmanned vehicle. This method is based on the switching control algorithm and backstepping theory. The fully automatic parking is hard to accomplish since it cannot converge to a specified norm, guarantee the convergence rate, and large uncertainties. Global exponential convergence to any prescribed norm bound ($\epsilon$ -convergence) is guaranteed, and the convergence rate is explicitly given. For the design of steering laws, a method based on backstepping is proposed and analyzed in detail. The backstepping-based design of a multichain system is obtained. Moreover, an exponentially $\epsilon$ -convergent control algorithm is adopted to guarantee both converge norm and convergence rate. Real road experiment results are presented and the results show the effectiveness of the control strategies. [ABSTRACT FROM AUTHOR]

Robotic grasping ability lags far behind human skills and poses a significant challenge in the robotics research area. According to the grasping part of an object, humans can select the appropriate grasping postures of their fingers. When humans grasp the same part of an object, different poses of the palm will cause them to select different grasping postures. Inspired by these human skills, in this article, we propose new grasping posture prediction networks (GPPNs) with multiple inputs, which acquire information from the object image and the palm pose of the dexterous hand to predict appropriate grasping postures. The GPPNs are further combined with grasping rectangle detection networks (GRDNs) to construct multilevel convolutional neural networks (ML-CNNs). In this study, a force-closure index was designed to analyze the grasping quality, and force-closure grasping postures were generated in the GraspIt! environment. Depth images of objects were captured in the Gazebo environment to construct the dataset for the GPPNs. Herein, we describe simulation experiments conducted in the GraspIt! environment, and present our study of the influences of the image input and the palm pose input on the GPPNs using a variable-controlling approach. In addition, the ML-CNNs were compared with the existing grasp detection methods. The simulation results verify that the ML-CNNs have a high grasping quality. The grasping experiments were implemented on the Shadow hand platform, and the results show that the ML-CNNs can accurately complete grasping of novel objects with good performance. [ABSTRACT FROM AUTHOR]

Cut-in driving behavior is one of the basic micro traffic actions for vehicles. A risk assessment helps vehicles execute the behavior well and determine how to react to the same maneuver from other traffic participants when following a leading car or truck. This article presents a discretionary cut-in driving behavior risk assessment method based on field driving data and a united algorithm that is designed to a combine decision tree and a support vector machine to achieve enhanced sensitivity for the riskiest traffic conditions. To build the learning database, a wavelet method is employed to filter naturalistic driving data, incorporating the K-means approach. An unsupervised data learning method is used to categorize the impact on vehicles in the target lane, indicated by a target vehicle’s average and maximum deceleration, into three groups. Experiment results based on self-collected and public databases show that tested vehicles are aware of the risk presented by other cars’ and trucks’ cut-in driving as well as their own impact on traffic participants in the target lane. [ABSTRACT FROM AUTHOR]

In intelligent driving, situational assessment (SA) is an important technology, which helps to improve the cognitive ability of intelligent vehicles in the environment. Uncertainty analysis is very significant in situation assessment. This article proposes an SA method based on uncertainty risk analysis. Under uncertain conditions, according to the random environment model and Gaussian distribution model, the collision probability between multiple vehicles is estimated by comprehensive trajectory prediction. The proposed method considers collision probabilities of different prediction points within and outside the prediction range and obtains long-term accurate prediction results. The method is suitable for the situation risk assessment of sensor systems in the presence of unexpected dynamic obstacles, sensor failures or communication losses in traffic, and different environmental sensing accuracy. The experimental results show that in the dynamic traffic environment, the proposed scenario assessment method can not only accurately predict and assess the situation risks within the prediction range, but also provide accurate scenario risk assessment outside the prediction range. [ABSTRACT FROM AUTHOR]

This article presents a finite-time tracking control scheme for autonomous vehicles. Trajectory tracking is difficult to control due to dynamic couplings and various disturbances and uncertainties. To cope with a lumped disturbance, a fuzzy logic system is designed. In particular, a novel adaptive algorithm is constructed to adjust the gain online, and it works well even without the prior information of the lumped disturbance bounds. An auxiliary system with a simple structure is built to tackle the actuator saturation. Moreover, the proposed control method can guarantee finite-time error convergence, chattering elimination, and strong robustness. Experiment results are presented to show the effectiveness of the control schemes. [ABSTRACT FROM AUTHOR]

This paper presents a latent space reinforcement learning method for interpretable decision-making of autonomous vehicles at highway on-ramps. This method is based on the latent model and the combination model of the hidden Markov model and Gaussian mixture regression (HMM-GMR). It is difficult for the traditional decision-making method to understand the environment because its input is high-dimensional and lacks an understanding of the task. By utilizing the HMM-GMR model, we can obtain the interpretable state providing semantic information and environment understanding. A framework is proposed to unify representation learning with the deep reinforcement learning (DRL) approach, in which the latent model is used to reduce the dimension of interpretable state by extracting underlying task-relevant information. Experimental results are presented and the results show the right balance between driving safety and efficiency in the challenging scenarios of highway on-ramps merging. [ABSTRACT FROM AUTHOR]

This article investigates the integrated lane-keeping and roll control for autonomous ground vehicles (AGVs) considering the transient performance and system disturbances. The robust integral of the sign of error (RISE) control strategy is proposed to achieve the lane-keeping control purpose with rollover prevention, by guaranteeing the asymptotic stability of the closed-loop system, attenuating systematic disturbances, and maintaining the controlled states within the prescribed performance boundaries. Three contributions have been made in this article: 1) a new prescribed performance function (PPF) that does not require accurate initial errors is proposed to guarantee the tracking errors restricted within the predefined asymptotic boundaries; 2) a modified neural network (NN) estimator which requires fewer adaptively updated parameters is proposed to approximate the unknown vertical dynamics; and 3) the improved RISE control based on PPF is proposed to achieve the integrated control objective, which analytically guarantees both the controller continuity and closed-loop system asymptotic stability by integrating the signum error function. The overall system stability is proved with the Lyapunov function. The controller effectiveness and robustness are finally verified by comparative simulations using two representative driving maneuvers, based on the high-fidelity CarSim–Simulink simulation. [ABSTRACT FROM AUTHOR]

Singularity issue has long been a concern of the task-space control design for Euler–Lagrange systems. In classical task-space controls, robots are often assumed to operate in the task space, where singularities do not exist. Such an assumption limits their potential applications in various workspaces. To address the potential singularity issue associated with Euler–Lagrange systems, this article proposes an adaptive fuzzy-region-based control for Euler–Lagrange systems with kinematically singular configurations. Singular regions are described by the potential energy function. The proposed controller includes a joint-space control, which is active when the system approaches singular regions, and a task-space control, which is used to track the desired trajectory. Therefore, the system can smoothly transit from singular regions to nonsingular regions or can achieve singularity avoidance during the tracking task. In order to achieve singularity avoidance while reducing control effort, the coefficients of the potential energy function are adjusted dynamically based on the designed fuzzy system. Rigorous analysis shows that singularity issues can be properly handled, and the asymptotic stability of the system is ensured. Experiments are conducted to demonstrate the effectiveness of the proposed controller. [ABSTRACT FROM AUTHOR]

The use of object detection algorithms has become extremely important in autonomous vehicles. Object detection at high accuracy and a fast inference speed is essential for safe autonomous driving. Therefore, the balance between the effectiveness and efficiency of the object detector must be considered. This article proposes a one-stage object detection framework for improving the detection accuracy while supporting a true real-time operation based on the YOLOv4. The backbone network in the proposed framework is the CSPDarknet53_dcn(P). The last output layer in the CSPDarknet53 is replaced with deformable convolution to improve the detection accuracy. In order to perform feature fusion, a new feature fusion module PAN++ is designed and five scales detection layers are used to improve the detection accuracy of small objects. In addition, this article proposes an optimized network pruning algorithm to solve the problem that the real-time performance of the algorithm cannot be satisfied due to the limited computing resources of the vehicle-mounted computing platform. The method of sparse scaling factor is used to improve the existing channel pruning algorithm. Compared to the YOLOv4, the YOLOV4-5D improves the mean average precision by 4.23% on the BDD data sets and 1.68% on the KITTI data sets. Finally, by pruning the model, the inference speed of YOLOV4-5D is increased 31.3% and the memory is only 98.1 MB when the detection accuracy is almost unchanged. Nevertheless, the proposed algorithm is capable of real-time detection at faster than 66 frames/s (fps) and shows higher accuracy than the previous approaches with a similar fps. [ABSTRACT FROM AUTHOR]

With the aims of safe, smart and sustainable future mobility, a personalized approach of trajectory planning and control based on user preferences is developed for lane-change of autonomous vehicles in this paper. First, a safe area during the lane change process is identified by using constraint Delaunay triangulation. Then, an improved rapidly-exploring Random Trees (i-RRT) is developed with B-spline to generate the feasible trajectory cluster, which is subject to the safe area boundaries and the vehicle dynamics. To extract a personalized trajectory from this cluster, we firstly adopt the fuzzy linguistic preference relation (FLPR) method to identify users’ preferences on driving, which can be reflected by their subjective objectives including driving safety, ride comfort and vehicle stability. Then, the technique for order preference by similarity to ideal situation (TOPSIS) is utilized to solve the multi-objective optimisation problem formulated by considering the user preferences. The algorithms proposed above are integrated, and both simulation and experimental validation are conducted under lane-change scenarios of autonomous driving. Simulation and experiment results show that proposed approach is able to successfully realize personalized trajectory planning and lane-change control, satisfying users’ various preferences and simultaneously ensure vehicle safety, demonstrating its feasibility and effectiveness. [ABSTRACT FROM AUTHOR]

More natural and intuitive control is expected to maximize the auxiliary effect of the powered prosthetic leg for lower limb amputees. In order to realize the stable and flexible walking of prosthetic legs in different terrains according to human intention, a brain–computer interface (BCI) based on motor imagery (MI) is developed. For the raw electroencephalogram (EEG) signals, discrete wavelet transform (DWT) is utilized to extract the time–frequency domain features, which are used as the input signals of the common spatial pattern (CSP) to obtain the time–frequency–space domain features of EEG signals. Then, a support vector machine (SVM) classifier and a directed acyclic graph (DAG) structure are combined to classify multiclass imaginary tasks. According to the result of human intention recognition, the prosthetic leg performs the corresponding gait trajectory generated by coding the ground reaction force (GRF). In addition, a sensory feedback loop is established by functional electrical stimulation (FES), which feeds back the movement of the prosthetic leg to human in real time. The effectiveness and feasibility of the developed EEG-based volitional control of powered prosthetic legs have been validated by three subjects, all of whom were able to fulfill smoothly walking on the floor, ascending stairs, and descending stairs according to their own intentions using prosthetic legs. Note to Practitioners—This article was inspired by the problem that the control mode of the prosthetic leg walking is not natural and intuitive, but it can also be applicable to other powered prosthetic limbs. Existing methods of controlling the walking patterns of the prosthetic leg usually require explicit human manipulation and are not convenient to use. In this article, a novel strategy was proposed, that is, using the human mind to choose the ambulation mode of prosthetic legs. This article described the methods of human brain activity information acquisition and intention recognition. Then, gaits inspired by the healthy leg are designed for the prosthetic leg in three terrains, including walking on the floor, ascending stairs, and descending stairs. In addition, this article provided a way for human to perceive the movement of prosthetic legs, which can make human control the prosthetic leg more smoothly. Experiments on healthy subjects have shown that this approach is feasible. However, no experiments have been conducted on lower limb amputees, and adding more walking patterns may reduce the accuracy of human intention recognition. In the future work, we will study how to accurately identify more categories of human intentions and further improve the control strategy so as to enhance the auxiliary effect of prosthetic legs on amputees. [ABSTRACT FROM AUTHOR]

For human–robot co-manipulation by robotic exoskeletons, the interaction forces provide a communication channel through which the human and the robot can coordinate their actions. In this article, an optimization approach for reshaping the physical interactive trajectory is presented in the co-manipulation tasks, which combines impedance control to enable the human to adjust both the desired and the actual trajectories of the robot. Different from previous studies, the proposed method significantly reshapes the desired trajectory during physical human–robot interaction (pHRI) based on force feedback, without requiring constant human guidance. The proposed scheme first formulates a quadratically constrained programming problem, which is then solved by neural dynamics optimization to obtain a smooth and minimal-energy trajectory similar to the natural human movement. Then, we propose an adaptive neural-network controller based on the barrier Lyapunov function (BLF), which enables the robot to handle the uncertain dynamics and the joint space constraints directly. To validate the proposed method, we perform experiments on the exoskeleton robot with human operators for co-manipulation tasks. The experimental results demonstrate that the proposed controller could complete the co-manipulation tasks effectively. [ABSTRACT FROM AUTHOR]

This paper presents an object classification method for vision and light detection and ranging (LIDAR) fusion of autonomous vehicles in the environment. This method is based on convolutional neural network (CNN) and image upsampling theory. By creating a point cloud of LIDAR data upsampling and converting into pixel-level depth information, depth information is connected with Red Green Blue data and fed into a deep CNN. The proposed method can obtain informative feature representation for object classification in autonomous vehicle environment using the integrated vision and LIDAR data. This method is also adopted to guarantee both object classification accuracy and minimal loss. Experimental results are presented and show the effectiveness and efficiency of object classification strategies. [ABSTRACT FROM AUTHOR]

Vehicle trajectory prediction helps automated vehicles and advanced driver-assistance systems have a better understanding of traffic environment and perform tasks such as criticality assessment in advance. In this study, an integrated vehicle trajectory prediction method is proposed by combining physics- and maneuver-based approaches. These two methods were combined for the reason that the physics-based trajectory prediction method could ensure accuracy in the short term with the consideration of vehicle running dynamic parameters, and the maneuver-based prediction approach has a long-term insight into future trajectories with maneuver estimation. In this study, the interactive multiple model trajectory prediction (IMMTP) method is proposed by combining the two predicting models. The probability of each model in the interactive multiple models could recursively adjust according to the predicting variance of each model. In addition, prediction uncertainty is considered by employing unscented Kalman filters in the physics-based prediction model. To the maneuver-based method, random elements for uncertainty are introduced to the trajectory of each maneuver inferred by using the dynamic Bayesian network. The approach is applied and analyzed in the lane-changing scenario by using naturalistic driving data. Comparison results indicate that IMMTP could achieve a more accurate prediction trajectory with a long prediction horizon. [ABSTRACT FROM PUBLISHER]

The recognition of driver's braking intensity is of great importance for advanced control and energy management for electric vehicles. In this paper, the braking intensity is classified into three levels based on novel hybrid unsupervised and supervised learning methods. First, instead of selecting threshold for each braking intensity level manually, an unsupervised Gaussian mixture model is used to cluster the braking events automatically with brake pressure. Then, a supervised Random Forest model is trained to classify the correct braking intensity levels with the state signals of vehicle and powertrain. To obtain a more efficient classifier, critical features are analyzed and selected. Moreover, beyond the acquisition of discrete braking intensity level, a novel continuous observation method is proposed based on artificial neural networks to quantitative analyze and recognize the brake intensity using the prior determined features of vehicle states. Experimental data are collected in an electric vehicle under real-world driving scenarios. Finally, the classification and regression results of the proposed methods are evaluated and discussed. The results demonstrate the feasibility and accuracy of the proposed hybrid learning methods for braking intensity classification and quantitative recognition with various deceleration scenarios. [ABSTRACT FROM AUTHOR]