Search

An artificial immune system was modelled with self/non-self selection to overcome abnormity in a mobile robot demo. The immune modelling includes the innate immune modelling and the adaptive immune modelling. The self/non-self selection includes detection and recognition, and the self/non-self detection is based on the normal model of the demo. After the detection, the non-self recognition is based on learning unknown non-self for the adaptive immunization. The learning was designed on the neural network or on the learning mechanism from examples. The last step is elimination of all the non-self and failover of the demo. The immunization of the mobile robot demo is programmed with Java to test effectiveness of the approach. Some worms infected the mobile robot demo, and caused the abnormity. The results of the immunization simulations show that the immune program can detect 100% worms, recognize all known Worms and most unknown worms, and eliminate the worms. Moreover, the damaged files of the mobile robot demo can all be repaired through the normal model and immunization. Therefore, the immune modelling of the mobile robot demo is effective and programmable in some anti-worms and abnormity detection applications.

In this paper, a robust model reference adaptive controller based on variable-structure strategy is designed for a class of nonlinear systems with uncertain parameters and bounded external disturbances. Without a priori knowledge of bounded disturbance,both the unknown system parameters and the upper bound of external disturbance are estimated by the design of adaptive law to ensure the accessibility of sliding mode and the global stability of the closed-loop system. Theoretical analysis and simulation results show the robustness and effectiveness of this controller.

In this paper, the iterative Retinex algorithm is improved to handle dynamic range compression problems. Based on the analysis of the McCann-Sobel algorithm, the iterative smoothing operation can be interpreted as an asymmetric diffusion. In order to characterized the sharp discontinuities in illumination, an edge-stopping function is introduced into the iterative procedure, which is inspired by anisotropic diffusion. Using the improved illumination estimator, dynamic range of images can be arbitrarily manipulated while suppressing undesirable artifacts. Experiments show that the proposed algorithms outperforms several derivatives for dynamic range compression based on the iterative Retinex.

As a benchmark control problem with nonlinearity and instability, the controller design for inverted pendulums becomes more difficult when there are model uncertainties and unknown disturbances in the plant dynamics. In this paper, a kernel-based reinforcement learning controller is developed for inverted pendulums with unknown dynamics and stochastic disturbances. The learning controller makes use of approximate policy iteration with kernel-based least-squares temporal difference learning for policy evaluation. Due to the nonlinear approximation ability of kernel methods, good convergence property and learning efficiency can be realized in the approximate policy iteration process so that the controller performance can be optimized in a few iterations. Simulation results demonstrate that the proposed learning controller for stochastic inverted pendulums can achieve much better performance than previous learning control approaches such as Q-learning with function approximation and least-squares policy iteration (LSPI).

With the fast development of robotics and intelligent vehicles, there has been much research work on modeling and motion control of autonomous vehicles. However, due to model complexity, and unknown disturbances from dynamic environment, the motion control of autonomous vehicles is still a difficult problem. In this paper, a novel self-learning path-tracking control method is proposed for a car-like robotic vehicle, where kernel-based approximate dynamic programming (ADP) is used to optimize the controller performance with little prior knowledge on vehicle dynamics. The kernel-based ADP method is a recently developed reinforcement learning algorithm called kernel least-squares policy iteration (KLSPI), which uses kernel methods with automatic feature selection in policy evaluation to get better generalization performance and learning efficiency. By using KLSPI, the lateral control performance of the robotic vehicle can be optimized in a self-learning and data-driven style. Compared with previous learning control methods, the proposed method has advantages in learning efficiency and automatic feature selection. Simulation results show that the proposed method can obtain an optimized path-tracking control policy only in a few iterations, which will be very practical for real applications.

This paper presents a novel method for passing vehicle detection through a monocular camera on the highway, based on the road surface dynamic texture. From the marking line information, the algorithm firstly establishes a model for the texture characters of each lane on the highway. Then, it is estimated whether the area of interesting (AOI) exists by means of using the continuity information of the road surface texture and the similarity information of the surface texture between the adjacent lanes. Finally, whether a vehicle appears in the AOI is confirmed based on the information of the marking line. The experiment demonstrates that this approach can detect the passing vehicle through from the back of the ego-vehicle quickly and correctly. The algorithm is robust and low cost, and it meets the demand of the real-time system.

In temporal independent component analysis (TICA) the components are no longer random variables as required in classical independent component analysis (ICA), but stochastic processes. This paper addresses two problems of TICA when the time structure information of components is taken into account and innovation process is introduced. First it is demonstrated that the assumption that the component in ICA should be random variable can be relaxed to stochastic process as in TICA, and the classical ICA methods can also be available to TICA. Secondly, the influence of innovation process on the two crucial requirements, i.e., statistical independency and nongaussianity, is discussed. The analyses show that it has little impact on the former but can increase the nongaussianity of latent component, which leads to much faster convergence in optimal algorithms. Experimental results show that innovation process is an efficient preprocessing in TICA.

This work presents a new approach for tuning conclusions of fuzzy rules based on reinforcement learning. Unlike the most of existing fuzzy reinforcement learning algorithms, which are based on value function, while our approach called policy gradient fuzzy reinforcement learning (PGFRL) bases on gradient estimate. In PGFRL, the algorithm GPOMDP is employed to estimate the performance gradient with respect to the parameters of fuzzy rules. In our work we prove the convergence of fuzzy rules' parameters to a local optimum given necessary conditions. The experiment results show the effectiveness of PGFRL.

Memetic evolutionary algorithms (MEAs) combine the global search of evolutionary learning methods and the fine-tune ability of local search methods so that they are orders of magnitude more accurate than traditional evolutionary algorithms in many problem domains. However, little work has been done on the mathematical model and convergence analysis of MEAs. In this paper, a theoretical model as well as the convergence analysis of a class of gradient-based MEAs is presented. The results of this paper are extensions of the research work on the abstract model and convergence analysis of general evolutionary algorithms. By modeling the local search of gradient methods as an abstract strong evolution operator, the theoretical framework for abstract memetic evolutionary algorithms is derived. Moreover, the global convergence theorems and the convergence rate estimations of gradient-based MEAs are also established.

This paper addresses the navigation problem of mobile robots in unknown environments, where global path planning methods cannot be applied. In such cases, reactive navigation controllers are commonly employed to deal with the uncertainties in motion planning and control. To realize the automatic design of reactive navigation controllers, a self-learning navigation method is proposed in this paper. The self-learning reactive navigation method is based on a Markov decision model of the navigation problem and uses reinforcement learning algorithms to optimize the action policies of mobile robots. Neural networks are employed to approximate value functions in continuous state spaces so that the self-learning navigation controller has good generalization ability and learning efficiency. Simulation results illustrate the effectiveness of the proposed method.

This paper presents some theorems about separable kernels: when a kernel is separable for the given data and how to make a kernel to be separable. Based on these theorems, a new method of kernels parameters' selection is presented. The advantage of this new method is its efficiency. The experiments have shown that this method can get satisfactory results quickly.

Virtual reality is an effective way for visualization of science experiments. This paper presents a VR enhanced display system for the experiment of HgCdTe(MCT) single crystal growth by Bridgman method in two steps: In the first step, we construct a numerical simulation model (time dependent) based on the heat transfer in furnace and ampoule. This gives information about the most important parameter during the whole growth process: the temperature distribution in furnace and ampoule. Then, with the obtained temperatures, we have developed a VR environment in the second step to enhance the display of the key parameters information of crystal growth. This helps the researcher to understand the process of crystal growth intuitively.

In this paper, we present a simple method to detect obstacles, which is qualitative in that it returns only yes/no answers regarding the presence of obstacles in the field of view. The method is based on the difference of disparity between the ground plane and non-ground plane. The two main features of the method are: neither the camera internal parameters nor the external calibration information is required; and does not need to find the corresponding point in the stereo pair images. Some constructive results are presented.

Based on the idea of dynamic programming, reinforcement learning (RL) has become an important model-free method to solve difficult optimal control problems. In this paper, a novel neural RL method is proposed to solve the time-optimal control problem of a class of under-actuated robots, which is called the acrobot. The RL method uses a modified residual gradient reinforcement learning algorithm called RGNP (residual gradient with nonstationary policy). The RGNP algorithm not only has guaranteed convergence under certain conditions but also can ensure the performance of the approximated optimal policy, which is superior to the previous residual gradient algorithms. Simulation results of the learning control of the acrobot illustrate the effectiveness of the proposed method.

In this paper, a neural-network-based learning control method is proposed for the high-speed path tracking of unmanned ground vehicles (UGVs). In our work, a reinforcement-learning controller is combined with a conventional PID controller so that the robustness of PID control and the optimization ability of learning control can both be utilized. The proposed method uses an adaptive-critic learning controller with two outputs to tune the PD parameters online. The architecture of the learning controller includes a critic neural network and two action neural networks and the adaptive-heuristic-critic (AHC) learning algorithm is used to adjust the weights. Simulation results show that the path tracking performance of high-speed vehicles can be improved by the proposed method.

In this paper, a novel hybrid learning method is proposed for reinforcement learning problems with continuous state and action spaces. The reinforcement learning problems are modeled as Markov decision processes (MDPs) and the hybrid learning method combines evolutionary algorithms with gradient-based adaptive heuristic critic (AHC) algorithms to approximate the optimal policy of MDPs. The suggested method takes the advantages of evolutionary learning and gradient-based reinforcement learning to solve reinforcement learning problems. Simulation results on the learning control of an acrobot illustrate the efficiency of the presented method.

In this paper, we present a study on the visual navigation for a mobile robot. We perform the boundary detecting and color cluster techniques respectively to segment the road and nonroad. We propose a conception about useful information to decrease the computation, and compose the information with a special color coordinate base and the geometrical feature into a hybrid feature space to improve the color clustering. >

Reinforcement learning is a class of model-free learning control methods that can solve Markov decision problems. One difficulty for the application of reinforcement learning control is its slow convergence, especially in MDPs with continuous state space. In this paper, a modified structure of CMAC neural networks is proposed to accelerate reinforcement learning control. The modified structure is designed by incorporating a priori knowledge of learning control problems so that the efficiency and generalization ability of reinforcement learning can be improved. Simulation results on the cart-pole balancing problem illustrate the effectiveness of the proposed method.

The control of Autonomous Land Vehicle (ALV) is a kind of typical nonlinear control. Because the state of traffic and road are complex, so the control of ALV is complex and uncertain. Especially if the ALV runs in normal state of traffic, the control of ALV is becoming more complex and difficult. A good idea is that to let the control system of ALV to simulate human driver and develop a machine-leaning system in the control system of ALV. So that as the control system of ALV is driving ALV along road, the control system can learn from the experience of human driver according to the state of traffic and road. In other words, the control system of ALV is able to become more and more 'clever.' Of course it is a challenging and very difficult task. This paper analyzes the principle of machine-leaning system used for ALV and discusses the engineering method of developing a machine- leaning system used for ALV, when the ALV runs along highway in normal traffic. This paper advances the administrative levels of machine-leaning system and a method of fusing human intelligence into machine intelligence. This paper introduces our preliminary research on machine-leaning system used for ALV also, which is a kind of online machine-leaning system without teacher.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Lateral vehicle control is a nonlinear control problem. Because there exists many uncertain factors (e.g., imprecision of vehicle modeling and information measurement), it is limited to take advantage of the traditional control method based on a precise mathematical model to design a controller for lateral vehicle control. During the past several years, fuzzy control has emerged as one of the most active areas for research in dealing with the nonlinear control problem and has made many achievements. We designed a fuzzy logic controller for the road following of the autonomous land vehicle. The main sensor we used is the video camera that can provide the environment information around the vehicle. We have implemented this controller in the simulation. The controller drove the vehicle to follow the curved road and we got good tracking accuracy. In comparing with the PID controller, the fuzzy logic controller does not need a precise vehicle model and has better adaptability to parameter variation of the vehicle than the PID.

In this paper, we propose a novel fuzzy morphology (FM) which is induced by threshold decomposition. Its operators are the measures on a (sigma) _ring in range space of the membership functions of fuzzy sets, which depend upon the ordinary binary morphological operators of threshold sets of fuzzy sets. Some of the characteristic of these FM are similar to those of the traditional morphology. All the operators of these FM prove to be the morphological operators in the sense of complete lattice. It is shown how one can use binary morphological operators, thresholding techniques and stacking properties to implement these FM's operators. The VLSI implementation is simple and fast. The concept of fuzzification of set_intersection is introduced. This paper also presents a general algebraic approach to analysis fuzzy morphological operators on the space of fuzzy sets.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper, we give some results of threshold decomposition of soft morphological (SM's) filters. Some of new properties of SM have been developed by means of these results. We give the necessary and sufficient condition for the SM's operators to be the morphological operators in the sense of complete lattice. We propose the binary SM, which simplify the analysis of SM. An algebraic approach to bounded signal processing has been given. The VLSI implementation of SM's filters have been discussed.