Search

Recently, deep reinforcement learning (RL) has been a hot topic due to its high capability in solving complex decision-making tasks. Although deep RL has achieved remarkable results in many fields, efficient exploration remains one of its most challenging issues. Conventional exploration heuristic methods, such as random exploration, have proven to be not suitable for complex RL tasks along with high-dimensional states and large action spaces. In this paper, we propose a novel lightweight and general neural network module for effective global exploration, called the Noisy Attention Exploration Module (NAEM), of which the key insight is to introduce parametric and learnable Gaussian noise into the attention mechanism for global exploration. NAEM is a general structure based on the Convolutional Block Attention Module (CBAM), which retains the ability of attention to enhance feature representation for any CNNs. In order to evaluate our module, we embed it into both value-based and actor-critic RL algorithms to test their performance improvement over related agents. The experimental results show that both of the modified agents achieve a performance improvement of more than 130% on most Atari games when compared with their original versions. In addition, for the NoisyNet agents, the training time can be reduced by about 30% through using NAEM as an alternative exploration.

A novel radiation hardened 12T memory cell (RH-12T) is proposed to address single event upset (SEU) problems in 65nm CMOS technology. It eliminates the possibility of a sensitive “0” storage node upset by surrounding the output nodes with NMOS, realizing a full resistance for any single node upset. Furthermore, less sensitive node pairs are obtained in circuit design compared to the DICE cell, which reduces the sensitivity to single event multiple-node upsets (SE-MNU). Hspice simulation shows that it has an excellent performance considering read and write access time with an acceptable SNM. Circuit-SEU simulation demonstrates that it is not only immune to upsets any single sensitive node, but also tolerant to multiple node upset on specific nodes with sharing charge of 100fC.

In this paper, a new project variable metric algorithm, for solving the electromagnetic inverse problems, is proposed. The electromagnetic inverse problem is considered to be a nonlinear programming with equality constraints. By using an appropriate parameter, the problem is transferred an unconstrained optimization problem. The method is proposed by computing an explicit search direction per each iterative.

Pneumatic actuators have a number of advantages over electric motors, including strength-to-weight ratio, tunable compliance at the mechanism level, robustness, as well as price. Their properties are in many ways similar to muscle properties, which further makes them a good choice for bio-inspired robotic designs. However they are considered harder to control, and often avoided. Here we report results on modeling and control of a 2-dof robot, as well as preliminary results on a state-of-the-art 38-dof humanoid. Contrary to popular belief, we found it surprisingly easy to work with these pneumatically-actuated robots and obtained high tracking performance. We were also able to achieve end-effector control of the highly redundant arm of the humanoid. This was done by building parametric models, fitting them to experimental data, designing model-based feedback controllers, and optimizing their performance.

This paper studies global asymptotic stability of a general class of neural networks with time delays by utilizing Razumikhin theorem and the linear matrix inequality technique. Distinct difference from other analytical approaches lies in ldquolinearizationrdquo of the neural network model, by which the considered neural network model is transformed into a linear time-variant system. New sufficient conditions ensuring global asymptotic stability of the unique equilibrium point of delayed neural networks are obtained. The obtained conditions show to be less conservative and restrictive than those reported in the literature. A numerical simulation is given to illustrate the validity of our results.

Synchronization stability in complex dynamical networks with coupling delays is investigated. Nonlinear complex network system is transformed to linear system. A new criterion for global synchronization stability for the complex networks with coupling delays has been derived using an approach combining the Lyapunov-Krasovskii functional with LMI techniques. The results are related to the size of delays. It has been illustrated that the proposed result generalizes and improves those reported in the literature. A numerical simulation is also given to illustrate the validity of our result.