Search

Estimation bias is an important index for evaluating the performance of reinforcement learning (RL) algorithms. The popular RL algorithms, such as Q -learning and deep Q -network (DQN), often suffer overestimation due to the maximum operation in estimating the maximum expected action values of the next states, while double Q -learning (DQ) and double DQN may fall into underestimation by using a double estimator (DE) to avoid overestimation. To keep the balance between overestimation and underestimation, we propose a novel integrated DE (IDE) architecture by combining the maximum operation and DE operation to estimate the maximum expected action value. Based on IDE, two RL algorithms: 1) integrated DQ (IDQ) and 2) its deep network version, that is, integrated double DQN (IDDQN), are proposed. The main idea of the proposed RL algorithms is that the maximum and DE operations are integrated to eliminate the estimation bias, where one estimator is stochastically used to perform action selection based on the maximum operation, and the convex combination of two estimators is used to carry out action evaluation. We theoretically analyze the reason of estimation bias caused by using nonmaximum operation to estimate the maximum expected value and investigate the possible reasons of underestimation existence in DQ. We also prove the unbiasedness of IDE and convergence of IDQ. Experiments on the grid world and Atari 2600 games indicate that IDQ and IDDQN can reduce or even eliminate estimation bias effectively, enable the learning to be more stable and balanced, and improve the performance effectively.

In this paper, we investigate the outer synchronization between two uncertain complex delayed networks with noise coupling. With an adaptive control scheme, sufficient conditions for the stochastic outer synchronization are developed based on the LaSalle invariance principle for stochastic differential equations. Different from existing results, our method is valid for networks with linear or nonlinear inner coupling. Finally, numerical simulations are performed to verify the effectiveness of the proposed synchronization scheme.

Wireless cellular networks are often modelled as different regular grids and the channel assignment problem for interference avoidance is formulated as a coloring problem of the grid graph, where channels assigned to interfering stations at distance i must be at least δ i apart, while the same channel can be reused in stations whose distance is at least σ. In this paper, we consider the channel assignment problem in a class of wireless cellular networks modelled as such regular grids which can be obtained by adding two edges to connect two pair of diagonal vertices of every square cell in the square grids. We confine our discussion to the case that the co-channel reuse distance σ is 4. We present one channel assignment algorithm for the case where the minimum channel separation δ i is 1 for all but adjacent stations and δ 1 ≥ 3 for adjacent stations, which is proved to be optimal. Also we present one potential optimal algorithm for the case that the minimum channel separation δ i is σ-i.

Cellular networks are often modelled as different regular grids and the channel assignment problem for interference avoidance is formulated as a coloring problem of the grid graph, where channels assigned to interfering stations at distance i must be at least δ i apart, while the same channel can be reused in vertices whose distance is at least σ. In this paper, we consider the channel assignment problem for cellular networks modelled as triangular grids for the case where the co-channel reuse distance σ is 4 and the minimum channel separation δi is σ-i. We give the lower and upper bounds of the problem. And to obtain the upper bound, we present a channel assignment algorithm, which is potentially optimal.

Wireless networks are often modelled as different grids and the channel assignment problem for interference avoidance is formulated as a coloring problem of the grid graph, where channels (i.e., colors) assigned to interfering stations (i.e., vertices) at distance i must be at least δi apart, while the same channel can be reused in vertices whose distance is at least σ. In this paper, we consider the channel assignment problem for wireless networks modelled as hexagonal and square grids. We present optimal channel assignment algorithms for the case where the co-channel reuse distance σ is 4 and the minimum channel separation δi is σ-i

Let G be a graph of order p, and let a and b and n be nonnegative integers with 1⩽a⩽b, and let g and f be two integer‐valued functions defined on V(G) such that a⩽g(x)⩽f(x)⩽b for all x∈V(G). A (g,f)‐factor of graph G is defined as a spanning subgraph F of G such that g(x)⩽dF(x)⩽f(x) for each x∈V(G). Then a graph G is called a (g,f,n)‐critical graph if after deleting any n vertices of G the remaining graph of G has a (g,f)‐factor. In this paper, we prove that every graph G is a (g,f,n)‐critical graph if its minimum degree is greater than p+a+b−2(a+1)p−bn+1. Furthermore, it is showed that the result in this paper is best possible in some sense.