A Summary on Some Typical Adaptive Dynamic Programming Schemes

This paper sums up four typical schemes of adaptive dynamic programming (ADP). The diagrams are provided and the algorithms of various schemes are described, which is convenient for comparison. Some schemes in this paper belong to the group of action-dependent (AD) adaptive critic designs, which features without a model network in the design. For simplicity of notation, we do not use the prefix AD. The learning process of ADP is accomplished by updating the weights of the networks. The weight updating processes of some networks in GDHP scheme are introduced.