Search

Redundant degree-of-freedom (DOF) manipulators offer increased flexibility and are better suited for obstacle avoidance, yet precise control of these systems remains a significant challenge. This paper addresses the issues of slow training convergence and suboptimal stability that plague current deep reinforcement learning (DRL)-based control strategies for redundant DOF manipulators. We propose a novel DRL-based intelligent control strategy, FK-DRL, which integrates the manipulator's forward kinematics (FK) model into the control framework. Initially, we conceptualize the control task as a Markov decision process (MDP) and construct the FK model for the manipulator. Subsequently, we expound on the integration principles and training procedures for amalgamating the FK model with existing DRL algorithms. Our experimental analysis, applied to 7-DOF and 4-DOF manipulators in simulated and real-world environments, evaluates the FK-DRL strategy's performance. The results indicate that compared to classical DRL algorithms, the FK-DDPG, FK-TD3, and FK-SAC algorithms improved the success rates of intelligent control tasks for the 7-DOF manipulator by 21%, 87%, and 64%, respectively, and the training convergence speeds increased by 21%, 18%, and 68%, respectively. These outcomes validate the proposed algorithm's effectiveness and advantages in redundant manipulator control using DRL and FK models. [ABSTRACT FROM AUTHOR]

【Background and objective】 Haihe basin is one of the most polluted regions in northern China with most of its pollutants originating from agricultural production. The purpose of this paper is to analyze spatial distribution of nitrogen (N) and phosphorus (P) in the basin, as well as their effect on water quality and origins. 【Method】 Using the InVEST model, we analyzed N and P load from farmlands, as well as their discharges to the river, from which we estimated their concentration in the runoff. The sources of N and P were analyzed using GIS based on hydrological network in the basin. 【Result】 ①The N and P released from farmland to the river was 2.41 kg/hm2, 0.56 kg/hm2 respectively in 2015, with total N and P discharged to the river being 3.493 4 million tons and 0.807 7 million tons respectively. Average N and P concentration in runoff in the river was 5.97 mg/L and 1.47 mg/L respectively, with N and P concentrations in 55% of the river exceeded the Surface Water Class V Standard. ②N and P released from farmlands in the mountain areas was lower than that from the middle plain regions. N and P concentration and N and P discharged to the river vary spatially, with the latter concentrated in Tuhaimajia, Zhangwei and Ziya sub-basins, while the former located in Tuhaimajia river, middle and low reaches of Heilonggang river, the tributaries of Daqing river and the upstream of Ziya river. N and P concentrations in the upstream and the tributaries of the basin were higher than that in the mainstream. ③The hotspot areas of N and P account for 14.45% and 16.02% of the total areas, which contributed 23.39% and 27.71% of N and P discharged to the river. 【Conclusion】 N and P pollution from farmland in Haihe basin is severe, with the mountainous areas in the northwest more polluted than the central and southern plains. N and P pollution in the central and southern basin, as well as the areas in the proximity of Shijiazhuang and Qinhuangdao are most concerning.