Search

Robust reinforcement learning (RL) aims to find a policy that optimizes the worst-case performance in the face of uncertainties. In this paper, we focus on action robust RL with the probabilistic policy execution uncertainty, in which, instead of always carrying out the action specified by the policy, the agent will take the action specified by the policy with probability $1-\rho$ and an alternative adversarial action with probability $\rho$. We establish the existence of an optimal policy on the action robust MDPs with probabilistic policy execution uncertainty and provide the action robust Bellman optimality equation for its solution. Furthermore, we develop Action Robust Reinforcement Learning with Certificates (ARRLC) algorithm that achieves minimax optimal regret and sample complexity. Furthermore, we conduct numerical experiments to validate our approach's robustness, demonstrating that ARRLC outperforms non-robust RL algorithms and converges faster than the robust TD algorithm in the presence of action perturbations.

Due to the broad range of applications of multi-agent reinforcement learning (MARL), understanding the effects of adversarial attacks against MARL model is essential for the safe applications of this model. Motivated by this, we investigate the impact of adversarial attacks on MARL. In the considered setup, there is an exogenous attacker who is able to modify the rewards before the agents receive them or manipulate the actions before the environment receives them. The attacker aims to guide each agent into a target policy or maximize the cumulative rewards under some specific reward function chosen by the attacker, while minimizing the amount of manipulation on feedback and action. We first show the limitations of the action poisoning only attacks and the reward poisoning only attacks. We then introduce a mixed attack strategy with both the action poisoning and the reward poisoning. We show that the mixed attack strategy can efficiently attack MARL agents even if the attacker has no prior information about the underlying environment and the agents' algorithms.

Due to the broad range of applications of reinforcement learning (RL), understanding the effects of adversarial attacks against RL model is essential for the safe applications of this model. Prior theoretical works on adversarial attacks against RL mainly focus on either observation poisoning attacks or environment poisoning attacks. In this paper, we introduce a new class of attacks named action poisoning attacks, where an adversary can change the action signal selected by the agent. Compared with existing attack models, the attacker's ability in the proposed action poisoning attack model is more restricted, which brings some design challenges. We study the action poisoning attack in both white-box and black-box settings. We introduce an adaptive attack scheme called LCB-H, which works for most RL agents in the black-box setting. We prove that the LCB-H attack can force any efficient RL agent, whose dynamic regret scales sublinearly with the total number of steps taken, to choose actions according to a policy selected by the attacker very frequently, with only sublinear cost. In addition, we apply LCB-H attack against a popular model-free RL algorithm: UCB-H. We show that, even in the black-box setting, by spending only logarithm cost, the proposed LCB-H attack scheme can force the UCB-H agent to choose actions according to the policy selected by the attacker very frequently.

It is important of the detection equipment and technology in the pipeline which means to prevent pipeline accidents before they occur. In this paper, an electromagnetic holography detection robot for nondestructive detection of oil and gas pipelines is designed. The mechanical structure and function, electronic system and upper computer analysis system of the robot are introduced. The integration of defect detection and defect identification can realize the identification and evaluation of high-risk factors for safe operation of oil and gas pipelines, such as pipeline corrosion, weld defects, mechanical damage, etc. The experimental results show that the robot can identify, locate and quantify the defects in the process of pipeline defect detection, which is an important means to guide the reasonable maintenance and integrity management of pipeline.

Intrinsic electrocatalytic activities of a material and electrode architecture for supporting the active material are crucial factors that determine the electrochemical sensing performance. In this work, tetragonal β-MnO2 is taken as an electrocatalytic material for enzymeless glucose sensing. To construct a three dimensional nanostructure electrode, β-MnO2 is directly grown on flexible carbon fiber fabric forming the micro/nanorod arrays (RA). For comparison, β-MnO2 micro/nanorod powder (RP) is immobilized on graphite electrode by Nafion binder. Both β-MnO2 RA and RP electrodes are investigated systematically as enzymeless glucose sensors and the results indicate the high sensitivity of 1650.6 μA mM−1 cm−2, wide linear range up to 4.5 mM for the RA electrode, which shows much better performance than that of RA electrode. The sensitivity of RA electrode towards electro-oxidation of glucose is about 1.7-fold as much as that for RP electrode, mainly benefiting from the favorable electrode architecture. Besides, the RA electrode shows extraordinary stability, reproducibility and selectivity. Our investigation indicates that the electrode based on the β-MnO2 micro/nanorod array directly grown on carbon fiber fabric is outstanding candidate for high efficiency flexible enzymeless glucose sensing.

The hydroelectric generating set is suffered from the joint shock of water flow, machinery and electricity, its condition monitoring and diagnosis is complicated. Besides carefully to set suitable condition parameter measurement points, the diagnostic model is another key problem. In additional, how to link with the SCADA and MIS of hydropower station for the condition monitoring and diagnosis system must be elaborately considered. This paper briefly introduces the hardware structure, software design, and the diagnostic model study of condition monitoring and diagnosis system for a hydropower station. Parts of research achievements have been applied to the real machine successfully.