1. Research on HP Model Optimization Method Based on Reinforcement Learning
- Author
-
Zhou Fengli and Lin Xiaoli
- Subjects
0303 health sciences ,Polynomial ,Markov chain ,Computer science ,Ant colony optimization algorithms ,Swarm behaviour ,Particle swarm optimization ,02 engineering and technology ,03 medical and health sciences ,Local optimum ,Robustness (computer science) ,Genetic algorithm ,0202 electrical engineering, electronic engineering, information engineering ,Reinforcement learning ,020201 artificial intelligence & image processing ,Greedy algorithm ,Algorithm ,030304 developmental biology - Abstract
Protein structure prediction is an important factor in the area of bioinformatics. Predicting the two-dimensional structure of proteins based on the hydrophobic polarity model (HP model) is a typical non-deterministic polynomial(NP)-hard problem. Currently, HP model optimization methods include the greedy algorithm, particle swarm optimization, genetic algorithm, ant colony algorithm and the Monte-Carlo simulation method. However, the robustness of these methods are not sufficient, and it is easy to fall into a local optimum. Therefore, a HP model optimization method, based on reinforcement learning was proposed. In the full state space, a reward function based on energy function was designed and a rigid overlap detection rule was introduced. By using the characteristics of the continuous Markov optimal decision and maximizing global cumulative return, the global evolutionary relationship in biological sequences was fully exploited, and effective and stable predictions were retrieved. Eight classical sequences from publications and Uniref50 were selected as experimental objects. The robustness, convergence and running time were compared with the greedy algorithm and particle swarm optimization algorithm, respectively. Both reinforcement method and swarm optimization method can find all the lowest energy structures for these eight sequences, while the greedy algorithm only detects 62.5%. Compared with particle swarm optimization, the running time of the reinforcement method is 63.9% lower than that of particle swarm optimization.
- Published
- 2019