Optimal Query Policy and Task Offloading in Dynamic Environments

Optimal task offloading decisions can be made if all the system parameters are known perfectly. However, due to some practical constraints, the task node can only query a subset of helper nodes each time and only the knowledge about these queried nodes are updated accordingly. Considering the fact that the system parameters of the helper nodes vary over time, the task node needs to make the offloading decisions based on the available system knowledge including both the updated and the outdated ones. In this paper, we investigate the optimal query policy at the task node such that task offloading performance is optimized in the long term. We first solve this problem under the reinforcement learning (RL) framework assuming the knowledge of the dynamics of the system parameters. In the absence of the system dynamics, we further put forward an efficient low-complexity algorithm to learn the optimal query decision by exploiting the deep RL tools. Numerical results corroborate that the performance of the proposed algorithm is close to the optimal one.