Back to Search
Start Over
Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks
- Source :
- IEEE Transactions on Wireless Communications. 20:7519-7537
- Publication Year :
- 2021
- Publisher :
- Institute of Electrical and Electronics Engineers (IEEE), 2021.
-
Abstract
- Opportunistic computation offloading is an effective method to improve the computation performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. In particular, we aim to design an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. The online algorithm is practical in the sense that the decisions for each time frame are made without the assumption of knowing future channel conditions and data arrivals. We formulate the problem as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem that jointly determines the binary offloading (each user computes the task either locally or at the edge server) and system resource allocation decisions in sequential time frames. To address the coupling in the decisions of different time frames, we propose a novel framework, named LyDROO, that combines the advantages of Lyapunov optimization and deep reinforcement learning (DRL). Specifically, LyDROO first applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems. By doing so, it guarantees to satisfy all the long-term constraints by solving the per-frame subproblems that are much smaller in size. Then, LyDROO integrates model-based optimization and model-free DRL to solve the per-frame MINLP problems with low computational complexity. Simulation results show that under various network setups, the proposed LyDROO achieves optimal computation performance while stabilizing all queues in the system. Besides, it induces very low execution latency that is particularly suitable for real-time implementation in fast fading environments.<br />Comment: The paper has been accepted for publication by IEEE Trans. Wireless Communications, the source codes associated with the paper are available at https://github.com/revenol/LyDROO. arXiv admin note: text overlap with arXiv:2102.03286
- Subjects :
- Networking and Internet Architecture (cs.NI)
FOS: Computer and information sciences
Lyapunov function
Mathematical optimization
Mobile edge computing
Computational complexity theory
Computer science
Applied Mathematics
Computation
Lyapunov optimization
Computer Science Applications
Computer Science - Networking and Internet Architecture
symbols.namesake
symbols
Computation offloading
Reinforcement learning
Electrical and Electronic Engineering
Online algorithm
Subjects
Details
- ISSN :
- 15582248 and 15361276
- Volume :
- 20
- Database :
- OpenAIRE
- Journal :
- IEEE Transactions on Wireless Communications
- Accession number :
- edsair.doi.dedup.....fc8d921dc027e83e85a8f3fbdfaa62a0
- Full Text :
- https://doi.org/10.1109/twc.2021.3085319