Search

Vehicular Edge Computing (VEC) is a crucial application of Mobile Edge Computing (MEC) in vehicular networks. In VEC networks, the computation tasks of vehicle terminals (VTs) can be offloaded to nearby MEC servers, overcoming the limitations of VTs' processing power and reducing latency caused by distant cloud communication. However, a mismatch between VTs' demanding tasks and MEC servers' limited resources can overload MEC servers, impacting Quality of Service (QoS) for computationally intensive tasks. Additionally, vehicle mobility can disrupt communication with static MEC servers, further affecting VTs' QoS. To address these challenges, this paper proposes a vehicle collaborative partial computation offloading model. This model allows VTs to offload tasks to two types of service nodes: collaborative vehicles and MEC servers. Factors like a vehicle's mobility, remaining battery power, and available computational power are also considered when evaluating its suitability for collaborative offloading. Furthermore, we design a deep reinforcement learning-based strategy for collaborative partial computation offloading that minimizes overall task delay while meeting individual latency constraints. Experimental results demonstrate that compared to traditional approaches without vehicle collaboration, this scheme significantly reduces latency and achieves a significant reduction (around 2%) in the failure rate under tighter latency constraints. [ABSTRACT FROM AUTHOR]

To address the current challenge of smart devices in healthcare Internet of things (IoT) struggling to efficiently process intensive applications in real-time, a collaborative cloud-edge offloading model tailored for ultra-dense edge computing (UDEC) networks is developed. While numerous studies have delved into the optimization of offloading in mobile edge computing (MEC), it is imperative to consider non-orthogonal multiple access (NOMA) as a physical technology when addressing the offloading optimization process in MEC. The multiuser sharing of spectrum resources in NOMA can enhance the network spectrum utilization and reduce the computational delay when users transmit computing tasks. Consequently, a model for NOMA-assisted UDEC systems is proposed. The model takes into account joint offloading decisions, computational resources, and sub-channel resources and is modeled as a complex nonlinear mixed-integer programming problem. The aim is to decrease the task execution delay and energy consumption of smart devices while ensuring that users' maximum acceptable delay for processing medical computational tasks is met efficiently and in a timely manner. Deep deterministic policy gradient (DDPG), a deep reinforcement learning method, is employed to solve the joint optimization problem. The final simulation results show that the algorithm converges well. The proposed offloading scheme can reduce the system cost by 54.5 and 69.9% in comparison with scenarios where users solely perform local computations and offload their tasks to the base station (BS). The application of NOMA communication in our offloading scheme boosts network spectrum utilization and trims down the system cost by 87.09% when contrasted with orthogonal multiple access (OMA). [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) can enhance the computation capabilities of smart mobile devices for computation-intensive mobile applications via supporting computation offloading efficiently. However, the limitation of wireless resources and computational resources of edge servers often becomes the bottlenecks to realizing the developments of MEC. In order to address the computation offloading problem in the time-varying wireless networks, the offloading decisions and the allocation of radio and computation resources need to be jointly managed. Traditional optimization methods are challenging to deal with the combinatorial optimization problem in complex real-time dynamic network environments. Therefore, we propose a deep reinforcement learning (DRL)-based optimization approach, named DRJOA, which jointly optimizes offloading decisions, computation, and wireless resources allocation. The optimization algorithm based on DRL has the advantages of fast solving speed and strong generalization ability, which makes it possible to solve combinatorial optimization problems online. Simulation results show that our proposed DRJOA in this study dramatically outperforms the benchmark methods for offloading decisions and system utility. [ABSTRACT FROM AUTHOR]

Mobile edge computing requires more and more high-performance servers, resulting in increased energy consumption. As an effective means to reduce energy consumption, virtual machine placement (VMP) has been widely studied. In the edge computing environment, as the number of terminal device requests continues to increase, the scale of VMP becomes larger and larger, and existing research algorithms may take a long time to converge. The reason is that as the number of VMs increases, the search space of the policy becomes larger and the agent needs to interact with the environment for a longer time to make the best decision. In addition, existing research methods only consider reducing energy consumption, rarely consider the response latency of virtual machines, and almost ignore the dynamic changes of the edge environment. To overcome these drawbacks, we propose a virtual machine placement algorithm based on meta-reinforcement learning, which consists of an inner and outer loop. The inner loop designs a deep reinforcement learning algorithm combined with the order exchange and migration mechanism to generate the best decision, and the outer loop provides meta-strategy parameters for the inner loop based on meta-learning to accelerate the convergence capability of the inner loop, thereby obtaining efficient virtual machine placement decisions quickly from a new environment. Through simulation experiments, we demonstrate that our approach effectively reduces the energy consumption of the edge server and the response latency of VMs at different problem sizes compared to the three baseline algorithms. At the same time, it quickly adapts to the new environment with only a small number of gradient updates. [ABSTRACT FROM AUTHOR]

The continual development of mobile edge computing efficiently solves the problem that mobile devices are unable to handle computation-intensive tasks due to their computation capacity and battery restrictions. In this paper, we consider mobile awareness and dynamic battery charging in a multi-user and multi-server mobile edge computing system, where various tasks are generated successively on the user devices. Servers act as learning agents and collaborate with user devices to develop task partitioning and computation resource allocation strategies. With the purpose of decreasing task failure rate and improving system utility in the long term, which is closely related to latency, energy consumption, and server cost, optimal strategies are demanded by the system. We model the joint optimization problem as a multi-agent Markov decision process game. And a deep reinforcement learning method based on the multi-agent deep deterministic policy gradient algorithm is proposed, which employs neural networks and works in a centralized training and decentralized execution manner to optimize the strategies. Finally, simulation results demonstrate the effectiveness of our proposed algorithm in terms of reducing task failure rate and improving system utility. [ABSTRACT FROM AUTHOR]

In this paper, we consider an integrated sensing, communication, and computation (ISCC) system to alleviate the spectrum congestion and computation burden problem. Specifically, while serving communication users, a base station (BS) actively engages in sensing targets and collaborates seamlessly with the edge server to concurrently process the acquired sensing data for efficient target recognition. A significant challenge in edge computing systems arises from the inherent uncertainty in computations, mainly stemming from the unpredictable complexity of tasks. With this consideration, we address the computation uncertainty by formulating a robust communication and computing resource allocation problem in ISCC systems. The primary goal of the system is to minimize total energy consumption while adhering to perception and delay constraints. This is achieved through the optimization of transmit beamforming, offloading ratio, and computing resource allocation, effectively managing the trade-offs between local execution and edge computing. To overcome this challenge, we employ a Markov decision process (MDP) in conjunction with the proximal policy optimization (PPO) algorithm, establishing an adaptive learning strategy. The proposed algorithm stands out for its rapid training speed, ensuring compliance with latency requirements for perception and computation in applications. Simulation results highlight its robustness and effectiveness within ISCC systems compared to baseline approaches. [ABSTRACT FROM AUTHOR]

Various edge collaboration schemes that rely on reinforcement learning (RL) have been proposed to improve the quality of experience (QoE). Deep RL (DRL) maximizes cumulative rewards through large-scale exploration and exploitation. However, the existing DRL schemes do not consider the temporal states using a fully connected layer. Moreover, they learn the offloading policy regardless of the importance of experience. They also do not learn enough because of their limited experiences in distributed environments. To solve these problems, we proposed a distributed DRL-based computation offloading scheme for improving the QoE in edge computing environments. The proposed scheme selects the offloading target by modeling the task service time and load balance. We implemented three methods to improve the learning performance. Firstly, the DRL scheme used the least absolute shrinkage and selection operator (LASSO) regression and attention layer to consider the temporal states. Secondly, we learned the optimal policy based on the importance of experience using the TD error and loss of the critic network. Finally, we adaptively shared the experience between agents, based on the strategy gradient, to solve the data sparsity problem. The simulation results showed that the proposed scheme achieved lower variation and higher rewards than the existing schemes. [ABSTRACT FROM AUTHOR]

In many IoT scenarios, the resources of terminal devices are limited, and it is difficult to provide services with low latency and low energy consumption. Mobile edge computing is an effective solution by offloading computing tasks to edge server processing. There are some problems in the existing offloading decision algorithms: the offloading decision method based on heuristic algorithms cannot dynamically adjust the policy in the changing environment; the offloading algorithm based on deep reinforcement learning will lead to slow convergence and poor exploration effect due to the problem of dimension explosion. To solve the above problems, this paper designs an offloading decision algorithm to make dynamic decisions in a mobile edge computing network with multi-device access. The algorithm comprehensively considers the energy consumption of terminal equipment, offloading overhead, average delay and success rate of task completion, aiming to achieve the highest total revenue of the whole system in a period of time. In this work, the online offloading problem is abstracted as a Markov decision process. Based on the Double Dueling Deep Q-Network (D3QN) algorithm, the offloading decision is designed to adapt to the highly dynamic environment of the edge computing network and solve the problem of high state space complexity. In addition, this paper innovatively introduces a clustering algorithm into deep reinforcement learning (DRL) to preprocess the action space and solve the explosion problem of the action space dimension caused by the increase of terminal devices. The experimental results show that the proposed algorithm is superior to the baseline strategies such as Deep Q-Network (DQN) algorithm in convergence speed and total reward. [ABSTRACT FROM AUTHOR]

In mobile edge computing, service migration can not only reduce the access latency but also reduce the network costs for users. However, due to bandwidth bottleneck, migration costs should also be considered during service migration. In this way, the trade‐off between benefits of service migration and total service costs is very important for the cloud service providers. In this paper, we propose an efficient dynamic service migration algorithm named SMDQN, which is based on reinforcement learning. We consider each mobile application service can be hosted on one or more edge nodes and each edge node has limited resources. SMDQN takes total delay and migration costs into consideration. And to reduce the size of Markov decision process space, we devise the deep reinforcement learning algorithm to make a fast decision. We implement the algorithm and test the performance and stability of it. The simulation result shows that it can minimize the service costs and adapt well to different mobile access patterns. [ABSTRACT FROM AUTHOR]

• We propose a task offloading framework based on a trust evaluation mechanism. • We build the MEC model, and the problem is modeled as a Markov decision process. • We use the entropy weight method to estimate the optimization objectives. • Experimental results show that DDTMOA has better performance than other algorithms. As a new computing model, mobile edge computing (MEC) is designed to better deal with various forms of service requests, such as computing intensity and delay sensitivity, in the era of big data and the Internet of Things (IoT). However, the development of MEC is still in its infancy, and many issues need to be further investigated. One of the key issues that needs to be addressed in MEC is rational task offloading. Due to the dynamic, real time and complex nature of the MEC environment, the security and reliability of edge data are becoming increasingly important. Based on the above problems, we construct a task offloading integrated trust evaluation mechanism and, combined with the double deep Q-network (DDQN) algorithm in deep reinforcement learning (DRL), propose a novel task offloading algorithm, named DDTMOA. Simulation results show that the DDTMOA algorithm can effectively reduce the average task response time and total system energy consumption while ensuring task offloading performance compared to other classical algorithms. [ABSTRACT FROM AUTHOR]

When an unmanned aerial vehicle (UAV) performs tasks such as power patrol inspection, water quality detection, field scientific observation, etc., due to the limitations of the computing capacity and battery power, it cannot complete the tasks efficiently. Therefore, an effective method is to deploy edge servers near the UAV. The UAV can offload some of the computationally intensive and real-time tasks to edge servers. In this paper, a mobile edge computing offloading strategy based on reinforcement learning is proposed. Firstly, the Stackelberg game model is introduced to model the UAV and edge nodes in the network, and the utility function is used to calculate the maximization of offloading revenue. Secondly, as the problem is a mixed-integer non-linear programming (MINLP) problem, we introduce the multi-agent deep deterministic policy gradient (MADDPG) to solve it. Finally, the effects of the number of UAVs and the summation of computing resources on the total revenue of the UAVs were simulated through simulation experiments. The experimental results show that compared with other algorithms, the algorithm proposed in this paper can more effectively improve the total benefit of UAVs. [ABSTRACT FROM AUTHOR]

Currently, most edge-based Deep Reinforcement Learning (Deep-RL) applications have been deployed in the edge network, however, their mainstream studies are still short of adequate considerations on its limited compute and bandwidth resources. In this paper, we investigate the near on-policy of actions taking in distributed Deep-RL architecture, and propose a "hybrid near on-policy" Deep-RL framework, called Coknight , by leveraging a game-theory based DNN partition approach. We first formulate the partition problem into a variant of knapsack problem in device-edge setting, and then transform it into a potential game with a formal proof. Finally, we show the problem is NP-complete whereby an efficient distributed algorithm based on the potential game theory is developed from device perspective to achieve fast and dynamic partitioning. Coknight not only significantly improves the resource efficiency of the Deep-RL but also allows the inference to enforce the scalability of the actor policy. We prototype the framework with extensive experiments to validate our findings. The experimental results show that with the premise of a rapid convergence guarantee, Coknight , compared with Seed-RL , can reduce GPU utilization by 30% while providing large-scale scalability. • Leverage the DNN partition technology in the device-edge settings to accelerate DRL. • Formulate partition problem as a knapsack problem and prove its NP-Completeness. • Transform the problem into a multi-player game and prove it is a potential game. • Design a dynamic algorithm with polynomial time complexity for DNN partition. • Implement the framework to conduct physical tests and large-scale simulation studies. [ABSTRACT FROM AUTHOR]

The ubiquitous Internet of Things (IoTs) devices spawn growing mobile services of applications with computationally-intensive and latency-sensitive features, which increases the data traffic sharply. Driven by container technology, microservice is emerged with flexibility and scalability by decomposing one service into several independent lightweight parts. To improve the quality of service (QoS) and alleviate the burden of the core network, caching microservices at the edge of networks empowered by the mobile edge computing (MEC) paradigm is envisioned as a promising approach. However, considering the stochastic retrieval requests of IoT devices and time-varying network topology, it brings challenges for IoT devices to decide the caching node selection and microservice replacement independently without complete information of dynamic environments. In light of this, a MEC-enabled di stributed cooperative m icroservice ca ching scheme, named DIMA, is proposed in this paper. Specifically, the microservice caching problem is modeled as a Markov decision process (MDP) to optimize the fetching delay and hit ratio. Moreover, a distributed double dueling deep Q-network (D3QN) based algorithm is proposed, by integrating double DQN and dueling DQN, to solve the formulated MDP, where each IoT device performs actions independently in a decentralized mode. Finally, extensive experimental results are demonstrated that the DIMA is well-performed and more effective than existing baseline schemes. [ABSTRACT FROM AUTHOR]

In a multiple-services mobile edge computing network environment, the requirements and popularity of services, the computing requirement and the amount of data transferred from users to edge servers are dynamic with time. How to adaptively adjust the caching subset of total service types in the resource-limited edge server and determine the task offloading destination and resource allocation decisions to obtain the optimum overall system performance is a challenge problem. To solve this challenge, this paper firstly converted it into a Markov decision process, then proposed a soft actor-critic (SAC) -based deep reinforcement learning algorithm to jointly determine not only the discrete decisions of service caching and task offloading but also the continuous allocation of bandwidth and computing resource. The algorithm employed an effective trick of converting multiple continuous action outputs into discrete action choices to deal with the key design challenge that arised from continuous-discrete hybrid action space while improving the accuracy of algorithm decisions. Additionally, the algorithm integrated an efficient reward function that added auxiliary reward terms to improve resource utilization. Extensive numerical results show that compared with other baseline algorithms, the proposed algorithm can effectively reduce the long-term average completion delay of tasks while accessing excellent performance in terms of stability. [ABSTRACT FROM AUTHOR]

In mobile edge computing environment, due to resources constraints of edge devices, when user locations continue changing, the network will be delayed or interrupted, which affects the quality of user's service access. Previous studies have shown that deploying multiple microservice instances with the same function on multiple edge servers through container technology can solve this problem. However, how to choose the optimal microservice instance from multiple servers in a cloud-edge hybrid environment needs to be further investigated. This paper studies the selection of microservices problem based on the dynamic and heterogeneous characters of the cloud-edge collaborative environment, which is defined as a microservice selection and scheduling optimization problem (MSSP) to minimize users' service access delay. To cope with the complexity of cloud-edge collaborative environment and improve learning efficiency, MSSP is regarded as a Markov decision-making process, a Deep Deterministic Policy Gradient algorithm for microservice selection called MS_DDPG is then proposed to solve this problem, and the microservice selection strategy experience pool is established in MS_DDPG. Performance evaluations of MS_DDPG based on a real dataset and some synthetic dataset have been conducted, and the results show that MS_DDPG outperforms the other three baseline algorithms. In terms of average access delay, MS_DDPG is reduced by 23.82%. We also validate the performance of MS_DDPG by increasing the number of user requests, and the results also show that MS_DDPG obtains better performance in scalability. [ABSTRACT FROM AUTHOR]

Mobile user terminals in 5G networks can generate massive computational workloads, which require sufficient computation and caching resources, and the processors of user terminals cannot tackle these workloads. Emerging mobile edge computing (MEC) has become the key to solving the computation problem by offloading computation‐intensive workloads to the MEC server. To make full use of the limited resources on the MEC side, service caching can pre‐store specific executable programs, databases, or libraries for executing offloaded workloads from user terminals. In this study, a service caching‐assisted MEC model is designed. According to the proposed MEC model, a decentralized model‐free deep reinforcement learning algorithm‐based server caching optimization policy (DDSCOP) is proposed to minimize the long‐term weighted average cost. Considering the time‐varying request workloads from user terminals, the stochastic channel state, and the arrival of renewable resources of the MEC server, DDSCOP can find the near‐optimal service caching decision‐making policy by training the neural networks, that is, whether the service caching can be hosted by the MEC server, and which MEC server hosts the service caching. The numerical experimental results verify the convergence and effectiveness of DDSCOP through extensive parameters configuration, and DDSCOP outperforms the three baseline algorithms. [ABSTRACT FROM AUTHOR]

Smart mobile devices have recently emerged as a promising computing platform for computation tasks. However, the task performance is restricted by the computing power and battery capacity of mobile devices. Mobile edge computing, an extension of cloud computing, solves this problem well by providing computational support to mobile devices. In this paper, we discuss a mobile edge computing system with a server and multiple mobile devices that need to perform computation tasks with priorities. The limited resources of the mobile edge computing server and mobile device make it challenging to develop an offloading strategy to minimize both delay and energy consumption in the long term. To this end, an online algorithm is proposed, namely, the double reinforcement learning computation offloading (DRLCO) algorithm, which jointly decides the offloading decision, the CPU frequency, and transmit power for computation offloading. Concretely, we first formulate the power scheduling problem for mobile users to minimize energy consumption. Inspired by reinforcement learning, we solve the problem by presenting a power scheduling algorithm based on the deep deterministic policy gradient (DDPG). Then, we model the task offloading problem to minimize the delay of tasks and propose a double Deep Q-networks (DQN) based algorithm. In the decision-making process, we fully consider the influence of task queue information, channel state information, and task information. Moreover, we propose an adaptive prioritized experience replay algorithm to improve the model training efficiency. We conduct extensive simulations to verify the effectiveness of the scheme, and the simulation results show that compared with the conventional schemes, our method reduces the delay by 48% and the energy consumption by 53%. • An online computing offload model for mobile edge computing system. • Based on double DQN and DDPG to reduce delay and energy consumption. • An adaptive prioritized experience replay algorithm to improve training efficiency. [ABSTRACT FROM AUTHOR]

Mobile technologies is evolving so rapidly in every aspect, utilizing every single resource in the form of applications which creates advancement in day to day life. This technological advancements overcomes the traditional computing methods which increases communication delay, energy consumption for mobile devices. In today's world, Mobile Edge Computing is evolving as a scenario for improving in these limitations so as to provide better output to end users. This paper proposed a secure and energy-efficient computational offloading scheme using LSTM. The prediction of the computational tasks done using the LSTM algorithm. A strategy for computation offloading based on the prediction of tasks, and the migration of tasks for the scheme of edge cloud scheduling based on a reinforcement learning routing algorithm help to optimize the edge computing offloading model. Experimental results show that our proposed algorithm Intelligent Energy Efficient Offloading Algorithm (IEEOA), can efficiently decrease total task delay and energy consumption, and bring much security to the devices due to the firewall nature of LSTM. [ABSTRACT FROM AUTHOR]

With Mobile Edge Computing (MEC), computational intensive applications can be offloaded to the nearby edge servers to support latency-sensitive applications on mobile devices. Different from the cloud, edge servers usually have limited resources, and then selecting which edge server to run the offloaded computation becomes an important issue. Although server selection has received considerable attention, not much work has been done to consider the limited coverage of the edge server and the frequent user movement, which introduce many dynamic changing factors affecting the workload of the edge server and making it hard to achieve long-term optimum in the edge server selection. To deal with these challenges, we model the problem of continuous server selection as a Markov Decision Process (MDP). The difficulty of this problem is that achieving long-term optimum requires future knowledge, such as user mobility, server workload, etc, which is not known a priori. We do not have such knowledge and thus cannot find the optimal policy through traditional methods. To address this problem, we propose a Deep Reinforcement Learning (DRL) based algorithm to learn the selection policy based on the observed performance of past server selections. Specifically, a Long Short-Term Memory (LSTM) based neural network is exploited to encode the historical information which helps infer future knowledge of the dynamically changing factors. Then the DRL model selects the optimal server automatically based on the extracted system states. Extensive trace-driven evaluations demonstrate that the proposed DRL-based algorithm has the lowest overall cost compared to existing solutions. [ABSTRACT FROM AUTHOR]

Owing to the insufficient processing ability of wireless devices (WDs), it is difficult for WDs to process these data within the deadline associated with the quality of service requirements. Offloading computation tasks (workloads) to emerging mobile edge computing servers with small or macro base stations is an effective and feasible solution. However, the offloaded data will be fully exposed and vulnerable to security threats. In this paper, we introduce a wireless communication and computation model of partial computation offloading and resource allocation considering the time-varying channel state, the bandwidth constraint, the stochastic arrival of workloads, and privacy preservation. To simultaneously optimize the computation and execution delays, the power consumption, and the bandwidth resources, we model the optimization problem as a Markov decision process (MDP) to minimize the weighted sum cost of the system. Owing to the difficult problems of lack of priori knowledge and the curse of dimensionality, we propose a decentralized optimization scheme on partial computation offloading and resource allocation based on deep reinforcement learning (DOCRRL). According to the time-varying channel state, the arrival rate of computation workloads, and the signal-to-interference-plus-noise ratio, the DOCRRL algorithm can learn the optimal policy for decision-making under stringent latency and risk constraints that prevent the curse of dimensionality from arising owing to the high-dimensional action space and state space. The numerical results reveal that DOCRRL can explore and learn the optimal decision-making policy without priori knowledge; it outperforms four baseline schemes in simulation environments. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) can play an important role in wireless systems as it can be deployed flexibly to help improve coverage and quality of communication. In this paper, we consider a UAV-assisted Mobile Edge Computing (MEC) system, in which a UAV equipped with computing resources can provide offloading services to nearby user equipments (UEs). The UE offloads a portion of the computing tasks to the UAV, while the remaining tasks are locally executed at this UE. Subject to constraints on discrete variables and energy consumption, we aim to minimize the maximum processing delay by jointly optimizing user scheduling, task offloading ratio, UAV flight angle and flight speed. Considering the non-convexity of this problem, the high-dimensional state space and the continuous action space, we propose a computation offloading algorithm based on Deep Deterministic Policy Gradient (DDPG) in Reinforcement Learning (RL). With this algorithm, we can obtain the optimal computation offloading policy in an uncontrollable dynamic environment. Extensive experiments have been conducted, and the results show that the proposed DDPG-based algorithm can quickly converge to the optimum. Meanwhile, our algorithm can achieve a significant improvement in processing delay as compared with baseline algorithms, e.g., Deep Q Network (DQN). [ABSTRACT FROM AUTHOR]

With the development of the wireless network, increasing mobile applications are emerging and receiving great popularity. These applications cover a wide area, such as traffic monitoring, smart homes, real-time vision processing, objective tracking, and so on, and typically require computation-intensive resources to achieve a high quality of experience. Although the performance of mobile devices (MDs) has been continuously enhanced, running all the applications on a single MD still causes high energy consumption and latency. Fortunately, mobile edge computing (MEC) allows MDs to offload their computation-intensive tasks to proximal eNodeBs (eNBs) to augment computational capabilities. However, the current task offloading schemes mainly concentrate on average-based performance metrics, failing to meet the deadline constraint of the tasks. Based on the deep reinforcement learning (DRL) approach, this paper proposes an Energy-aware Task Offloading with Deadline constraint (DRL-E2D) algorithm for a multi-eNB MEC environment, which is to maximize the reward under the deadline constraint of the tasks. In terms of the actor-critic framework, we integrate the action representation into DRL-E2D to handle the large discrete action space problem, i.e., using the low-complexity k-nearest neighbor as an approximate approach to extract optimal discrete actions from the continuous action space. The extensive experimental results show that DRL-E2D achieves better performance than the comparison algorithms on all parameter settings, indicating that DRL-E2D is robust to the state changes in the MEC environment. [ABSTRACT FROM AUTHOR]

Today, Mobile Cloud Computing (MCC) alone can no longer respond to the increasing volume of data and satisfy the necessary delays in real-time applications. In addition, challenges such as security, energy consumption, storage space, bandwidth, lack of mobility support, and lack of location awareness have made this problem more challenging. Expanding applications such as online gaming, Augmented Reality (AR), Virtual Reality (VR), metaverse, e-health, and the Internet of Things (IoT) have brought up new paradigms for processing big data. Some of the paradigms that have emerged in the last decade are trying to alleviate cloud computing problems jointly. Mobile Edge Computing (MEC) and Fog Computing (FC) are the most critical techniques that serve the IoT. One of the common points of the above paradigms is the offloading of IoT tasks. This paper reviews machine learning-based computation offloading mechanisms in the edge and fog environment. This review covers three significant areas of machine learning: supervised learning, unsupervised learning, and reinforcement learning. We discuss various performance metrics, tools, and case studies and analyze their advantages and disadvantages. We systematically elaborate on open issues and research challenges that are crucial for the next decade. [ABSTRACT FROM AUTHOR]

The method of charging mobile devices with wireless power transfer (WPT) from the base station (BS) integrated with mobile edge computing (MEC) increases the potential of MEC. The increasing demand for intelligent computation offloading requires effective decisions among local or remote computation specifically in wireless fading channels of the dynamic environment. Our main aim is to generate an effective offloading decision between local and remote computation in a real-time environment for each wireless channel while preserving optimal computation rate. In this article, we consider a wireless powered MEC system that governs a binary offloading decision to execute the task locally at the edge devices or the remote server. We propose a reinforcement learning based intelligent online offloading (RLIO) framework that adopts an optimal offloading action based on reinforcement methods. This framework acquires a worthy decision among local or remote computation for the time varying wireless channel conditions in dense networks. Numerical results show that the proposed framework can achieve optimal performance while preserving the computation time compared with existing optimization methods. Second, the average execution cost of RLIO is less than 0.4 ms per channel, which enables real-time and optimal offloading in dynamic and large-scale networks. [ABSTRACT FROM AUTHOR]

Supporting latency-sensitive and computation-intensive applications is hardly possible for mobile devices (MDs) with limited battery capacity and low computing resources. Therefore, mobile edge computing (MEC) and wireless power transfer (WPT) have emerged as promising technologies that allow MDs to offload part or all of their workloads to MEC servers and harvest energy for prolonging their battery lifetime. However, the MEC server's limited computing resources, available communication channel quality, and time-limited energy harvesting (EH) challenge the computation offloading. In this paper, we study the joint problem of decentralized computation offloading and resource allocation (JDCORA) in the environment of non-orthogonal multiple access (NOMA)-assisted MEC with multiple EH-enabled MDs. To learn decentralized offloading policies, we propose a multi-agent deep reinforcement learning (MADRL)-based scheme towards minimizing energy consumption and task completion time, which considers the cooperation between MDs to adjust their strategies. In particular, we improve the multi-agent deep deterministic policy gradient (MADDPG) by applying the features of double actors, double centralized critics, soft value estimation, critic regularization and proportional-based prioritized experience replay (pPER) and propose an algorithm called multi-agent twin actors regularized critics (MATARC). Simulation results demonstrate that the proposed MATARC has a better convergence performance compared to other baseline methods and also reduces the average energy consumption, task completion time and task drop rate. • Formulating the problem of decentralized computation offloading in NOMA-assisted MEC. • Aiming at reducing delay and energy of MDs under deadline and battery level constraints. • Proposing an offloading scheme based on multi-agent DRL and CTDE framework. • Developing an algorithm by applying double actors, soft value estimation, critic regularization and PER. • Extensive simulation to evaluate proposed algorithm using time, energy, task drop rate metrics. [ABSTRACT FROM AUTHOR]

In order to meet the demand for efficient computing services in big data scenarios, a cloud edge collaborative computing allocation strategy based on deep reinforcement learning by combining the powerful computing capabilities of cloud is proposed. First, based on the comprehensive consideration of computing resources, bandwidth, and migration decisions, an optimization problem is constructed that minimizes the sum of all user task execution delays and energy consumption weights. Second, a dynamic offloading scheduling algorithm based on Q -learning is proposed based on the optimization problem. This algorithm makes full use of the computing power for cloud and edge, which effectively meets the demand for efficient computing services in Internet of Things' scenarios. Finally, facing the environment dynamic changes of edge nodes in edge cloud, the algorithm can adaptively adjust the migration strategy. Experiments show that when the number of Internet of Things' devices is 30, the total energy consumption of Internet of Things' devices of proposed algorithm is reduced by 24.67% and 19.44%, respectively, compared with other algorithms. The experimental results show that proposed algorithm can effectively improve the success rate of task offloading and execution, which can reduce the local energy consumption. [ABSTRACT FROM AUTHOR]

Aiming at the problem of task offloading with dependency in mobile edge computing, this paper proposed a deep reinforcement learning based offloading scheduling method to minimize the execution time of mobile applications. This method described the process of task scheduling as a Markov decision process. It adopted a sequence to sequence deep neural network to represent the scheduling policy, and then trained the deep neural network with the proximal policy optimization method. Simulation results show that the proposed method has good convergence ability and outperforms six baseline algorithms in different environments, demonstrating the effectiveness and reliability of the proposed method. [ABSTRACT FROM AUTHOR]

With the continuous development of mobile edge computing, people are more willing to offload tasks to edge servers that are closer to users than cloud services for a better user experience. Due to problems such as user mobility and limited coverage of edge servers, how to ensure service quality and prevent service interruption is a problem we have to consider. This article explores the problem of service migration to decide when, where, and how to migrate an ongoing service from the current edge server to the target edge server. At the same time, we will consider the premise that edge servers can only obtain partial user information, and model the service migration problem as a partially observable Markov decision process. To minimize user delay and system energy consumption, we propose a service migration decision algorithm based on Deep recurrent Q-learning (DRQNSM). Multiple experiments show that our algorithm has a better performance than some classical reinforcement learning algorithms. [ABSTRACT FROM AUTHOR]

A mobile edge computing (MEC)-enabled blockchain system is proposed in this study for secure data storage and sharing in internet of things (IoT) networks, with the MEC acting as an overlay system to provide dynamic computation offloading services. Considering latency-critical, resource-limited, and dynamic IoT scenarios, an adaptive system resource allocation and computation offloading scheme is designed to optimize the scalability performance for MEC-enabled blockchain systems, wherein the scalability is quantified as MEC computational efficiency and blockchain system throughput. Specifically, we jointly optimize the computation offloading policy and block generation strategy to maximize the scalability of MEC-enabled blockchain systems and meanwhile guarantee data security and system efficiency. In contrast to existing works that ignore frequent user movement and dynamic task requirements in IoT networks, the joint performance optimization scheme is formulated as a Markov decision process (MDP). Furthermore, we design a deep deterministic policy gradient (DDPG)-based algorithm to solve the MDP problem and define the multiple and variable number of consecutive time slots as a decision epoch to conduct model training. Specifically, DDPG can solve an MDP problem with a continuous action space and it only requires a straightforward actor–critic architecture, making it suitable for tackling the dynamics and complexity of the MEC-enabled blockchain system. As demonstrated by simulations, the proposed scheme can achieve performance improvements over the deep Q network (DQN)-based scheme and some other greedy schemes in terms of long-term transactional throughput. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid development of Internet of Things (IoTs) and artificial intelligence, vehicular networks have transformed from simple interactive systems to smart integrated networks. The accompanying intelligent connected vehicles (ICVs) can communicate with each other and connect to the urban traffic information network, to support intelligent applications, i.e., autonomous driving, intelligent navigation, and in-vehicle entertainment services. These applications are usually delay-sensitive and compute-intensive, with the result that the computation resources of vehicles cannot meet the quality requirements of service for vehicles. To solve this problem, vehicular edge computing networks (VECNs) that utilize mobile edge computing offloading technology are seen as a promising paradigm. However, existing task offloading schemes lack consideration of the highly dynamic feature of vehicular networks, which makes them unable to give time-varying offloading decisions for dynamic changes in vehicular networks. Meanwhile, the current mobility model cannot truly reflect the actual road traffic situation. Toward this end, we study the task offloading problem in VECNs with the synchronized random walk model. Then, we propose a reinforcement learning-based scheme as our solution, and verify its superior performance in processing delay reduction and dynamic scene adaptability. [ABSTRACT FROM AUTHOR]

In wireless Internet of Things (IoT) systems, the multi-input multi-output (MIMO) and cognitive radio (CR) techniques are usually involved into the mobile edge computing (MEC) structure to improve the spectrum efficiency and transmission reliability. However, such a CR based MIMO IoT system will suffer from a variety of smart attacks from wireless environments, even the MEC servers in IoT systems are not secure enough and vulnerable to these attacks. In this paper, we investigate a secure communication problem in a cognitive MIMO IoT system comprising of a primary user (PU), a secondary user (SU), a smart attacker and several MEC servers. The target of our system design is to optimize utility of the SU, including its efficiency and security. The SU will choose an idle MEC server that is not occupied by the PU in the CR scenario, and allocates a proper offloading rate of its computation tasks to the server, by unloading such tasks with proper transmit power. In such a CR IoT system, the attacker will select one type of smart attacks. Then two deep reinforcement learning based resource allocation strategies are proposed to find an optimal policy of maximal utility without channel state information(CSI), one of which is the Dyna architecture and Prioritized sweeping based Edge Server Selection (DPESS) strategy, and the other is the Deep Q-network based Edge Server Selection (DESS) strategy. Specifically, the convergence speed of the DESS scheme is significantly improved due to the trained convolutional neural network (CNN) by utilizing the experience replay technique and stochastic gradient descent (SGD). In addition, the Nash equilibrium and existence conditions of the proposed two schemes are theoretically deduced for the modeled MEC game against smart attacks. Compared with the traditional Q-learning algorithm, the average utility and secrecy capacity of the SU can be improved by the proposed DPESS and DESS schemes. Numerical simulations are also presented to verify the better performance of our proposals in terms of efficiency and security, including the higher convergence speed of the DESS strategy. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the scheduling design of a mobile edge computing (MEC) system, where active mobile devices with computation tasks randomly appear in a cell. Every task can be computed at either the mobile device or the MEC server. We jointly optimize the task offloading decision, uplink transmission device selection and power allocation by formulating the problem as an infinite-horizon Markov decision process (MDP). Compared with most of the existing literature, this is the first attempt to address the transmission and computation optimization with random device arrivals in an infinite time horizon to our best knowledge. Due to the uncertainty in the device number and location, the conventional approximate MDP approaches addressing the curse of dimensionality cannot be applied. An alternative and suitable low-complexity solution framework is proposed in this work. We first introduce a baseline scheduling policy, whose value function can be derived analytically with the statistics of random mobile device arrivals. Then, one-step policy iteration is adopted to obtain a sub-optimal scheduling policy whose performance can be bounded analytically. The complexity of deriving the sub-optimal policy is reduced dramatically compared with conventional solutions of MDP by eliminating the complicated value iteration. To address a more general scenario where the statistics of random mobile device arrivals are unknown, a novel and efficient algorithm integrating reinforcement learning and stochastic gradient descent (SGD) is proposed to improve the system performance in an online manner. Simulation results show that the gain of the sub-optimal policy over various benchmarks is significant. [ABSTRACT FROM AUTHOR]

Recent years have witnessed the booming development of mobile devices (MDs), along with the surging popularity of mobile applications. Despite the unceasing progress of MDs, the great tension between the limited capability of MDs and the intensive demands of mobile applications remains intractable. Thanks to the proposal of mobile edge computing, the tension was partly alleviated by offloading computation-intensive and delay-critical tasks to edge servers, which are deployed with powerful computing and energy resources in close proximity to MDs. Researchers have paid numerous attention on designing efficient computation-offloading policies, latest of which are based on deep reinforcement learning (DRL). However, most existing DRL-based approaches suffer from poor generalizations in novel environments. To address the issue, this paper investigates the generalization of computation-offloading policy learning and proposes a novel scheduling approach by exploiting learning-experience utility. First, we define a group of experience utilities according to their contributions to the learning process. Then, novel maintaining and sampling strategies are proposed based on the experience utility to effectively improve the generalization performances of learned computation-offloading policies. Finally, extensive experiments are conducted and the substantial performance improvements of the proposed approach over the state-of-the-art approaches are verified by the experimental results. [ABSTRACT FROM AUTHOR]

Offloading computation-intensive tasks (e.g., blockchain consensus processes and data processing tasks) to the edge/cloud is a promising solution for blockchain-empowered mobile edge computing. However, the traditional offloading approaches (e.g., auction-based and game-theory approaches) fail to adjust the policy according to the changing environment and cannot achieve long-term performance. Moreover, the existing deep reinforcement learning-based offloading approaches suffer from the slow convergence caused by high-dimensional action space. In this paper, we propose a new model-free deep reinforcement learning-based online computation offloading approach for blockchain-empowered mobile edge computing in which both mining tasks and data processing tasks are considered. First, we formulate the online offloading problem as a Markov decision process by considering both the blockchain mining tasks and data processing tasks. Then, to maximize long-term offloading performance, we leverage deep reinforcement learning to accommodate highly dynamic environments and address the computational complexity. Furthermore, we introduce an adaptive genetic algorithm into the exploration of deep reinforcement learning to effectively avoid useless exploration and speed up the convergence without reducing performance. Finally, our experimental results demonstrate that our algorithm can converge quickly and outperform three benchmark policies. [ABSTRACT FROM AUTHOR]

Mobile edge computing for autonomous driving needs to manage heterogeneous resources and process large amounts of data or multi-purpose payload. There needs to be deploying, scheduling and migrating tasks on edge nodes to ensure the reliability of tasks or maximize the utilization of resources. However, applying autonomous learning methods on autonomous driving is exceptionally difficult, due to the complexity of multi-dimensional context and the sensitivity to hyperparameters. In this paper, we propose a learning approach to quality-of-service (QoS) prediction of services via multi-dimensional context, and develop a stable approach for service deployment that requires minimal hyperparameter tuning and a modest number of trials to learn multilayer neural network policies. This approach can automatically trades off exploration against exploitation by automatically tuning hyperparameter based on maximum entropy reinforcement learning. We then demonstrate that this approach achieves state-of-the-art performance on Autoware benchmark environments. • This paper is an extension of our icws2017 paper: Xiong, W. , Wu, Z. , Li, B. , & Gu, Q., A Learning Approach to QoS Prediction via Multi-Dimensional Context, 2017 IEEE International Conference on Web Services. We propose a learning approach to quality-of-service (QoS) prediction of services via multi-dimensional context. • We propose a self-adaptive approach for service deployment under Mobile Edge Computing. • We conduct comprehensive experiments on a real-world system, demonstrating the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Abstract Cloud of Things (CoT) is a significant paradigm for bridging cloud resource and mobile terminals. Mobile edge computing (MEC) is a supporting architecture for CoT. The objectives of this paper are to describe and evaluate a method to handle the computation offloading problem during user mobility which minimizes the offloading failure rate in heterogeneous network. Furthermore, users' mobility and their choices for offloading lead to the everchanging condition of wireless network and opportunistic resource available. By modeling such dynamic mobile edge environment, quantizing the user cost, failure penalty and diversified QoS requirements, computation offloading problem is converted into an online decision-making problem in a stochastic process. We divide the decision-making into two phases: offloading planning phase and offloading running phase. In both phases the learning agent can continuously improve the control policy. We also conduct a failure recovery policy to tackle different types of failure and is included in the decision-making process. The numerical results show that the proposed online learning offloading method for mobile users can derive the optimal offloading scheme compared with the baseline algorithms. [ABSTRACT FROM AUTHOR]

• The combination of mobile devices and edge computing reduces the total computing overhead and makes resource allocation reasonable. • Evaluate system performance by the weighted sum of delay and energy consumption. • Change the parameter update method to accelerate the convergence speed of the deep reinforcement learning algorithm. • A neural network is used to approximate the behavior value function, and a target q network is used to update the target. With the arrival of the 5th generation mobile networks (5 G) era, the data needed by mobile devices (MDs) is explosively growing. High-consumption, low-latency applications are huge challenges for resource-constrained Internet of things (IoT) devices. Mobile edge computing overcomes the limitations of computing resources on MDs by offloading tasks generated by MDs and assigning them to nearby MEC servers. Therefore, mobile edge computing (MEC) becomes important. This paper presents a task offloading strategy for the multi-device multi-server system. To meet the task requirements of different MDs, we formulate an overhead minimization problem to optimize the delay and energy consumption of the system. We propose the Double Deep Q Network (Double-DQN) algorithm to perform location selection strategies for tasks generated on the mobile devices and allocate respective computing resources. Simulation results show that the algorithm can allocate resources reasonably and reduce the overhead of the entire system. [Display omitted] [ABSTRACT FROM AUTHOR]

With the rapid development of vehicular networks, vehicle-to-everything (V2X) communications have huge number of tasks to be calculated, which brings challenges to the scarce network resources. Cloud servers can alleviate the terrible situation regarding the lack of computing abilities of vehicular user equipment (VUE), but the limited resources, the dynamic environment of vehicles, and the long distances between the cloud servers and VUE induce some potential issues, such as extra communication delay and energy consumption. Fortunately, mobile edge computing (MEC), a promising computing paradigm, can ameliorate the above problems by enhancing the computing abilities of VUE through allocating the computational resources to VUE. In this paper, we propose a joint optimization algorithm based on a deep reinforcement learning algorithm named the double deep Q network (double DQN) to minimize the cost constituted of energy consumption, the latency of computation, and communication with the proper policy. The proposed algorithm is more suitable for dynamic scenarios and requires low-latency vehicular scenarios in the real world. Compared with other reinforcement learning algorithms, the algorithm we proposed algorithm improve the performance in terms of convergence, defined cost, and speed by around 30%, 15%, and 17%. [ABSTRACT FROM AUTHOR]

In the Industrial Internet of Things (IIoT), there exist numerous sensor devices with weak computing power and small energy storage. To meet the real-time and big data computing requirements of industrial production, EIIoT (Edge computing-based IIoT) that combines mobile edge computing with IIoT has emerged. It is necessary to offload computing tasks to nearby edge servers for data storage and processing in EIIoT, thus inevitably causing the edge servers to overload. To this end, we propose a jointly constrained optimization model of delay and energy consumption based on queuing theory; this model can effectively solve the task offloading problem in EIIoT. Subsequently, to satisfy the unique offloading requirements of EIIoT, we improve the MAPPO (multi agent proximal policy optimization) algorithm structure to form a lightweight optimal task offloading algorithm called Multi-Agent Deep Reinforcement Learning based on Queuing theory (MAQDRL), which is more suitable for EIIoT. In the algorithm, we systematically integrate queuing theory and use Multi-Agent Deep Reinforcement Learning (MADRL) to obtain the optimal offloading strategy in dynamic and random multiuser offloading environments. We also improve the structure of neural networks of MADRL by analyzing the structural characteristics of the input data. As a result, the algorithm that we proposed exhibits good convergence and exceptional performance in terms of the task arrival rate, bandwidth, energy consumption, latency and other indicators. The simulation results indicate that compared with other classical algorithms, MAQDRL is effective for solving the EIIoT offloading problem. • Construct an EIIoT characteristics-based intelligent lightweight task offloading framework. • Design an M/M/N model fully considering calculation delays based on queuing theory. • Develop a lightweight MADRL algorithm to obtain the optimal task offloading strategy for EIIoT. [ABSTRACT FROM AUTHOR]

Ultra reliability and low latency communications (URLLC) are considered as one of the most important use cases for computation tasks in the Internet of Vehicles (IoV) edge computing networks. To meet the task requirements in IoV networks, communication and computing resources are allocated in an effective and efficient way. Thus, this paper proposes a multi-objective reinforcement learning strategy, called intelligent communication and computation resource allocation (ICCRA), which combines communication and computing resource allocation to reduce the total system cost consisting of latency and reliability. Specifically, this strategy can be decomposed into three algorithms. The algorithm called joint computation offloading and collaboration is a general framework of the strategy, it first uses the K-nearest neighbor method to select offloading layers for computation tasks, such as cloud computing layer, mobile edge computing layer, and local computing layer. Then, when selecting the local computing layer to perform the task, the algorithm called collaborative vehicle selection is used to find the target vehicle to execute collaborative computing. The allocation of communication and computing resources is defined as two independent objectives, the algorithm called multi-objective resource allocation uses the reinforcement learning to achieve the optimal solution, at the mobile edge computing layer. Simulation results show that compared with local computing, edge computing, and random computing strategies, the proposed strategy reduces the total system cost effectively. For instance, compared with edge computing strategy, the total system cost of the proposed strategy can be reduced about 50% on average with the different number of vehicle tasks. [ABSTRACT FROM AUTHOR]

With the ongoing advancement of electric power Internet of Things (IoT), traditional power inspection methods face challenges such as low efficiency and high risk. Unmanned aerial vehicles (UAVs) have emerged as a more efficient solution for inspecting power facilities due to their high maneuverability, excellent line-of-sight communication capabilities, and strong adaptability. However, UAVs typically grapple with limited computational power and energy resources, which constrain their effectiveness in handling computationally intensive and latency-sensitive inspection tasks. In response to this issue, we propose a UAV task offloading strategy based on deep reinforcement learning (DRL), which is designed for power inspection scenarios consisting of mobile edge computing (MEC) servers and multiple UAVs. Firstly, we propose an innovative UAV-Edge server collaborative computing architecture to fully exploit the mobility of UAVs and the high-performance computing capabilities of MEC servers. Secondly, we established a computational model concerning energy consumption and task processing latency in the UAV power inspection system, enhancing our understanding of the trade-offs involved in UAV offloading strategies. Finally, we formalize the task offloading problem as a multi-objective optimization issue and simultaneously model it as a Markov Decision Process (MDP). Subsequently, we proposed a task offloading algorithm based on a Deep Deterministic Policy Gradient (OTDDPG) to obtain the optimal task offloading strategy for UAVs. The simulation results demonstrated that this approach outperforms baseline methods with significant improvements in task processing latency and energy consumption. [ABSTRACT FROM AUTHOR]

At present, with the intelligence that has been achieved in computer and communication technologies, vehicles can provide many convenient functions to users. However, it is difficult for a vehicle to deal with computationally intensive and latency-sensitive tasks occurring in the vehicle environment by itself. To this end, mobile edge computing (MEC) services have emerged. However, MEC servers (MECSs), which are fixed on the ground, cannot flexibly respond to temporal dynamics where tasks are temporarily increasing, such as commuting time. Therefore, research has examined the provision of edge services using additional unmanned aerial vehicles (UAV) with mobility. Since these UAVs have limited energy and computing power, it is more important to optimize energy efficiency through load balancing than it is for ground MEC servers (MECSs). Moreover, if only certain servers run out of energy, the service coverage of a MEC server (MECS) may be limited. Therefore, all UAV MEC servers (UAV MECSs) need to use energy evenly. Further, in a high-mobility vehicle environment, it is necessary to have effective task migration because the UAV MECS that provides services to the vehicle changes rapidly. Therefore, in this paper, a federated deep Q-network (DQN)-based task migration strategy that considers the load deviation and energy deviation among UAV MECSs is proposed. DQN is used to create a local model for migration optimization for each of the UAV MECSs, and federated learning creates a more effective global model based on the fact that it has common spatial features between adjacent regions. To evaluate the performance of the proposed strategy, the performance is analyzed in terms of delay constraint satisfaction, load deviation, and energy deviation. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. In this paper, an MEC enabled multi-user multi-input multi-output (MIMO) system with stochastic wireless channels and task arrivals is considered. In order to minimize long-term average computation cost in terms of power consumption and buffering delay at each user, a deep reinforcement learning (DRL)-based dynamic computation offloading strategy is investigated to build a scalable system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn decentralized computation offloading policies at all users respectively, where local execution and task offloading powers will be adaptively allocated according to each user's local observation. Numerical results demonstrate that the proposed DDPG-based strategy can help each user learn an efficient dynamic offloading policy and also verify the superiority of its continuous power allocation capability to policies learned by conventional discrete action space-based reinforcement learning approaches like deep Q-network (DQN) as well as some other greedy strategies with reduced computation cost. Besides, power-delay tradeoff for computation offloading is also analyzed for both the DDPG-based and DQN-based strategies. [ABSTRACT FROM AUTHOR]

Mobile edge computing is a method to solve the computing-intensive task requirements of robots. Traditional algorithms are based on intelligent algorithms or convex optimization methods, and the iteration time is long. Deep reinforcement learning can be solved in a single forward pass, but only for a fixed number of robots. Through the analysis and research of deep reinforcement learning, this paper performed input regularization before the input layer in the deep reinforcement learning neural network, and added a convolution layer after the output layer so that the network could adaptively meet the unloading requirements of the number of dynamic mobile robots. Finally, it carried out the simulation experiments to verify the effectiveness and feasibility of proposed algorithm, compared with the adaptive gen etic algorithm and reinforcement learning . [ABSTRACT FROM AUTHOR]

In the cooperative vehicle infrastructure system, the road side unit (RSU) equipped with a mobile edge computing (MEC) server and sensors could provide vehicle infrastructure cooperation services for vehicles, such as optimization and cooperative driving, enhanced visibility, and so on. In view of this, the MEC server needs to fuse the sensing information from sensors on the vehicles and RSU, respectively. In the case of bad channel conditions, uploading the raw sensing information from the vehicles results in high uplink transmission latency. To deal with it, the vehicles can process the information locally and just deliver the results to the RSU. However, due to the limited computing resources on the vehicles, the processing accuracy of the raw information on the vehicles is lower than that on the MEC server. Besides, processing locally leads to higher vehicle energy consumption. Thus, in this paper, we aim to jointly optimize execution latency, processing accuracy, and energy consumption of the MEC-based cooperative vehicle infrastructure system. Firstly, we design the terminal machine learning task model and the edge machine learning task model on the vehicle side and RSU side, respectively. Then, we formulate a long-term multi-objective optimization problem. Owing to the stochastic traffic and time-varying communication conditions, we reformulate it as a Markov decision process and propose a two-stage deep reinforcement learning-based offloading and resource allocation (TDORA) strategy to determine the task offloading and the transmit power of each vehicle. Simulation results demonstrate the efficacy of the proposed strategy. [ABSTRACT FROM AUTHOR]

Service migration in mobile edge computing is a promising approach to improving the quality of service (QoS) for mobile users and reducing the network operational cost for service providers as well. However, these benefits are not free, coming at costs of bulk-data transfer, and likely service disruption, which could consequently increase the overall service costs. To gain the benefits of service migration while minimizing its cost across the edge nodes, in this paper, we leverage reinforcement learning (RL) method to design a cost-effective framework, called Mig-RL , for the service migration with a reduction of total service costs as a goal in a mobile edge environment. The Mig-RL leverages the infrastructure of edge network and deploys a migration agent through Q-learning to learn the optimal policy with respect to the service migration status. We distinguish the Mig-RL from other existing works in several major aspects. First, we fully exploit the nature of this problem in a modest migration space, which allows us to constrain the number of service replicas whereby a defined state–action space could be effectively handled, as opposed to those methods that need to always approximate a huge state–action space for policy optimality. Second, we advocate a migration policy-base as a cache to save the learning process by retrieving the most effective policy whenever a similar migration pattern is encountered as time goes on. Finally, by exploiting the idea of software defined network, we also investigate the efficient implementation of Mig-RL in mobile edge network. Experimental results based on some real and synthesized access sequences show that Mig-RL , compared with the selected existing algorithms, can substantially minimize the service costs, and in the meantime, efficiently improve the QoS by adapting to the changes of mobile access patterns. • A Q-learning-based framework is proposed for service migration in mobile edges. • A policy-cache-based method is introduced to optimize the learning process of the framework. • A software-defined approach is presented to effectively implement the framework. • A prototype implementation is evaluated with the trace data gathered from Shenzhen taxi system and metro stations. [ABSTRACT FROM AUTHOR]