Search

To address the current challenge of smart devices in healthcare Internet of things (IoT) struggling to efficiently process intensive applications in real-time, a collaborative cloud-edge offloading model tailored for ultra-dense edge computing (UDEC) networks is developed. While numerous studies have delved into the optimization of offloading in mobile edge computing (MEC), it is imperative to consider non-orthogonal multiple access (NOMA) as a physical technology when addressing the offloading optimization process in MEC. The multiuser sharing of spectrum resources in NOMA can enhance the network spectrum utilization and reduce the computational delay when users transmit computing tasks. Consequently, a model for NOMA-assisted UDEC systems is proposed. The model takes into account joint offloading decisions, computational resources, and sub-channel resources and is modeled as a complex nonlinear mixed-integer programming problem. The aim is to decrease the task execution delay and energy consumption of smart devices while ensuring that users' maximum acceptable delay for processing medical computational tasks is met efficiently and in a timely manner. Deep deterministic policy gradient (DDPG), a deep reinforcement learning method, is employed to solve the joint optimization problem. The final simulation results show that the algorithm converges well. The proposed offloading scheme can reduce the system cost by 54.5 and 69.9% in comparison with scenarios where users solely perform local computations and offload their tasks to the base station (BS). The application of NOMA communication in our offloading scheme boosts network spectrum utilization and trims down the system cost by 87.09% when contrasted with orthogonal multiple access (OMA). [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) can enhance the computation capabilities of smart mobile devices for computation-intensive mobile applications via supporting computation offloading efficiently. However, the limitation of wireless resources and computational resources of edge servers often becomes the bottlenecks to realizing the developments of MEC. In order to address the computation offloading problem in the time-varying wireless networks, the offloading decisions and the allocation of radio and computation resources need to be jointly managed. Traditional optimization methods are challenging to deal with the combinatorial optimization problem in complex real-time dynamic network environments. Therefore, we propose a deep reinforcement learning (DRL)-based optimization approach, named DRJOA, which jointly optimizes offloading decisions, computation, and wireless resources allocation. The optimization algorithm based on DRL has the advantages of fast solving speed and strong generalization ability, which makes it possible to solve combinatorial optimization problems online. Simulation results show that our proposed DRJOA in this study dramatically outperforms the benchmark methods for offloading decisions and system utility. [ABSTRACT FROM AUTHOR]

The continual development of mobile edge computing efficiently solves the problem that mobile devices are unable to handle computation-intensive tasks due to their computation capacity and battery restrictions. In this paper, we consider mobile awareness and dynamic battery charging in a multi-user and multi-server mobile edge computing system, where various tasks are generated successively on the user devices. Servers act as learning agents and collaborate with user devices to develop task partitioning and computation resource allocation strategies. With the purpose of decreasing task failure rate and improving system utility in the long term, which is closely related to latency, energy consumption, and server cost, optimal strategies are demanded by the system. We model the joint optimization problem as a multi-agent Markov decision process game. And a deep reinforcement learning method based on the multi-agent deep deterministic policy gradient algorithm is proposed, which employs neural networks and works in a centralized training and decentralized execution manner to optimize the strategies. Finally, simulation results demonstrate the effectiveness of our proposed algorithm in terms of reducing task failure rate and improving system utility. [ABSTRACT FROM AUTHOR]

Mobile edge computing requires more and more high-performance servers, resulting in increased energy consumption. As an effective means to reduce energy consumption, virtual machine placement (VMP) has been widely studied. In the edge computing environment, as the number of terminal device requests continues to increase, the scale of VMP becomes larger and larger, and existing research algorithms may take a long time to converge. The reason is that as the number of VMs increases, the search space of the policy becomes larger and the agent needs to interact with the environment for a longer time to make the best decision. In addition, existing research methods only consider reducing energy consumption, rarely consider the response latency of virtual machines, and almost ignore the dynamic changes of the edge environment. To overcome these drawbacks, we propose a virtual machine placement algorithm based on meta-reinforcement learning, which consists of an inner and outer loop. The inner loop designs a deep reinforcement learning algorithm combined with the order exchange and migration mechanism to generate the best decision, and the outer loop provides meta-strategy parameters for the inner loop based on meta-learning to accelerate the convergence capability of the inner loop, thereby obtaining efficient virtual machine placement decisions quickly from a new environment. Through simulation experiments, we demonstrate that our approach effectively reduces the energy consumption of the edge server and the response latency of VMs at different problem sizes compared to the three baseline algorithms. At the same time, it quickly adapts to the new environment with only a small number of gradient updates. [ABSTRACT FROM AUTHOR]

With the rapid development of innovative applications, lots of computation-intensive and delay-sensitive tasks are emerging. Task offloading, which is regarded as a key technology in the emerging mobile edge computing paradigm, aims at offloading the tasks from mobile devices (MDs) to edge servers or the remote cloud to reduce system delay and energy consumption of MDs. However, most existing task offloading studies either didn't consider the dependencies among tasks or simply designed heuristic schemes to solve dependent task offloading problems. Different from these studies, we propose a deep reinforcement learning (DRL) based task offloading scheme to jointly offload tasks with dependencies. Specifically, we model the dependencies among tasks by directed acyclic graphs and formulate the task offloading problem as minimizing the average cost of energy and time (CET) of users. To solve this NP-hard problem, we propose a deep Q-network learning-based framework that creatively utilizes deep neural networks to extract system features. Simulation results show that our proposed scheme outperforms the existing DRL scheme and heuristic scheme in reducing the average CET of all users and can obtain near-optimal solutions. [ABSTRACT FROM AUTHOR]

In many IoT scenarios, the resources of terminal devices are limited, and it is difficult to provide services with low latency and low energy consumption. Mobile edge computing is an effective solution by offloading computing tasks to edge server processing. There are some problems in the existing offloading decision algorithms: the offloading decision method based on heuristic algorithms cannot dynamically adjust the policy in the changing environment; the offloading algorithm based on deep reinforcement learning will lead to slow convergence and poor exploration effect due to the problem of dimension explosion. To solve the above problems, this paper designs an offloading decision algorithm to make dynamic decisions in a mobile edge computing network with multi-device access. The algorithm comprehensively considers the energy consumption of terminal equipment, offloading overhead, average delay and success rate of task completion, aiming to achieve the highest total revenue of the whole system in a period of time. In this work, the online offloading problem is abstracted as a Markov decision process. Based on the Double Dueling Deep Q-Network (D3QN) algorithm, the offloading decision is designed to adapt to the highly dynamic environment of the edge computing network and solve the problem of high state space complexity. In addition, this paper innovatively introduces a clustering algorithm into deep reinforcement learning (DRL) to preprocess the action space and solve the explosion problem of the action space dimension caused by the increase of terminal devices. The experimental results show that the proposed algorithm is superior to the baseline strategies such as Deep Q-Network (DQN) algorithm in convergence speed and total reward. [ABSTRACT FROM AUTHOR]

Adopting Internet of Vehicles (IoV) technology has led to many new uses to improve traffic control, safety, and entertainment services. Still, the increasing amount of information these applications produce poses significant obstacles regarding response time, power usage, and other related issues. To enhance the functionality of IoV systems, this paper introduces a new way that utilizes Deep Reinforcement Learning (DRL) and Mobile Edge Computing (MEC) to improve its performance. Specifically, our DRL framework employs a convolutional neural network-based Asynchronous Advantage Actor-Critic (A3C) algorithm, chosen for its efficacy in processing spatial data relevant to IoV systems such as vehicle locations and speeds. The optimization problem considers the vehicle's location and speed, the MEC server's resources, and the IoV application's requirements by formulating it as a Markov Decision Process (MDP). Utilizing the A3C approach, our Deep Neural Network (DNN) method infers an optimal offloading policy. We optimized the proposed algorithm with strategies that include adaptive learning rate, gradient clipping, entropy regularization, and generalized advantage estimation. The optimized algorithm considers factors such as distance, bandwidth, and communication requirements to provide efficient task-offloading solutions, leading to better system utility and performance. The proposed strategy outperforms comparable models through comprehensive simulations, providing an average enhancement of 20.50% in energy consumption, 15.86% in latency, and 11.94% in execution time, emphasizing the effectiveness and superiority of the suggested algorithm in dealing with various workloads while reducing energy consumption, latency, and execution times. [ABSTRACT FROM AUTHOR]

In this work, we investigate a reconfigurable intelligent surface (RIS) assisted mobile edge computing (MEC) network, where multiple users offload tasks to edge servers (ES) via RIS for accelerating computation. However, in practical MEC networks, computational tasks often exhibit diverse priorities, resulting in varying degrees of computational utility. This variability poses significant challenges to system optimization. In response to this challenge, we propose a deep reinforcement learning (DRL)-based approach to improve the performance of the RIS-assisted MEC network, where the task priority and resource allocations are jointly considered. Specifically, we exploit functions to evaluate the computational benefit of different tasks. Then, we devise a joint resource allocation scheme, which jointly considers RISs' phase shifts, user-RIS allocation, and task offloading, to maximize the system utility. To solve the problem under the complicated environment of fading channels and various task priorities, we employ the DRL approach to obtain effective resource allocation strategies to improve the system performance. Simulations are finally conducted to validate the effectiveness and superiority of the proposed schemes in this work. [ABSTRACT FROM AUTHOR]

The ubiquitous Internet of Things (IoTs) devices spawn growing mobile services of applications with computationally-intensive and latency-sensitive features, which increases the data traffic sharply. Driven by container technology, microservice is emerged with flexibility and scalability by decomposing one service into several independent lightweight parts. To improve the quality of service (QoS) and alleviate the burden of the core network, caching microservices at the edge of networks empowered by the mobile edge computing (MEC) paradigm is envisioned as a promising approach. However, considering the stochastic retrieval requests of IoT devices and time-varying network topology, it brings challenges for IoT devices to decide the caching node selection and microservice replacement independently without complete information of dynamic environments. In light of this, a MEC-enabled di stributed cooperative m icroservice ca ching scheme, named DIMA, is proposed in this paper. Specifically, the microservice caching problem is modeled as a Markov decision process (MDP) to optimize the fetching delay and hit ratio. Moreover, a distributed double dueling deep Q-network (D3QN) based algorithm is proposed, by integrating double DQN and dueling DQN, to solve the formulated MDP, where each IoT device performs actions independently in a decentralized mode. Finally, extensive experimental results are demonstrated that the DIMA is well-performed and more effective than existing baseline schemes. [ABSTRACT FROM AUTHOR]

In mobile edge computing environment, due to resources constraints of edge devices, when user locations continue changing, the network will be delayed or interrupted, which affects the quality of user's service access. Previous studies have shown that deploying multiple microservice instances with the same function on multiple edge servers through container technology can solve this problem. However, how to choose the optimal microservice instance from multiple servers in a cloud-edge hybrid environment needs to be further investigated. This paper studies the selection of microservices problem based on the dynamic and heterogeneous characters of the cloud-edge collaborative environment, which is defined as a microservice selection and scheduling optimization problem (MSSP) to minimize users' service access delay. To cope with the complexity of cloud-edge collaborative environment and improve learning efficiency, MSSP is regarded as a Markov decision-making process, a Deep Deterministic Policy Gradient algorithm for microservice selection called MS_DDPG is then proposed to solve this problem, and the microservice selection strategy experience pool is established in MS_DDPG. Performance evaluations of MS_DDPG based on a real dataset and some synthetic dataset have been conducted, and the results show that MS_DDPG outperforms the other three baseline algorithms. In terms of average access delay, MS_DDPG is reduced by 23.82%. We also validate the performance of MS_DDPG by increasing the number of user requests, and the results also show that MS_DDPG obtains better performance in scalability. [ABSTRACT FROM AUTHOR]

The integration of Cellular Vehicle-to-Everything (C-V2X) and Mobile Edge Computing (MEC) is critical for satisfying the demanding requirements of vehicular applications, which are characterized by ultra-low latency and ultra-high reliability. In this paper, we address the challenge of jointly optimizing computation offloading and resource allocation in C-V2X network. To achieve this, we propose a hierarchical MEC/C-V2X network that accounts for the dynamic changes of the vehicular network and the diversity of computation offloading patterns. Additionally, we establish a collaborative computation offloading model that supports multiple offloading patterns. We formulate the dynamic computation offloading and resource allocation problem as a sequential decision problem based on the Markovian decision process. To enable automated and intelligent decision-making, we propose a deep reinforcement learning algorithm called ORAD, based on the deep deterministic policy gradient algorithm, to maximize offloading success rate in real-time. The numerical results demonstrate that the proposed algorithm effectively provides the optimal policy, resulting in the offloading success rate of vehicular tasks being improved by 2.73% to 95.51%. [ABSTRACT FROM AUTHOR]

Owing to the insufficient processing ability of wireless devices (WDs), it is difficult for WDs to process these data within the deadline associated with the quality of service requirements. Offloading computation tasks (workloads) to emerging mobile edge computing servers with small or macro base stations is an effective and feasible solution. However, the offloaded data will be fully exposed and vulnerable to security threats. In this paper, we introduce a wireless communication and computation model of partial computation offloading and resource allocation considering the time-varying channel state, the bandwidth constraint, the stochastic arrival of workloads, and privacy preservation. To simultaneously optimize the computation and execution delays, the power consumption, and the bandwidth resources, we model the optimization problem as a Markov decision process (MDP) to minimize the weighted sum cost of the system. Owing to the difficult problems of lack of priori knowledge and the curse of dimensionality, we propose a decentralized optimization scheme on partial computation offloading and resource allocation based on deep reinforcement learning (DOCRRL). According to the time-varying channel state, the arrival rate of computation workloads, and the signal-to-interference-plus-noise ratio, the DOCRRL algorithm can learn the optimal policy for decision-making under stringent latency and risk constraints that prevent the curse of dimensionality from arising owing to the high-dimensional action space and state space. The numerical results reveal that DOCRRL can explore and learn the optimal decision-making policy without priori knowledge; it outperforms four baseline schemes in simulation environments. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) can play an important role in wireless systems as it can be deployed flexibly to help improve coverage and quality of communication. In this paper, we consider a UAV-assisted Mobile Edge Computing (MEC) system, in which a UAV equipped with computing resources can provide offloading services to nearby user equipments (UEs). The UE offloads a portion of the computing tasks to the UAV, while the remaining tasks are locally executed at this UE. Subject to constraints on discrete variables and energy consumption, we aim to minimize the maximum processing delay by jointly optimizing user scheduling, task offloading ratio, UAV flight angle and flight speed. Considering the non-convexity of this problem, the high-dimensional state space and the continuous action space, we propose a computation offloading algorithm based on Deep Deterministic Policy Gradient (DDPG) in Reinforcement Learning (RL). With this algorithm, we can obtain the optimal computation offloading policy in an uncontrollable dynamic environment. Extensive experiments have been conducted, and the results show that the proposed DDPG-based algorithm can quickly converge to the optimum. Meanwhile, our algorithm can achieve a significant improvement in processing delay as compared with baseline algorithms, e.g., Deep Q Network (DQN). [ABSTRACT FROM AUTHOR]

The processing of large volumes of data sets unprecedented demands on the computing power of devices, and it is evident that resource-constrained mobile devices struggle to satisfy the need. As a distributed computing paradigm, edge computing can release mobile devices from computation-intensive tasks, reducing the strain and improving processing efficiency. Traditional offloading methods are less adaptable and do not work in some harsh settings. We simplify the problem to binary offloading decisions in this research and offer a new Asynchronous Update Reinforcement Learning-based Offloading (ARLO) algorithm. The method employs a distributed learning strategy, with five sub-networks and a central public network. Each sub-network has the same structure, as they interact with their environment to learn and update the public network. The sub-network pulls the parameters of the central public network every once in a while. Each sub-network has an experienced pool that minimizes data correlation and is particularly successful in preventing situations where the model falls into a local optimum solution. The main reason for using asynchronous multithreading is that it allows multiple threads to learn the strategy simultaneously, making the learning process faster. At the same time, when the model is trained, five threads can run simultaneously and can handle tasks from different users. The results of simulations show that the algorithm is adaptive and can make optimized offloading decisions on time, even in a time-varying Internet environment, with a significant increase in computational efficiency compared to traditional methods and other reinforcement learning methods. [ABSTRACT FROM AUTHOR]

With the development of the wireless network, increasing mobile applications are emerging and receiving great popularity. These applications cover a wide area, such as traffic monitoring, smart homes, real-time vision processing, objective tracking, and so on, and typically require computation-intensive resources to achieve a high quality of experience. Although the performance of mobile devices (MDs) has been continuously enhanced, running all the applications on a single MD still causes high energy consumption and latency. Fortunately, mobile edge computing (MEC) allows MDs to offload their computation-intensive tasks to proximal eNodeBs (eNBs) to augment computational capabilities. However, the current task offloading schemes mainly concentrate on average-based performance metrics, failing to meet the deadline constraint of the tasks. Based on the deep reinforcement learning (DRL) approach, this paper proposes an Energy-aware Task Offloading with Deadline constraint (DRL-E2D) algorithm for a multi-eNB MEC environment, which is to maximize the reward under the deadline constraint of the tasks. In terms of the actor-critic framework, we integrate the action representation into DRL-E2D to handle the large discrete action space problem, i.e., using the low-complexity k-nearest neighbor as an approximate approach to extract optimal discrete actions from the continuous action space. The extensive experimental results show that DRL-E2D achieves better performance than the comparison algorithms on all parameter settings, indicating that DRL-E2D is robust to the state changes in the MEC environment. [ABSTRACT FROM AUTHOR]

The proliferation of Internet-of-Things (IoTs) devices provides a promising platform for various intelligent applications such as virtual reality. However, because of the limited onboard computation resource of IoT devices, the computation task suffers from long latency. Mobile edge computing (MEC), which is a critical technology, allows offloading the computation tasks to an edge server to alleviate the shortage of computation resources and accelerate the computation tasks of IoT devices. Owing to the flexibility and mobility advantages of unmanned aerial vehicles (UAVs), UAV-assisted MEC has been widely researched. However, existing studies mostly explore a centralized offloading strategy. Therefore, when the number of IoT devices increases, the performance of the centralized strategy is reduced. The present study explores an intelligent strategy to minimize computation latency using a distributed algorithm. We develop a distributed algorithm named DeepMECagent based on deep reinforcement learning to optimize the computation offloading with minimum computation latency. In the considered scenario, a UAV functions as an aerial edge server to collect and process the computation tasks offloaded by ground IoT devices.The simulation results demonstrate the effective-ness of the proposed approach for minimizing the computation latency, where the computation latency of the proposed algorithm improves 39.71%,87.42%, and 88.55%, respectively, while compared with the centralized-DQN, Q-table, and the random algorithm. Given the expiration time as 1 second, the number of completed tasks within the expiration time of the proposed DeepMECagent is around 2 × and 1.25 × compared with the random algorithm and the Q-table algorithm, respectively. [ABSTRACT FROM AUTHOR]

In mobile edge computing (MEC), randomly offloading tasks to edge servers (ES) can cause wireless devices (WD) to compete for limited bandwidth resources, leading to overall performance degradation. Reinforcement learning can provide suitable strategies for task offloading and resource allocation through exploration and trial-and-error, helping to avoid blind offloading. However, traditional reinforcement learning algorithms suffer from slow convergence and a tendency to get stuck in suboptimal local minima, significantly impacting the energy consumption and data timeliness of edge computing task unloading. To address these issues, we propose Parallel Exploration with Asynchronous Training-based Deep Reinforcement Learning (PEATDRL) algorithm for MEC network offloading decisions. Its objective is to maximize system performance while limiting energy consumption in an MEC environment characterized by time-varying wireless channels and random user task arrivals. Firstly, our model employs two independent DNNs for parallel exploration, each generating different offloading strategies. This parallel exploration enhances environmental adaptability, avoids the limitations of a single DNN, and addresses the issue of agents getting stuck in suboptimal local minima due to the explosion of decision combinations, thereby improving decision performance. Secondly, we set different learning rates for the two DNNs during the training phase and trained them at various intervals. This asynchronous training strategy increases the randomness of decision exploration, prevents the two DNNs from converging to the same suboptimal local solution, and improves convergence efficiency by enhancing sample utilization. Finally, we examine the impact of different parallel levels and training step differences on system performance metrics and explain the parameter choices. Experimental results show that the proposed method provides a viable solution to the performance issues caused by slow convergence and local minima, with PEATDRL improving task queue convergence speed by more than 20% compared to baseline algorithms. [ABSTRACT FROM AUTHOR]

Industrial Internet of Things (IIoT) has been envisioned as a killer application of 5G and beyond. However, due to the shortness of computation ablility and batery capacity, it is challenging for IIoT devices to process latency-sensitive and resource-sensitive tasks. Moblie Edge Computing (MEC), as a promising paradigm for handling tasks with high quality of service (QoS) requirement and for energy-constrained IIoT devices, allows IIoT devices to offload their tasks to MEC servers, which can significantly reduce the task process delay and energy consumptions. However, the deployment of the MEC servers rely heavily on communication infrastructure, which greatly reduce the flexibility. Toward this end, in this paper, we consider multiple Unmanned Aerial Vehicles (UAV) eqqipped with transceivers as aerial MEC servers to provide IIoT devices computation offloading opportunities due to their high controbility. IIoT devices can choose to offload the tasks to UAVs through air-ground links, or to offload the tasks to the remote cloud center through ground cellular network, or to process the tasks locally. We formulate the multi-UAV-Enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. To obtain the energy-efficient and low complexity solution, we propose an intelligent computation offloading algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Numerical results show that the proposed MDSPR converges fast and outperforms the benchmark algorithms, including random method, deep Q-learning method and double deep Q-learning method in terms of energy efficiency and task successful rate. [ABSTRACT FROM AUTHOR]

The method of charging mobile devices with wireless power transfer (WPT) from the base station (BS) integrated with mobile edge computing (MEC) increases the potential of MEC. The increasing demand for intelligent computation offloading requires effective decisions among local or remote computation specifically in wireless fading channels of the dynamic environment. Our main aim is to generate an effective offloading decision between local and remote computation in a real-time environment for each wireless channel while preserving optimal computation rate. In this article, we consider a wireless powered MEC system that governs a binary offloading decision to execute the task locally at the edge devices or the remote server. We propose a reinforcement learning based intelligent online offloading (RLIO) framework that adopts an optimal offloading action based on reinforcement methods. This framework acquires a worthy decision among local or remote computation for the time varying wireless channel conditions in dense networks. Numerical results show that the proposed framework can achieve optimal performance while preserving the computation time compared with existing optimization methods. Second, the average execution cost of RLIO is less than 0.4 ms per channel, which enables real-time and optimal offloading in dynamic and large-scale networks. [ABSTRACT FROM AUTHOR]

Today, Mobile Cloud Computing (MCC) alone can no longer respond to the increasing volume of data and satisfy the necessary delays in real-time applications. In addition, challenges such as security, energy consumption, storage space, bandwidth, lack of mobility support, and lack of location awareness have made this problem more challenging. Expanding applications such as online gaming, Augmented Reality (AR), Virtual Reality (VR), metaverse, e-health, and the Internet of Things (IoT) have brought up new paradigms for processing big data. Some of the paradigms that have emerged in the last decade are trying to alleviate cloud computing problems jointly. Mobile Edge Computing (MEC) and Fog Computing (FC) are the most critical techniques that serve the IoT. One of the common points of the above paradigms is the offloading of IoT tasks. This paper reviews machine learning-based computation offloading mechanisms in the edge and fog environment. This review covers three significant areas of machine learning: supervised learning, unsupervised learning, and reinforcement learning. We discuss various performance metrics, tools, and case studies and analyze their advantages and disadvantages. We systematically elaborate on open issues and research challenges that are crucial for the next decade. [ABSTRACT FROM AUTHOR]

Internet of Things devices generate a large number of heterogeneous workloads in real-time that require specific application to tackle, and the inability to communicate between devices and communication base stations due to complex scenarios is a thorny issue. Service caching play a key role in managing specific-request workload from devices, and unmanned aerial vehicles with computation and communication functions can effectively solve communication barrier between devices and ground base stations. In addition, the joint optimization of workload offloading and service cache placement is a key issue. Accordingly, we design an unmanned aerial vehicle-enabled mobile edge computing system with multiple devices, unmanned aerial vehicles and edge servers. The proposed framework takes into account the randomness of workload arrival, the time-varying nature of channel states, the limitations of the hosting service caching, and wireless communication blocking. Furthermore, we designed workload offloading and service caching hosting decision-making optimization problems to minimize the long-term weighted average latency and energy consumption costs. To tackle this joint optimization problem, we propose a request-specific workload offloading and service caching decision-making scheme based on the medley deep reinforcement learning scheme. To this end, the proposed scheme is decomposed into two-stage optimization subproblems: the workload offloading decision-making problem and the service caching hosting selection problem. In terms of the first subproblem, we model each device as a learning agent and propose the workloads offloading decision-making scheme based on multi-agent deep deterministic policy gradient. For the second subproblem, we present the decentralized double deep Q-learning scheme to tackle the service caching hosting policy. According to the comprehensive experimental results, the proposed scheme is able to converge rapidly on various parameter configurations and whose performance surpasses the other four baseline learning algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we investigate a multiuser mobile edge computing (MEC)-aided smart Internet of vehicle (IoV) network, where one edge server can help accomplish the intensive calculating tasks from the vehicular users. For the MEC networks, most existing works mainly focus on minimizing the system latency to guarantee the user's quality of service (QoS) through designing some offloading strategies, which, however, fail to consider the pricing from the server and hence fail to take into account the budget constraint from the users. To address this issue, we jointly incorporate the budget constraint into the system design of the MEC-based IoV networks and then propose a joint deep reinforcement learning (DRL) approach combined with the convex optimization algorithm. Specifically, a deep Q-network (DQN) is firstly used to make the offloading decision, and then, the Lagrange multiplier method is employed to allocate the calculating capability of the server to multiple users. Simulations are finally presented to demonstrate that the proposed schemes outperform the conventional ones. In particular, the proposed scheme can effectively reduce the system latency by up to 56% compared to the conventional schemes. [ABSTRACT FROM AUTHOR]

With the emergence and development of 5G technology, Mobile Edge Computing (MEC) has been closely integrated with Internet of Vehicles (IoV) technology, which can effectively support and improve network performance in IoV. However, the high-speed mobility of vehicles and diversity of communication quality make computing task offloading strategies more complex. To solve the problem, this paper proposes a computing resource allocation scheme based on deep reinforcement learning network for mobile edge computing scenarios in IoV. Firstly, the task resource allocation model for IoV in corresponding edge computing scenario is determined regarding the computing capacity of service nodes and vehicle moving speed as constraints. Besides, the mathematical model for task offloading and resource allocation is established with the minimum total computing cost as objective function. Then, deep Q-learning network based on deep reinforcement learning network is proposed to solve the mathematical model of resource allocation. Moreover, experience replay method is used to solve the instability of nonlinear approximate function neural network, which can avoid falling into dimension disaster and ensure the low-overhead and low-latency operation requirements of resource allocation. Finally, simulation results show that proposed scheme can effectively allocate the computing resources of IoV in edge computing environment. When the number of user uploaded data is 10K bits and the number of terminals is 15, it still shows the excellent network performance of low-overhead and low-latency. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid development of Internet of Things (IoTs) and artificial intelligence, vehicular networks have transformed from simple interactive systems to smart integrated networks. The accompanying intelligent connected vehicles (ICVs) can communicate with each other and connect to the urban traffic information network, to support intelligent applications, i.e., autonomous driving, intelligent navigation, and in-vehicle entertainment services. These applications are usually delay-sensitive and compute-intensive, with the result that the computation resources of vehicles cannot meet the quality requirements of service for vehicles. To solve this problem, vehicular edge computing networks (VECNs) that utilize mobile edge computing offloading technology are seen as a promising paradigm. However, existing task offloading schemes lack consideration of the highly dynamic feature of vehicular networks, which makes them unable to give time-varying offloading decisions for dynamic changes in vehicular networks. Meanwhile, the current mobility model cannot truly reflect the actual road traffic situation. Toward this end, we study the task offloading problem in VECNs with the synchronized random walk model. Then, we propose a reinforcement learning-based scheme as our solution, and verify its superior performance in processing delay reduction and dynamic scene adaptability. [ABSTRACT FROM AUTHOR]

In wireless Internet of Things (IoT) systems, the multi-input multi-output (MIMO) and cognitive radio (CR) techniques are usually involved into the mobile edge computing (MEC) structure to improve the spectrum efficiency and transmission reliability. However, such a CR based MIMO IoT system will suffer from a variety of smart attacks from wireless environments, even the MEC servers in IoT systems are not secure enough and vulnerable to these attacks. In this paper, we investigate a secure communication problem in a cognitive MIMO IoT system comprising of a primary user (PU), a secondary user (SU), a smart attacker and several MEC servers. The target of our system design is to optimize utility of the SU, including its efficiency and security. The SU will choose an idle MEC server that is not occupied by the PU in the CR scenario, and allocates a proper offloading rate of its computation tasks to the server, by unloading such tasks with proper transmit power. In such a CR IoT system, the attacker will select one type of smart attacks. Then two deep reinforcement learning based resource allocation strategies are proposed to find an optimal policy of maximal utility without channel state information(CSI), one of which is the Dyna architecture and Prioritized sweeping based Edge Server Selection (DPESS) strategy, and the other is the Deep Q-network based Edge Server Selection (DESS) strategy. Specifically, the convergence speed of the DESS scheme is significantly improved due to the trained convolutional neural network (CNN) by utilizing the experience replay technique and stochastic gradient descent (SGD). In addition, the Nash equilibrium and existence conditions of the proposed two schemes are theoretically deduced for the modeled MEC game against smart attacks. Compared with the traditional Q-learning algorithm, the average utility and secrecy capacity of the SU can be improved by the proposed DPESS and DESS schemes. Numerical simulations are also presented to verify the better performance of our proposals in terms of efficiency and security, including the higher convergence speed of the DESS strategy. [ABSTRACT FROM AUTHOR]

The purpose is to solve the problems of high transmission rate and low delay in the deployment of mobile edge computing network, ensure the security and effectiveness of the Internet of things (IoT), and save resources. Dynamic power management is adopted to control the working state transition of Edge Data Center (EDC) servers. A load prediction model based on long-short term memory (LSTM) is creatively proposed. The innovation of the model is to shut down the server in idle state or low utilization in EDC, consider user mobility and EDC location information, learn the global optimal dynamic timeout threshold strategy and N-policy through trial and error reinforcement learning method, reasonably control the working state switching of the server, and realize load prediction and analysis. The results show that the performance of AdaGrad optimization solver is the best when the feature dimension is 3, the number of LSTM network layers is 6, the time series length is 30–45, the batch size is 128, the training time is 788 s, the number of units is 250, and the number of times is 350. Compared with the traditional methods, the proposed load prediction model and power management mechanism improve the prediction accuracy by 4.21%. Compared with autoregressive integrated moving average (ARIMA) load prediction, the dynamic power management method of LSTM load prediction can reduce energy consumption by 12.5% and realize the balance between EDC system performance and energy consumption. The system can effectively meet the requirements of multi-access edge computing (MEC) for low delay, high bandwidth and high reliability, reduce unnecessary energy consumption and waste, and reduce the cost of MEC service providers in actual operation. This exploration has important reference value for promoting the energy-saving development of Internet-related industries. [ABSTRACT FROM AUTHOR]

Computation offloading at mobile edge computing (MEC) servers can mitigate the resource limitation and reduce the communication latency for mobile devices. Thereby, in this study, we proposed an offloading model for a multi-user MEC system with multi-task. In addition, a new caching concept is introduced for the computation tasks, where the application program and related code for the completed tasks are cached at the edge server. Furthermore, an efficient model of task offloading and caching integration is formulated as a nonlinear problem whose goal is to reduce the total overhead of time and energy. However, solving these types of problems is computationally prohibitive, especially for large-scale of mobile users. Thus, an equivalent form of reinforcement learning is created where the state spaces are defined based on all possible solutions and the actions are defined on the basis of movement between the different states. Afterwards, two effective Q-learning and Deep-Q-Network-based algorithms are proposed to derive the near-optimal solution for this problem. Finally, experimental evaluations verify that our proposed model can substantially minimize the mobile devices' overhead by deploying computation offloading and task caching strategy reasonably. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) as an effective tool has attracted great attention in wireless communication field nowadays. In this paper, we investigate the offloading decision and resource allocation problem in mobile edge computing (MEC) systems based on RL methods. Different from existing literature, our research focuses on improving mobile operators' revenue by maximizing the amount of the offloaded tasks while decreasing the energy expenditure and time-delays. Considering the dynamic characteristics of wireless environment, the above problem is modeled as a Markov decision process (MDP). Since the action space of the MDP is multidimensional continuous variables mixed with discrete variables, traditional RL algorithms are powerless. Therefore, an actor-critic (AC) with eligibility traces algorithm is proposed to resolve the problem. The actor part introduces the parameterized normal distribution to generate the probabilities of continuous stochastic actions, and the critic part employs a linear approximator to estimate the value of states, based on which the actor part updates policy parameters in the direction of performance improvement. Furthermore, an advantage function is designed to reduce the variance of the learning process. Simulation results indicate that the proposed algorithm can find the best strategy to maximize the amount of the tasks executed by the MEC server while decreasing the energy consumption and time-delays. [ABSTRACT FROM AUTHOR]

In a high-speed free-flow scenario, a joint optimization scheme for content caching and resource allocation is proposed based on mobile edge computing in Internet of Vehicles. Vehicle trajectory prediction provides the basis for the realization of vehicle-cloud collaborative cache. By pre-caching the business data of requesting vehicles to edge cloud networks and oncoming vehicles, requesting vehicles can obtain data through V2V link and V2I link at the same time, which reduces the data acquisition delay. Therefore, this paper considers the situation where bandwidth of V2I and V2V link and the total amount of edge cloud caches are limited. Then, the bandwidth and cache joint allocation strategy to minimize the weighted average delay of data acquisition is studied. An edge cooperative cache algorithm based on deep deterministic policy gradient is further developed. Different from Q-learning and deep reinforcement learning algorithms, the proposed cache algorithm can be well applied to variable continuous bandwidth allocation action space. Besides, it effectively improves the convergence speed by using interactive iteration of value function and strategy function. Finally, the simulation results of vehicle driving path at the start and stop are obtained by analyzing real traffic data. Simulation results show that the proposed scheme can achieve better performance than several other newer cooperative cache schemes. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. In this paper, an MEC enabled multi-user multi-input multi-output (MIMO) system with stochastic wireless channels and task arrivals is considered. In order to minimize long-term average computation cost in terms of power consumption and buffering delay at each user, a deep reinforcement learning (DRL)-based dynamic computation offloading strategy is investigated to build a scalable system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn decentralized computation offloading policies at all users respectively, where local execution and task offloading powers will be adaptively allocated according to each user's local observation. Numerical results demonstrate that the proposed DDPG-based strategy can help each user learn an efficient dynamic offloading policy and also verify the superiority of its continuous power allocation capability to policies learned by conventional discrete action space-based reinforcement learning approaches like deep Q-network (DQN) as well as some other greedy strategies with reduced computation cost. Besides, power-delay tradeoff for computation offloading is also analyzed for both the DDPG-based and DQN-based strategies. [ABSTRACT FROM AUTHOR]

With the development of communication technology and mobile edge computing (MEC), self-driving has received more and more research interests. However, most object detection tasks for self-driving vehicles are still performed at vehicle terminals, which often requires a trade-off between detection accuracy and speed. To achieve efficient object detection without sacrificing accuracy, we propose an end–edge collaboration object detection approach based on Deep Reinforcement Learning (DRL) with a task prioritization mechanism. We use a time utility function to measure the efficiency of object detection task and aim to provide an online approach to maximize the average sum of the time utilities in all slots. Since this is an NP-hard mixed-integer nonlinear programming (MINLP) problem, we propose an online approach for task offloading and resource allocation based on Deep Reinforcement learning and Piecewise Linearization (DRPL). A deep neural network (DNN) is implemented as a flexible solution for learning offloading strategies based on road traffic conditions and wireless network environment, which can significantly reduce computational complexity. In addition, to accelerate DRPL network convergence, DNN outputs are grouped by in-vehicle cameras to form offloading strategies via permutation. Numerical results show that the DRPL scheme is at least 10% more effective and superior in terms of time utility compared to several representative offloading schemes for various vehicle local computing resource scenarios. [ABSTRACT FROM AUTHOR]

Mobile Edge Computing (MEC) and Non-Orthogonal Multiple Access (NOMA) have been treated as promising technologies to process the delay-sensitive tasks in the Industrial Internet of Things (IIoT) network. The cooperation among multiple MEC servers is essential to improve the processing capacity of MEC systems. However, the dynamic IIoT environment with unknown changing models, including time-varying wireless channels, diversified task requests, and dynamic load on wireless resources and multiple MEC servers, may continuously affect the task offloading decision and NOMA user pairing, which brings great challenges to the resource management in the NOMA-MEC-based IIoT network. In order to solve this problem, we design a distributed deep reinforcement learning (DRL) based solution to improve the task satisfaction ratio by jointly optimizing the task offloading decision and the sub-channel assignment to support the binary computing offloading policy. For each IIoT device agent, to deal with the problem of partial state observability, the Recurrent Neural Network (RNN) is employed to predict the load states of sub-channels and MEC servers, which is further used for the decision of the RL agent. Simulation results show that the proposed prediction-based-DRL (P-DRL) method can achieve higher task satisfaction ratio than exiting schemes. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) can dispatch its powerful servers close by to assist with the computation workloads that intelligent wireless terminals have offloaded. The MEC server's physical location is closer to the intelligent wireless terminals, which can satisfy the low latency and high reliability demands. In this paper, we formulate an MEC framework with multiple vehicles and service devices that considers the priority and randomness of arriving workloads from roadside units (RSUs), cameras, laser radars (Lidar) and the time-varying channel state between the service device and MEC server (MEC-S). To minimize the long-term weighted average cost of the proposed MEC system, we transit this issue (cost minimization problem) into the Markov decision process (MDP). Furthermore, considering the difficulty realizing the state transition probability matrix, the dimensional complexity of the state space, and the continuity of the action space, we propose a deterministic policy gradient (MADDPG)-based bandwidth partition and power allocation optimization policy. The proposed MADDPG-based policy is a model-free deep reinforcement learning (DRL) method, which can effectively deal with continuous action space and further guide multi-agent to execute decision-making. The comprehensive results verify that the proposed MADDPG-based optimization scheme has fine convergence and performance that is better than that of the other four baseline algorithms. [ABSTRACT FROM AUTHOR]

The edge-cloud computing architecture has been introduced to industrial circles to ensure the time constraints for industrial computing tasks. Besides the central cloud, various numbers of edge servers (ESes) are deployed in a distributed manner. In the meantime, most large factories currently use auto guided vehicles (AGVs). They usually travel along a given route and can help offload tasks to ESes. An ES maybe accessed by multiple AGVs, thus incurring offloading and processing delays due to resource competition. In this paper, we investigate the offloading scheduling issue for cyclical tasks and put forth the Multi-AGV Cyclical Offloading Optimization (MCOO) algorithm to reduce conflicts. The solution divides the offloading optimization problem into two parts. Firstly, the load balancing algorithm and greedy algorithm are utilized to find the optimal allocation of tasks for a single AGV under limited conditions. Then, multiple AGVs are asynchronously trained by applying the Reinforcement Learning-based A3C algorithm to optimize the offloading scheme. The simulation results show that the MCOO algorithm improves the global offloading performance both in task volume and adaptability compared with the baseline algorithms. [ABSTRACT FROM AUTHOR]