Search

With the rapid development of the Internet of Things (IoT) and Internet technology, the product of the combination of the two, the Industrial Internet, has also received extensive attention and there are more and more research achievements related to the Industrial Internet. In the industrial Internet system, the communication network system composed of sensors, communication nodes, controllers and other intelligent devices can realize efficient and convenient data interaction between people and machines, providing an important infrastructure and technical support for industrial big data analysis and intelligent production. However, in the current industrial Internet system, industrial equipment users generally have the problem of low computing energy efficiency, and the collected industrial data has a high-security risk in the transmission, processing and other processes. At the same time, the size and scale of the industrial Internet equipment group is huge, and the lack of rational resource allocation leads to excessive waste of computing resources in the system, which is also a prominent problem of the current industrial Internet system. In response to the above questions, this paper, on the basis of reading a large number of documents, integrates the improved DRL algorithm, End-Edge-Cloud architecture and blockchain to form a new industrial Internet architecture. The architecture realizes computing offload through the three-tier structure of terminal layer, edge layer and cloud layer, and guarantees the security of industrial data through the decentralized feature of blockchain, ultimately achieving the goal of reducing energy consumption, computing overhead and trusted computing. In the architecture proposed in this paper, the dynamic unloading of industrial data and computing tasks is achieved through a three-tier architecture. The MDP is used to build an optimization problem model, and the improved DRL algorithm is used to iteratively solve the optimal computing resource scheduling strategy. The main research contents of this paper include (1) Using MDP to model optimization problems; (2) Propose an industrial Internet system architecture that integrates and improves DRL, "end edge cloud" and blockchain; (3) The MDP problem is solved iteratively based on deep reinforcement learning. The simulation results show that the proposed architecture has more advantages than the existing six architectures in terms of computing cost, equipment energy consumption and total working time. [ABSTRACT FROM AUTHOR]

Online detection devices, powered by artificial intelligence technologies, enable the comprehensive and continuous detection of high-speed railways (HSRs). However, the computation-intensive and latency-sensitive nature of these detection tasks often exceeds local processing capabilities. Mobile Edge Computing (MEC) emerges as a key solution in the railway Internet of Things (IoT) scenario to address these challenges. Nevertheless, the rapidly varying channel conditions in HSR scenarios pose significant challenges for efficient resource allocation. In this paper, a computation offloading system model for detection tasks in the railway IoT scenario is proposed. This system includes direct and relay transmission models, incorporating Non-Orthogonal Multiple Access (NOMA) technology. This paper focuses on the offloading strategy for subcarrier assignment, mode selection, relay power allocation, and computing resource management within this system to minimize the average delay ratio (the ratio of delay to the maximum tolerable delay). However, this optimization problem is a complex Mixed-Integer Non-Linear Programming (MINLP) problem. To address this, we present a low-complexity subcarrier allocation algorithm to reduce the dimensionality of decision-making actions. Furthermore, we propose an improved Deep Deterministic Policy Gradient (DDPG) algorithm that represents discrete variables using selection probabilities to handle the hybrid action space problem. Our results indicate that the proposed system model adapts well to the offloading issues of detection tasks in HSR scenarios, and the improved DDPG algorithm efficiently identifies optimal computation offloading strategies. [ABSTRACT FROM AUTHOR]

In recent years, many mobile edge computing network solutions have enhanced data privacy and security and built a trusted network mechanism by introducing blockchain technology. However, this also complicates the task-offloading problem of blockchain-enabled mobile edge computing, and traditional evolutionary learning and single-agent reinforcement learning algorithms are difficult to solve effectively. In this paper, we propose a blockchain-enabled mobile edge computing task-offloading strategy based on multi-agent reinforcement learning. First, we innovatively propose a blockchain-enabled mobile edge computing task-offloading model by comprehensively considering optimization objectives such as task execution energy consumption, processing delay, user privacy metrics, and blockchain incentive rewards. Then, we propose a deep reinforcement learning algorithm based on multiple agents sharing a global memory pool using the actor–critic architecture, which enables each agent to acquire the experience of another agent during the training process to enhance the collaborative capability among agents and overall performance. In addition, we adopt attenuatable Gaussian noise into the action space selection process in the actor network to avoid falling into the local optimum. Finally, experiments show that this scheme's comprehensive cost calculation performance is enhanced by more than 10% compared with other multi-agent reinforcement learning algorithms. In addition, Gaussian random noise-based action space selection and a global memory pool improve the performance by 38.36% and 43.59%, respectively. [ABSTRACT FROM AUTHOR]

Industrial Internet of Things (IIoT) has been envisioned as a killer application of 5G and beyond. However, due to the shortness of computation ablility and batery capacity, it is challenging for IIoT devices to process latency-sensitive and resource-sensitive tasks. Moblie Edge Computing (MEC), as a promising paradigm for handling tasks with high quality of service (QoS) requirement and for energy-constrained IIoT devices, allows IIoT devices to offload their tasks to MEC servers, which can significantly reduce the task process delay and energy consumptions. However, the deployment of the MEC servers rely heavily on communication infrastructure, which greatly reduce the flexibility. Toward this end, in this paper, we consider multiple Unmanned Aerial Vehicles (UAV) eqqipped with transceivers as aerial MEC servers to provide IIoT devices computation offloading opportunities due to their high controbility. IIoT devices can choose to offload the tasks to UAVs through air-ground links, or to offload the tasks to the remote cloud center through ground cellular network, or to process the tasks locally. We formulate the multi-UAV-Enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. To obtain the energy-efficient and low complexity solution, we propose an intelligent computation offloading algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Numerical results show that the proposed MDSPR converges fast and outperforms the benchmark algorithms, including random method, deep Q-learning method and double deep Q-learning method in terms of energy efficiency and task successful rate. [ABSTRACT FROM AUTHOR]

Mobile Edge Computing (MEC) has been widely applied in various Internet of Things (IoT) scenarios due to its advantages of low latency and low energy consumption. However, the offloading of tasks generated by terminal devices to edge servers inevitably raises privacy leakage concerns. Given the limited resources in MEC networks, this paper proposes a task scheduling strategy, named DQN-DP, to minimize location privacy leakage under the constraint of offloading costs. The strategy is based on a differential privacy location obfuscation probability density function. Theoretical analysis demonstrates that the probability density function employed in this study is valid and satisfies ϵ -differential privacy in terms of security. Numerical results indicate that, compared to existing baseline approaches, the proposed DQN-DP algorithm effectively balances privacy leakage and offloading cost. Specifically, DQN-DP reduces privacy leakage by approximately 20% relative to baseline approaches. [ABSTRACT FROM AUTHOR]

With the proliferation of intelligent applications, mobile devices are increasingly handling computation-intensive tasks but often struggle with limited computing power and energy resources. Mobile Edge Computing (MEC) offers a solution by enabling these devices to offload computation-intensive tasks to resource-rich edge servers, thus reducing processing latency and energy consumption. However, existing task-offloading strategies often neglect critical security concerns. In this paper, we propose a security-aware task-offloading framework that utilizes Deep Reinforcement Learning (DRL) to solve these challenges. Our framework is designed to minimize the latency of task accomplishment and energy consumption while ensuring data security. We model system utility as a Markov Decision Process (MDP) and design a Proximal Policy Optimization (PPO)-based algorithm to derive optimal offloading strategies. Experimental results demonstrate that the proposed algorithm outperforms traditional methods regarding task execution latency and energy consumption. [ABSTRACT FROM AUTHOR]

With the boom in mobile internet services, computationally intensive applications such as virtual and augmented reality have emerged. Mobile edge computing (MEC) technology allows mobile devices to offload heavy computational tasks to edge servers, which are located at the edge of the network. This technique is considered an effective approach to help reduce the burden on devices and enable efficient task offloading. This paper addresses a dynamic real-time task-offloading problem within a stochastic multi-user MEC network, focusing on the long-term stability of system energy consumption and energy budget constraints. To solve this problem, a task-offloading strategy with long-term constraints is proposed, optimized through the construction of multiple queues to maintain users' long-term quality of experience (QoE). The problem is decoupled using Lyapunov theory into a single time-slot problem, modeled as a Markov decision process (MDP). A deep reinforcement learning (DRL)-based LMADDPG algorithm is introduced to solve the task-offloading decision. Finally, Experiments are conducted under the constraints of a limited MEC energy budget and the need to maintain the long-term energy stability of the system. The results from simulation experiments demonstrate that the algorithm outperforms other baseline algorithms in terms of task-offloading decisions. [ABSTRACT FROM AUTHOR]

Vehicular Edge Computing (VEC) is a crucial application of Mobile Edge Computing (MEC) in vehicular networks. In VEC networks, the computation tasks of vehicle terminals (VTs) can be offloaded to nearby MEC servers, overcoming the limitations of VTs' processing power and reducing latency caused by distant cloud communication. However, a mismatch between VTs' demanding tasks and MEC servers' limited resources can overload MEC servers, impacting Quality of Service (QoS) for computationally intensive tasks. Additionally, vehicle mobility can disrupt communication with static MEC servers, further affecting VTs' QoS. To address these challenges, this paper proposes a vehicle collaborative partial computation offloading model. This model allows VTs to offload tasks to two types of service nodes: collaborative vehicles and MEC servers. Factors like a vehicle's mobility, remaining battery power, and available computational power are also considered when evaluating its suitability for collaborative offloading. Furthermore, we design a deep reinforcement learning-based strategy for collaborative partial computation offloading that minimizes overall task delay while meeting individual latency constraints. Experimental results demonstrate that compared to traditional approaches without vehicle collaboration, this scheme significantly reduces latency and achieves a significant reduction (around 2%) in the failure rate under tighter latency constraints. [ABSTRACT FROM AUTHOR]

In this paper, we consider an integrated sensing, communication, and computation (ISCC) system to alleviate the spectrum congestion and computation burden problem. Specifically, while serving communication users, a base station (BS) actively engages in sensing targets and collaborates seamlessly with the edge server to concurrently process the acquired sensing data for efficient target recognition. A significant challenge in edge computing systems arises from the inherent uncertainty in computations, mainly stemming from the unpredictable complexity of tasks. With this consideration, we address the computation uncertainty by formulating a robust communication and computing resource allocation problem in ISCC systems. The primary goal of the system is to minimize total energy consumption while adhering to perception and delay constraints. This is achieved through the optimization of transmit beamforming, offloading ratio, and computing resource allocation, effectively managing the trade-offs between local execution and edge computing. To overcome this challenge, we employ a Markov decision process (MDP) in conjunction with the proximal policy optimization (PPO) algorithm, establishing an adaptive learning strategy. The proposed algorithm stands out for its rapid training speed, ensuring compliance with latency requirements for perception and computation in applications. Simulation results highlight its robustness and effectiveness within ISCC systems compared to baseline approaches. [ABSTRACT FROM AUTHOR]

The limited coverage extension of mobile edge computing (MEC) necessitates exploring cooperation with unmanned aerial vehicles (UAV) to leverage advanced features for future computation-intensive and mission-critical applications. Moreover, the workflow for task offloading in software-defined networking (SDN)-enabled 5G is significant to tackle in UAV-MEC networks. In this paper, deep reinforcement learning (DRL) SDN control methods for improving computing resources are proposed. DRL-based SDN controller, termed DRL-SDNC, allocates computational resources, bandwidth, and storage based on task requirements, upper-bound tolerable delays, and network conditions, using the UAV system architecture for task exchange between MECs. DRL-SDNC configures rule installation based on state observations and agent evaluation indicators, such as network congestion, user equipment computational capabilities, and energy efficiency. This paper also proposes the training deep network architecture for the DRL-SDNC, enabling interactive and autonomous policy enforcement. The agent learns from the UAV-MEC environment through experience gathering and updates its parameters using optimization methods. DRL-SDNC collaboratively adjusts hyperparameters and network architecture to enhance learning efficiency. Compared with baseline schemes, simulation results demonstrate the effectiveness of the proposed approach in optimizing resource efficiency and achieving satisfied quality of service for efficient utilization of computing and communication resources in UAV-assisted networking environments. [ABSTRACT FROM AUTHOR]

With the emergence and development of 5G technology, Mobile Edge Computing (MEC) has been closely integrated with Internet of Vehicles (IoV) technology, which can effectively support and improve network performance in IoV. However, the high-speed mobility of vehicles and diversity of communication quality make computing task offloading strategies more complex. To solve the problem, this paper proposes a computing resource allocation scheme based on deep reinforcement learning network for mobile edge computing scenarios in IoV. Firstly, the task resource allocation model for IoV in corresponding edge computing scenario is determined regarding the computing capacity of service nodes and vehicle moving speed as constraints. Besides, the mathematical model for task offloading and resource allocation is established with the minimum total computing cost as objective function. Then, deep Q-learning network based on deep reinforcement learning network is proposed to solve the mathematical model of resource allocation. Moreover, experience replay method is used to solve the instability of nonlinear approximate function neural network, which can avoid falling into dimension disaster and ensure the low-overhead and low-latency operation requirements of resource allocation. Finally, simulation results show that proposed scheme can effectively allocate the computing resources of IoV in edge computing environment. When the number of user uploaded data is 10K bits and the number of terminals is 15, it still shows the excellent network performance of low-overhead and low-latency. [ABSTRACT FROM AUTHOR]

Mobile edge computing extends the capabilities of the cloud to the edge to meet the latency performance required by new types of applications. Task caching reduces network energy consumption by caching task applications and associated databases in advance on edge devices. However, determining an effective caching strategy is crucial since users generate numerous repetitive tasks, but edge devices and storage resources are limited. We aimed to address the problem of highly coupled decision variables in dynamic task caching and computational offloading for multiuser multitasking in mobile edge computing systems. This paper presents a joint computation and caching framework with the aim of minimizing delays and energy expenditure for mobile users and transforming the problem into a form of reinforcement learning. Based on this, an improved deep reinforcement learning algorithm, P-DDPG, is proposed to achieve efficient computation offloading and task caching decisions for mobile users. The algorithm integrates a deep and deterministic policy grading and a prioritized empirical replay mechanism to reduce system costs. The simulations show that the designed algorithm performs better in terms of task latencies and lower computing power consumption. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) integration with 5G/6G technologies is an essential direction in mobile communications and computing. However, it is crucial to be aware of the potential privacy implications of task offloading in MEC scenarios, specifically the leakage of user location information. To address this issue, this paper proposes a location-privacy-preserved task offloading (LPTO) scheme based on safe reinforcement learning to balance computational cost and privacy protection. This scheme uses the differential privacy technique to perturb the user's actual location to achieve location privacy protection. We model the privacy-preserving location perturbation problem as a Markov decision process (MDP), and we develop a safe deep Q-network (DQN)-based LPTO (SDLPTO) scheme to select the offloading policy and location perturbation policy dynamically. This approach effectively mitigates the selection of high-risk state–action pairs by conducting a risk assessment for each state–action pair. Simulation results show that the proposed SDLPTO scheme has a lower computational cost and location privacy leakage than the benchmarks. These results highlight the significance of our approach in protecting user location privacy while achieving improved performance in MEC environments. [ABSTRACT FROM AUTHOR]

Developing an effective task offloading strategy has been a focus of research to improve the task processing speed of IoT devices in recent years. Some of the reinforcement learning-based policies can improve the dependence of heuristic algorithms on models through continuous interactive exploration of the edge environment; however, when the environment changes, such reinforcement learning algorithms cannot adapt to the environment and need to spend time on retraining. This paper proposes an adaptive task offloading strategy based on meta reinforcement learning with task latency and device energy consumption as optimization targets to overcome this challenge. An edge system model with a wireless charging module is developed to improve the ability of IoT devices to provide service constantly. A Seq2Seq-based neural network is built as a task strategy network to solve the problem of difficult network training due to different dimensions of task sequences. A first-order approximation method is proposed to accelerate the calculation of the Seq2Seq network meta-strategy training, which involves quadratic gradients. The experimental results show that, compared with existing methods, the algorithm in this paper has better performance in different tasks and network environments, can effectively reduce the task processing delay and device energy consumption, and can quickly adapt to new environments. [ABSTRACT FROM AUTHOR]

In the 5G era, the amount of network data has grown explosively. A large number of new computation-intensive applications have created demand for edge computing in mobile networks. Traditional optimization methods are difficult to adapt to the dynamic wireless network environment because they solve the problem online, which is not suitable in edge computing scenarios. Therefore, in order to obtain a mobile network with better performance, we propose a network frame with a resource allocation algorithm based on power consumption, delay and user cooperation. This algorithm can quickly realize the optimization of a network to improve performance. Specifically, compared with heuristic algorithms, such as particle swarm optimization, ant colony algorithm, etc., commonly used to solve such problems, the algorithm proposed in this paper can reduce some aspects of network performance (including delay and user energy consumption) by about 10% in a network dominated by downlink tasks. The performance of the algorithm under certain network conditions was demonstrated through simulations. [ABSTRACT FROM AUTHOR]

At present, with the intelligence that has been achieved in computer and communication technologies, vehicles can provide many convenient functions to users. However, it is difficult for a vehicle to deal with computationally intensive and latency-sensitive tasks occurring in the vehicle environment by itself. To this end, mobile edge computing (MEC) services have emerged. However, MEC servers (MECSs), which are fixed on the ground, cannot flexibly respond to temporal dynamics where tasks are temporarily increasing, such as commuting time. Therefore, research has examined the provision of edge services using additional unmanned aerial vehicles (UAV) with mobility. Since these UAVs have limited energy and computing power, it is more important to optimize energy efficiency through load balancing than it is for ground MEC servers (MECSs). Moreover, if only certain servers run out of energy, the service coverage of a MEC server (MECS) may be limited. Therefore, all UAV MEC servers (UAV MECSs) need to use energy evenly. Further, in a high-mobility vehicle environment, it is necessary to have effective task migration because the UAV MECS that provides services to the vehicle changes rapidly. Therefore, in this paper, a federated deep Q-network (DQN)-based task migration strategy that considers the load deviation and energy deviation among UAV MECSs is proposed. DQN is used to create a local model for migration optimization for each of the UAV MECSs, and federated learning creates a more effective global model based on the fact that it has common spatial features between adjacent regions. To evaluate the performance of the proposed strategy, the performance is analyzed in terms of delay constraint satisfaction, load deviation, and energy deviation. [ABSTRACT FROM AUTHOR]

In many IoT scenarios, the resources of terminal devices are limited, and it is difficult to provide services with low latency and low energy consumption. Mobile edge computing is an effective solution by offloading computing tasks to edge server processing. There are some problems in the existing offloading decision algorithms: the offloading decision method based on heuristic algorithms cannot dynamically adjust the policy in the changing environment; the offloading algorithm based on deep reinforcement learning will lead to slow convergence and poor exploration effect due to the problem of dimension explosion. To solve the above problems, this paper designs an offloading decision algorithm to make dynamic decisions in a mobile edge computing network with multi-device access. The algorithm comprehensively considers the energy consumption of terminal equipment, offloading overhead, average delay and success rate of task completion, aiming to achieve the highest total revenue of the whole system in a period of time. In this work, the online offloading problem is abstracted as a Markov decision process. Based on the Double Dueling Deep Q-Network (D3QN) algorithm, the offloading decision is designed to adapt to the highly dynamic environment of the edge computing network and solve the problem of high state space complexity. In addition, this paper innovatively introduces a clustering algorithm into deep reinforcement learning (DRL) to preprocess the action space and solve the explosion problem of the action space dimension caused by the increase of terminal devices. The experimental results show that the proposed algorithm is superior to the baseline strategies such as Deep Q-Network (DQN) algorithm in convergence speed and total reward. [ABSTRACT FROM AUTHOR]

This paper proposes a novel dynamic offloading decision method which is inspired by deep reinforcement learning (DRL). In order to realize real-time communications in mobile edge computing systems, an efficient task offloading algorithm is required. When the decision of actions (offloading enabled, i.e., computing in clouds or offloading disabled, i.e., computing in local edges) is made by the proposed DRL-based dynamic algorithm in each unit time, it is required to consider real-time/seamless data transmission and energy-efficiency in mobile edge devices. Therefore, our proposed dynamic offloading decision algorithm is designed for the joint optimization of delay and energy-efficient communications based on DRL framework. According to the performance evaluation via data-intensive simulations, this paper verifies that the proposed dynamic algorithm achieves desired performance. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) can dispatch its powerful servers close by to assist with the computation workloads that intelligent wireless terminals have offloaded. The MEC server's physical location is closer to the intelligent wireless terminals, which can satisfy the low latency and high reliability demands. In this paper, we formulate an MEC framework with multiple vehicles and service devices that considers the priority and randomness of arriving workloads from roadside units (RSUs), cameras, laser radars (Lidar) and the time-varying channel state between the service device and MEC server (MEC-S). To minimize the long-term weighted average cost of the proposed MEC system, we transit this issue (cost minimization problem) into the Markov decision process (MDP). Furthermore, considering the difficulty realizing the state transition probability matrix, the dimensional complexity of the state space, and the continuity of the action space, we propose a deterministic policy gradient (MADDPG)-based bandwidth partition and power allocation optimization policy. The proposed MADDPG-based policy is a model-free deep reinforcement learning (DRL) method, which can effectively deal with continuous action space and further guide multi-agent to execute decision-making. The comprehensive results verify that the proposed MADDPG-based optimization scheme has fine convergence and performance that is better than that of the other four baseline algorithms. [ABSTRACT FROM AUTHOR]

The integration of Cellular Vehicle-to-Everything (C-V2X) and Mobile Edge Computing (MEC) is critical for satisfying the demanding requirements of vehicular applications, which are characterized by ultra-low latency and ultra-high reliability. In this paper, we address the challenge of jointly optimizing computation offloading and resource allocation in C-V2X network. To achieve this, we propose a hierarchical MEC/C-V2X network that accounts for the dynamic changes of the vehicular network and the diversity of computation offloading patterns. Additionally, we establish a collaborative computation offloading model that supports multiple offloading patterns. We formulate the dynamic computation offloading and resource allocation problem as a sequential decision problem based on the Markovian decision process. To enable automated and intelligent decision-making, we propose a deep reinforcement learning algorithm called ORAD, based on the deep deterministic policy gradient algorithm, to maximize offloading success rate in real-time. The numerical results demonstrate that the proposed algorithm effectively provides the optimal policy, resulting in the offloading success rate of vehicular tasks being improved by 2.73% to 95.51%. [ABSTRACT FROM AUTHOR]

Today, Mobile Cloud Computing (MCC) alone can no longer respond to the increasing volume of data and satisfy the necessary delays in real-time applications. In addition, challenges such as security, energy consumption, storage space, bandwidth, lack of mobility support, and lack of location awareness have made this problem more challenging. Expanding applications such as online gaming, Augmented Reality (AR), Virtual Reality (VR), metaverse, e-health, and the Internet of Things (IoT) have brought up new paradigms for processing big data. Some of the paradigms that have emerged in the last decade are trying to alleviate cloud computing problems jointly. Mobile Edge Computing (MEC) and Fog Computing (FC) are the most critical techniques that serve the IoT. One of the common points of the above paradigms is the offloading of IoT tasks. This paper reviews machine learning-based computation offloading mechanisms in the edge and fog environment. This review covers three significant areas of machine learning: supervised learning, unsupervised learning, and reinforcement learning. We discuss various performance metrics, tools, and case studies and analyze their advantages and disadvantages. We systematically elaborate on open issues and research challenges that are crucial for the next decade. [ABSTRACT FROM AUTHOR]

The edge-cloud computing architecture has been introduced to industrial circles to ensure the time constraints for industrial computing tasks. Besides the central cloud, various numbers of edge servers (ESes) are deployed in a distributed manner. In the meantime, most large factories currently use auto guided vehicles (AGVs). They usually travel along a given route and can help offload tasks to ESes. An ES maybe accessed by multiple AGVs, thus incurring offloading and processing delays due to resource competition. In this paper, we investigate the offloading scheduling issue for cyclical tasks and put forth the Multi-AGV Cyclical Offloading Optimization (MCOO) algorithm to reduce conflicts. The solution divides the offloading optimization problem into two parts. Firstly, the load balancing algorithm and greedy algorithm are utilized to find the optimal allocation of tasks for a single AGV under limited conditions. Then, multiple AGVs are asynchronously trained by applying the Reinforcement Learning-based A3C algorithm to optimize the offloading scheme. The simulation results show that the MCOO algorithm improves the global offloading performance both in task volume and adaptability compared with the baseline algorithms. [ABSTRACT FROM AUTHOR]

In meteorological and electric power Internet of Things scenarios, in order to extend the service life of relevant facilities and reduce the cost of emergency repair, the intelligent inspection swarm is introduced to cooperate with monitoring tasks, which collect and process the current scene data through a variety of sensors and cameras, and complete tasks such as emergency handling and fault inspection. Due to the limitation of computing resources and battery life of patrol inspection equipment, it will cause problems such as slow response in emergency and long time for fault location. Mobile Edge Computing is a promising technology, which can improve the quality of service of the swarm by offloading the computing task of the inspection equipment to the edge server nearby the network. In this paper, we study the problem of computing offloading of multidevices multi-tasks and multi-servers in the intelligent patrol inspection swarm under the condition of a dynamic network environment and limited resources of servers and inspection equipment. An effective adaptive learning offloading strategy based on distributed reinforcement learning and multi-classification is proposed to reduce the task processing delay and energy consumption of the intelligent inspection swarm and improve the service quality. Numerical experimental results demonstrate that the proposed strategy is superior to other offloading strategies in terms of time delay, energy consumption and quality of service. [ABSTRACT FROM AUTHOR]

In this paper, we consider a platform of flying mobile edge computing (F-MEC), where unmanned aerial vehicles (UAVs) serve as equipment providing computation resource, and they enable task offloading from user equipment (UE). We aim to minimize energy consumption of all UEs via optimizing user association, resource allocation and the trajectory of UAVs. To this end, we first propose a Convex optimizAtion based Trajectory control algorithm (CAT), which solves the problem in an iterative way by using block coordinate descent (BCD) method. Then, to make the real-time decision while taking into account the dynamics of the environment (i.e., UAV may take off from different locations), we propose a deep Reinforcement leArning based trajectory control algorithm (RAT). In RAT, we apply the Prioritized Experience Replay (PER) to improve the convergence of the training procedure. Different from the convex optimization based algorithm which may be susceptible to the initial points and requires iterations, RAT can be adapted to any taking off points of the UAVs and can obtain the solution more rapidly than CAT once training process has been completed. Simulation results show that the proposed CAT and RAT achieve the considerable performance and both outperform traditional algorithms. [ABSTRACT FROM AUTHOR]

In the last few years, the rapid advancement of the Internet of Things (IoT) and the widespread adoption of smart cities have posed new challenges to computing services. Traditional cloud computing models fail to fulfil the rapid response requirement of latency‐sensitive applications, while mobile edge computing (MEC) improves service efficiency and customer experience by transferring computing tasks to servers located at the network edge. However, designing an effective computing offloading strategy in complex scenarios involving multiple computing tasks, nodes, and services remains a pressing issue. In this paper, a computing offloading approach based on Deep Reinforcement Learning (DRL) is proposed for large‐scale heterogeneous computing tasks. First, Markov Decision Processes (MDPs) is used to formulate computing offloading decision and resource allocation problems in large‐scale heterogeneous MEC systems. Subsequently, a comprehensive framework comprising the "end‐edge‐cloud" along with the corresponding time‐overhead and resource allocation models is constructed. Finally, through extensive experiments on real datasets, the proposed approach is demonstrated to outperform existing methods in enhancing service response speed, reducing latency, balancing server loads, and saving energy. [ABSTRACT FROM AUTHOR]

Summary The rapid evolution of user equipment (UE) and 5G networks drives significant transformations, bringing technology closer to end‐users. Managing resources in densely crowded areas such as airports, train stations, and bus terminals poses challenges due to diverse user demands. Integrating mobile edge computing (MEC) and network function virtualization (NFV) becomes vital when the service provider's (SP) primary goal is maximizing profitability while maintaining service level agreement (SLA). Considering these challenges, our study addresses an online resource allocation problem in an MEC network where computing resources are limited, and the SP aims to boost profit by securely admitting more UE requests at each time slot. Each UE request arrival rate is unknown, and the requirement is specific resources with minimum cost and delay. The optimization problem objective is achieved by allocating resources to requests at the MEC network in appropriate cloudlets, utilizing abandoned instances, reutilizing idle and soft slice instances to shorten delay and reduce costs, and immediately scaling inappropriate instances, thus minimizing the instantiation of new instances. This paper proposes a deep reinforcement learning (DRL) method for request prediction and resource allocation to mitigate unnecessary resource waste. Simulation results demonstrate that the proposed approach effectively accepts network slice requests to maximize profit by leveraging resource availability, reutilizing instantiated resources, and upholding goodwill and SLA. Through extensive simulations, we show that our proposed DRL‐based approach outperforms other state‐of‐the‐art techniques, namely, MaxSR, DQN, and DDPG, by 76%, 33%, and 23%, respectively. [ABSTRACT FROM AUTHOR]

Summary: In mobile edge computing (MEC), due to the limited computing resources and power of mobile augmented reality (MAR) devices, cache identification results which can reduce power consumption and executing time of mobile devices are the solution to process MAR tasks. In this paper, we study an allocation cache problem in MAR systems. The allocation cache problem is formulated as maximizing the cache utility of cache hit ratio and user preference factor. To solve this problem, a cache resource allocation and cache space adjustment policy for edge computing systems is proposed. We also propose an improved double deep Q‐network (DDQN) algorithm to learn this policy. Simulation results show that the policy greatly improves the cache hit ratio compared with the traditional caching policy. [ABSTRACT FROM AUTHOR]

Computation offloading is an effective method in mobile edge computing (MEC) to relieve user equipment (UE) from the limited computation resource and battery capacity. Meanwhile, simultaneous wireless information and power transmission (SWIPT) can be applied to MEC to extend the operating time of the equipment. However, in multi-user network environment, diverse computation task requirements and changeable network channel states make it challenging to obtain offloading strategy timely and accurately. To address the issue, we propose an intelligent computation offloading scheme (iCOS) based on enhanced priority deep deterministic policy gradient (EPDDPG) algorithm to minimize the energy consumption of all the UEs by jointly optimizing the offloading decision, the central processing unit (CPU) frequency and the power split ratio in a dynamic SWIPT-MEC network. In particular, we improve the traditional fully-connected network structure to obtain both discrete and continuous action outputs, and accelerate neural network parameter updates by using prioritized experience tuples. Furthermore, we use dynamic voltage and frequency scaling (DVFS) technology to dynamically adjust the CPU frequency of local computing, and employ SWIPT technology to balance the charging and communication according to the obtained strategy. Simulation results show that the algorithm proposed in this paper can effectively reduce the energy cost of UEs, and complete more computation tasks within the delay limit. [ABSTRACT FROM AUTHOR]

The rapid development of 6G can help to bring autonomous driving closed to the reality. Drivers and passengers will have more time for work and leisure spending in the vehicles, further generating a lot of data requirements. However, edge resources from small base stations are insufficient to match the wide variety of services of the future vehicular networks. Besides, due to the high-speed nature of the vehicles, users have to switch the connections among different base stations, whereas such way will cause external latency during the data request. Therefore, it is vital to enable the local cache of vehicle users to realize the reliable autonomous driving. In this paper, we consider caching the contents in the local cache, small base station, and edge server. In practice, the request preference of some single users may be different from a whole region. To maximize the efficiency of content cache, we design a strategy that uses reinforcement learning algorithm to optimize cache schemes on different devices. The experimental results demonstrate that our strategy can enhance the cache hit ratio by 10%-20% compared with the well-known counterparts. [ABSTRACT FROM AUTHOR]

The continual development of mobile edge computing efficiently solves the problem that mobile devices are unable to handle computation-intensive tasks due to their computation capacity and battery restrictions. In this paper, we consider mobile awareness and dynamic battery charging in a multi-user and multi-server mobile edge computing system, where various tasks are generated successively on the user devices. Servers act as learning agents and collaborate with user devices to develop task partitioning and computation resource allocation strategies. With the purpose of decreasing task failure rate and improving system utility in the long term, which is closely related to latency, energy consumption, and server cost, optimal strategies are demanded by the system. We model the joint optimization problem as a multi-agent Markov decision process game. And a deep reinforcement learning method based on the multi-agent deep deterministic policy gradient algorithm is proposed, which employs neural networks and works in a centralized training and decentralized execution manner to optimize the strategies. Finally, simulation results demonstrate the effectiveness of our proposed algorithm in terms of reducing task failure rate and improving system utility. [ABSTRACT FROM AUTHOR]

When an unmanned aerial vehicle (UAV) performs tasks such as power patrol inspection, water quality detection, field scientific observation, etc., due to the limitations of the computing capacity and battery power, it cannot complete the tasks efficiently. Therefore, an effective method is to deploy edge servers near the UAV. The UAV can offload some of the computationally intensive and real-time tasks to edge servers. In this paper, a mobile edge computing offloading strategy based on reinforcement learning is proposed. Firstly, the Stackelberg game model is introduced to model the UAV and edge nodes in the network, and the utility function is used to calculate the maximization of offloading revenue. Secondly, as the problem is a mixed-integer non-linear programming (MINLP) problem, we introduce the multi-agent deep deterministic policy gradient (MADDPG) to solve it. Finally, the effects of the number of UAVs and the summation of computing resources on the total revenue of the UAVs were simulated through simulation experiments. The experimental results show that compared with other algorithms, the algorithm proposed in this paper can more effectively improve the total benefit of UAVs. [ABSTRACT FROM AUTHOR]

The emerging mobile edge computing (MEC) technology has been recently applied to improve the Quality of Experience (QoE) of network services, such as live video streaming. In this paper, we study an energy-aware adaptive live streaming scheme in wireless edge networks. In particular, we aim to design a joint uplink transmission and edge transcoding algorithm maximizing the video followers’ QoE, while minimizing the energy consumption of the video streamer. We formulate the problem as a Markov decision process (MDP), and propose a deep reinforcement learning (DRL) based framework, named SACCT, to determine the streamer’s encoding bitrate, the uploading power as well as the edge transcoding bitrates and frequency. We decompose the MDP problem into inter-frame and intra-frame problems to address the key design challenges that arise from continuous-discrete hybrid action space, time-varying state and action spaces, and unknown network variation. By doing so, SACCT integrates model-based optimization and model-free DRL to determine the intra-frame continuous resource allocation decisions and the inter-frame discrete bitrate adaptation decisions, respectively. To integrate both the numerical features (e.g., channel gain) and the categorical features (e.g., bitrate), we propose a communication Transformer (CT) as a backbone of SACCT by representing network states as communication tokens and running Transformers to model multi-scale dependencies. Extensive simulations manifest that compared with state-of-the-art approaches, SACCT can provide 128.23% (on average) extra reward. As such, by leveraging joint uplink adaption and edge transcoding, the proposed scheme enables an intelligent wireless network edge with QoE-assured and energy-aware live streaming services. [ABSTRACT FROM AUTHOR]

Adopting Internet of Vehicles (IoV) technology has led to many new uses to improve traffic control, safety, and entertainment services. Still, the increasing amount of information these applications produce poses significant obstacles regarding response time, power usage, and other related issues. To enhance the functionality of IoV systems, this paper introduces a new way that utilizes Deep Reinforcement Learning (DRL) and Mobile Edge Computing (MEC) to improve its performance. Specifically, our DRL framework employs a convolutional neural network-based Asynchronous Advantage Actor-Critic (A3C) algorithm, chosen for its efficacy in processing spatial data relevant to IoV systems such as vehicle locations and speeds. The optimization problem considers the vehicle's location and speed, the MEC server's resources, and the IoV application's requirements by formulating it as a Markov Decision Process (MDP). Utilizing the A3C approach, our Deep Neural Network (DNN) method infers an optimal offloading policy. We optimized the proposed algorithm with strategies that include adaptive learning rate, gradient clipping, entropy regularization, and generalized advantage estimation. The optimized algorithm considers factors such as distance, bandwidth, and communication requirements to provide efficient task-offloading solutions, leading to better system utility and performance. The proposed strategy outperforms comparable models through comprehensive simulations, providing an average enhancement of 20.50% in energy consumption, 15.86% in latency, and 11.94% in execution time, emphasizing the effectiveness and superiority of the suggested algorithm in dealing with various workloads while reducing energy consumption, latency, and execution times. [ABSTRACT FROM AUTHOR]

With the surge in tasks for in-vehicle terminals, the resulting network congestion and time delay cannot meet the service needs of users. Offloading algorithms are introduced to handle vehicular tasks, which will greatly improve the above problems. In this paper, the dependencies of vehicular tasks are represented as directed acyclic graphs, and network slices are integrated within the edge server. The Dynamic Selection Slicing-based Offloading Algorithm for in-vehicle tasks in MEC (DSSO) is proposed. First, a computational offloading model for vehicular tasks is established based on available resources, wireless channel state, and vehicle loading level. Second, the solution of the model is transformed into a Markov decision process, and the combination of the DQN algorithm and Dueling Network from deep reinforcement learning is used to select the appropriate slices and dynamically update the optimal offloading strategy for in-vehicle tasks in the effective interval. Finally, an experimental environment is set up to compare the DSSO algorithm with LOCAL, MINCO, and DJROM, the results show that the system energy consumption of DSSO algorithm resources is reduced by 10.31%, the time latency is decreased by 22.75%, and the ratio of dropped tasks is decreased by 28.71%. [ABSTRACT FROM AUTHOR]

The processing of large volumes of data sets unprecedented demands on the computing power of devices, and it is evident that resource-constrained mobile devices struggle to satisfy the need. As a distributed computing paradigm, edge computing can release mobile devices from computation-intensive tasks, reducing the strain and improving processing efficiency. Traditional offloading methods are less adaptable and do not work in some harsh settings. We simplify the problem to binary offloading decisions in this research and offer a new Asynchronous Update Reinforcement Learning-based Offloading (ARLO) algorithm. The method employs a distributed learning strategy, with five sub-networks and a central public network. Each sub-network has the same structure, as they interact with their environment to learn and update the public network. The sub-network pulls the parameters of the central public network every once in a while. Each sub-network has an experienced pool that minimizes data correlation and is particularly successful in preventing situations where the model falls into a local optimum solution. The main reason for using asynchronous multithreading is that it allows multiple threads to learn the strategy simultaneously, making the learning process faster. At the same time, when the model is trained, five threads can run simultaneously and can handle tasks from different users. The results of simulations show that the algorithm is adaptive and can make optimized offloading decisions on time, even in a time-varying Internet environment, with a significant increase in computational efficiency compared to traditional methods and other reinforcement learning methods. [ABSTRACT FROM AUTHOR]

Artificial intelligence and distributed algorithms have been widely used in mechanical fault diagnosis with the explosive growth of diagnostic data. A novel intelligent fault diagnosis system framework that allows intelligent terminals to offload computational tasks to Mobile edge computing (MEC) servers is provided in this paper, which can effectively address the problems of task processing delays and enhanced computational complexity. As the resources at the MEC and intelligent terminals are limited, performing reasonable resource allocation optimization can improve the performance, especially for a multi-terminals offloading system. In this study, to minimize the task computation delay, we jointly optimize the local content splitting ratio, the transmission/computation power allocation, and the MEC server selection under a dynamic environment with stochastic task arrivals. The challenging dynamic joint optimization problem is formulated as a reinforcement learning (RL) problem, which is designed as the computational offloading policies to minimize the long-term average delay cost. Two deep RL strategies, deep Q-learning network (DQN) and deep deterministic policy gradient (DDPG), are adopted to learn the computational offloading policies adaptively and efficiently. The proposed DQN strategy takes the MEC selection as a unique action while using the convex optimization approach to obtain the local content splitting ratio and the transmission/computation power allocation. Simultaneously, the actions of the DDPG strategy are selected as all dynamic variables, including the local content splitting ratio, the transmission/computation power allocation, and the MEC server selection. Numerical results demonstrate that both proposed strategies perform better than the traditional non-learning schemes. The DDPG strategy outperforms the DQN strategy in all simulation cases exhibiting minimal task computation delay due to its ability to learn all variables online. [ABSTRACT FROM AUTHOR]

In mobile edge computing environment, due to resources constraints of edge devices, when user locations continue changing, the network will be delayed or interrupted, which affects the quality of user's service access. Previous studies have shown that deploying multiple microservice instances with the same function on multiple edge servers through container technology can solve this problem. However, how to choose the optimal microservice instance from multiple servers in a cloud-edge hybrid environment needs to be further investigated. This paper studies the selection of microservices problem based on the dynamic and heterogeneous characters of the cloud-edge collaborative environment, which is defined as a microservice selection and scheduling optimization problem (MSSP) to minimize users' service access delay. To cope with the complexity of cloud-edge collaborative environment and improve learning efficiency, MSSP is regarded as a Markov decision-making process, a Deep Deterministic Policy Gradient algorithm for microservice selection called MS_DDPG is then proposed to solve this problem, and the microservice selection strategy experience pool is established in MS_DDPG. Performance evaluations of MS_DDPG based on a real dataset and some synthetic dataset have been conducted, and the results show that MS_DDPG outperforms the other three baseline algorithms. In terms of average access delay, MS_DDPG is reduced by 23.82%. We also validate the performance of MS_DDPG by increasing the number of user requests, and the results also show that MS_DDPG obtains better performance in scalability. [ABSTRACT FROM AUTHOR]

Distributed base station deployment, limited server resources and dynamically changing end users in mobile edge networks make the design of computing offloading schemes extremely challenging. Considering the advantages of deep reinforcement learning (DRL) in dealing with dynamic complex problems, this paper designs an optimal computing offloading and resource allocation strategy. Firstly, the authors consider a multi‐user mobile edge network scenario consisting of Macro‐cell Base Station (MBS), Small‐cell Base Station (SBS) and multiple terminal devices, the communication overhead and calculation overhead generated are formulated and described in detail. Besides, combined with the deterministic delay of tasks, the optimization objective of this paper is clarified to comprehensively consider system energy consumption. Then, a learning algorithm based on Deep Deterministic Policy Gradient (DDPG) is proposed to minimize system energy consumption. Finally, simulation experiments show that the authors' proposed DDPG algorithm can effectively optimize the target value, and the total system energy consumption is only 15.6 J, which is better than other compared algorithms. It is also proved that the proposed algorithm has excellent communication resource allocation ability. [ABSTRACT FROM AUTHOR]

With the growth of the Internet of Things, smart devices are subsequently generating a large number of computation-intensive and latency-sensitive tasks. Mobile edge computing can provide resources in close proximity, greatly reducing service latency and alleviating congestion in mobile core networks. Due to the instability of the mobile edge computing environment, it was difficult to guarantee the quality of service for users. To address this problem, a hybrid computation offloading framework based on Deep Deterministic Policy Gradient (DDPG) in IoT is proposed. The framework is a system consisting of edge servers and user devices. It is used to acquire the environment state through Software Defined Network technologies and generate the offloading strategy by Deep Deterministic Policy Gradient. The optimization objectives in this paper include the total system overhead of the mobile edge computing system, and considering both network load and computational load, an optimal offloading strategy can be obtained to enable users to obtain a better quality of service. Finally, the experimental results show that the algorithm outperforms the comparison algorithm and can reduce the system latency by 20%, while the network load and computational load are also more stable. [ABSTRACT FROM AUTHOR]

In mobile edge computing, service migration can not only reduce the access latency but also reduce the network costs for users. However, due to bandwidth bottleneck, migration costs should also be considered during service migration. In this way, the trade‐off between benefits of service migration and total service costs is very important for the cloud service providers. In this paper, we propose an efficient dynamic service migration algorithm named SMDQN, which is based on reinforcement learning. We consider each mobile application service can be hosted on one or more edge nodes and each edge node has limited resources. SMDQN takes total delay and migration costs into consideration. And to reduce the size of Markov decision process space, we devise the deep reinforcement learning algorithm to make a fast decision. We implement the algorithm and test the performance and stability of it. The simulation result shows that it can minimize the service costs and adapt well to different mobile access patterns. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) can play an important role in wireless systems as it can be deployed flexibly to help improve coverage and quality of communication. In this paper, we consider a UAV-assisted Mobile Edge Computing (MEC) system, in which a UAV equipped with computing resources can provide offloading services to nearby user equipments (UEs). The UE offloads a portion of the computing tasks to the UAV, while the remaining tasks are locally executed at this UE. Subject to constraints on discrete variables and energy consumption, we aim to minimize the maximum processing delay by jointly optimizing user scheduling, task offloading ratio, UAV flight angle and flight speed. Considering the non-convexity of this problem, the high-dimensional state space and the continuous action space, we propose a computation offloading algorithm based on Deep Deterministic Policy Gradient (DDPG) in Reinforcement Learning (RL). With this algorithm, we can obtain the optimal computation offloading policy in an uncontrollable dynamic environment. Extensive experiments have been conducted, and the results show that the proposed DDPG-based algorithm can quickly converge to the optimum. Meanwhile, our algorithm can achieve a significant improvement in processing delay as compared with baseline algorithms, e.g., Deep Q Network (DQN). [ABSTRACT FROM AUTHOR]

With the internet developing rapidly, mobile edge computing (MEC) has been proposed to offer computational capabilities to tackle the high latency caused by innumerable data and applications. Due to limited computing resources, the innovation of computation offloading technology for an MEC system remains challenging, and can lead to transmission delays and energy consumption. This paper focuses on a task-offloading scheme for an MEC-based system where each mobile device is an independent agent and responsible for making a schedule based on delay-sensitive tasks. Nevertheless, the time-varying network dynamics and the heterogeneous features of real-time data tasks make it difficult to find an optimal solution for task offloading. Existing centralized-based or distributed-based algorithms require huge computational resources for complex problems. To address the above problem, we design a novel deep reinforcement learning (DRL)-based approach by using a parameterized indexed value function for value estimation. Additionally, the task-offloading problem is simulated as a Markov decision process (MDP) and our aim is to reduce the total delay of data processing. Experimental results have shown that our algorithm significantly promotes the users' offloading performance over traditional methods. [ABSTRACT FROM AUTHOR]

In the 6G aerial network, all aerial communication nodes have computing and storage functions and can perform real-time wireless signal processing and resource management. In order to make full use of the computing resources of aerial nodes, this paper studies the mobile edge computing (MEC) system based on aerial base stations (AeBSs), proposes the joint optimization problem of computation the offloading and deployment control of AeBSs for the goals of the lowest task processing delay and energy consumption, and designs a deployment and computation offloading scheme based on federated deep reinforcement learning. Specifically, each low-altitude AeBS agent simultaneously trains two neural networks to handle the generation of the deployment and offloading strategies, respectively, and a high-altitude global node aggregates the local model parameters uploaded by each low-altitude platform. The agents can be trained offline and updated quickly online according to changes in the environment and can quickly generate the optimal deployment and offloading strategies. The simulation results show that our method can achieve good performance in a very short time. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) and device-to-device (D2D) communication can alleviate the resource constraints of mobile devices and reduce communication latency. In this paper, we construct a D2D-MEC framework and study the multi-user cooperative partial offloading and computing resource allocation. We maximize the number of devices under the maximum delay constraints of the application and the limited computing resources. In the considered system, each user can offload its tasks to an edge server and a nearby D2D device. We first formulate the optimization problem as an NP-hard problem and then decouple it into two subproblems. The convex optimization method is used to solve the first subproblem, and the second subproblem is defined as a Markov decision process (MDP). A deep reinforcement learning algorithm based on a deep Q network (DQN) is developed to maximize the amount of tasks that the system can compute. Extensive simulation results demonstrate the effectiveness and superiority of the proposed scheme. [ABSTRACT FROM AUTHOR]

The ubiquitous Internet of Things (IoTs) devices spawn growing mobile services of applications with computationally-intensive and latency-sensitive features, which increases the data traffic sharply. Driven by container technology, microservice is emerged with flexibility and scalability by decomposing one service into several independent lightweight parts. To improve the quality of service (QoS) and alleviate the burden of the core network, caching microservices at the edge of networks empowered by the mobile edge computing (MEC) paradigm is envisioned as a promising approach. However, considering the stochastic retrieval requests of IoT devices and time-varying network topology, it brings challenges for IoT devices to decide the caching node selection and microservice replacement independently without complete information of dynamic environments. In light of this, a MEC-enabled di stributed cooperative m icroservice ca ching scheme, named DIMA, is proposed in this paper. Specifically, the microservice caching problem is modeled as a Markov decision process (MDP) to optimize the fetching delay and hit ratio. Moreover, a distributed double dueling deep Q-network (D3QN) based algorithm is proposed, by integrating double DQN and dueling DQN, to solve the formulated MDP, where each IoT device performs actions independently in a decentralized mode. Finally, extensive experimental results are demonstrated that the DIMA is well-performed and more effective than existing baseline schemes. [ABSTRACT FROM AUTHOR]

In the cooperative vehicle infrastructure system, the road side unit (RSU) equipped with a mobile edge computing (MEC) server and sensors could provide vehicle infrastructure cooperation services for vehicles, such as optimization and cooperative driving, enhanced visibility, and so on. In view of this, the MEC server needs to fuse the sensing information from sensors on the vehicles and RSU, respectively. In the case of bad channel conditions, uploading the raw sensing information from the vehicles results in high uplink transmission latency. To deal with it, the vehicles can process the information locally and just deliver the results to the RSU. However, due to the limited computing resources on the vehicles, the processing accuracy of the raw information on the vehicles is lower than that on the MEC server. Besides, processing locally leads to higher vehicle energy consumption. Thus, in this paper, we aim to jointly optimize execution latency, processing accuracy, and energy consumption of the MEC-based cooperative vehicle infrastructure system. Firstly, we design the terminal machine learning task model and the edge machine learning task model on the vehicle side and RSU side, respectively. Then, we formulate a long-term multi-objective optimization problem. Owing to the stochastic traffic and time-varying communication conditions, we reformulate it as a Markov decision process and propose a two-stage deep reinforcement learning-based offloading and resource allocation (TDORA) strategy to determine the task offloading and the transmit power of each vehicle. Simulation results demonstrate the efficacy of the proposed strategy. [ABSTRACT FROM AUTHOR]

Wireless virtual reality (VR) is predicted to become a killer application in 5G and beyond, which provides an immersive experience and revolutionizes the way people communicate. It is well-known that rendering is the key performance bottleneck in wireless VR systems, especially for VR games. However, real-time rendering and data correlation are ignored by most researchers. In this paper, we propose an adaptive VR framework that enables high-quality wireless VR in future mmWave-enabled wireless networks with mobile edge computing (MEC), where real-time VR rendering tasks can be offloaded to MEC servers adaptively and the caching capability of MEC servers enables further performance improvement. First, we formulate the addressed problem to maximize the quality of experience (QoE) of the users, where association, adaptive offloading mode selection, and caching policy are jointly optimized. Considering the high complexity of the addressed problem, we then propose a distributed learning approach consisting of an offline training phase and an online running phase, which maintains scalability and adaptation capability. The offline phase is based on deep reinforcement learning (DRL) while the latter utilizes game theory. At last, simulation results show the superiority of the proposed algorithm over the other baseline algorithms in terms of QoE utility values, latency, and convergence time. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the scheduling design of a mobile edge computing (MEC) system, where active mobile devices with computation tasks randomly appear in a cell. Every task can be computed at either the mobile device or the MEC server. We jointly optimize the task offloading decision, uplink transmission device selection and power allocation by formulating the problem as an infinite-horizon Markov decision process (MDP). Compared with most of the existing literature, this is the first attempt to address the transmission and computation optimization with random device arrivals in an infinite time horizon to our best knowledge. Due to the uncertainty in the device number and location, the conventional approximate MDP approaches addressing the curse of dimensionality cannot be applied. An alternative and suitable low-complexity solution framework is proposed in this work. We first introduce a baseline scheduling policy, whose value function can be derived analytically with the statistics of random mobile device arrivals. Then, one-step policy iteration is adopted to obtain a sub-optimal scheduling policy whose performance can be bounded analytically. The complexity of deriving the sub-optimal policy is reduced dramatically compared with conventional solutions of MDP by eliminating the complicated value iteration. To address a more general scenario where the statistics of random mobile device arrivals are unknown, a novel and efficient algorithm integrating reinforcement learning and stochastic gradient descent (SGD) is proposed to improve the system performance in an online manner. Simulation results show that the gain of the sub-optimal policy over various benchmarks is significant. [ABSTRACT FROM AUTHOR]

In a high-speed free-flow scenario, a joint optimization scheme for content caching and resource allocation is proposed based on mobile edge computing in Internet of Vehicles. Vehicle trajectory prediction provides the basis for the realization of vehicle-cloud collaborative cache. By pre-caching the business data of requesting vehicles to edge cloud networks and oncoming vehicles, requesting vehicles can obtain data through V2V link and V2I link at the same time, which reduces the data acquisition delay. Therefore, this paper considers the situation where bandwidth of V2I and V2V link and the total amount of edge cloud caches are limited. Then, the bandwidth and cache joint allocation strategy to minimize the weighted average delay of data acquisition is studied. An edge cooperative cache algorithm based on deep deterministic policy gradient is further developed. Different from Q-learning and deep reinforcement learning algorithms, the proposed cache algorithm can be well applied to variable continuous bandwidth allocation action space. Besides, it effectively improves the convergence speed by using interactive iteration of value function and strategy function. Finally, the simulation results of vehicle driving path at the start and stop are obtained by analyzing real traffic data. Simulation results show that the proposed scheme can achieve better performance than several other newer cooperative cache schemes. [ABSTRACT FROM AUTHOR]