Search

Given the rapid growth of user-generated videos, internet traffic has been heavily dominated by online video streaming. Caching videos on edge servers in close proximity to users has been an effective approach to reduce the backbone traffic and the request response time, as well as to improve the video quality on the user side. Video popularity, however, can be highly dynamic over time. The cost of cache replacement at edge servers, particularly that related to service interruption during replacement, is not yet well understood. This paper presents a novel lightweight video caching algorithm for edge servers, seeking to optimize the hit rate with real-time decisions and minimized cost. Inspired by recent advances in deep Q-learning, our DQN-based online video caching (DQN-OVC) makes effective use of the rich and readily available information from users and networks. We decompose the Q-value function as a product of the video value function and the action function, which significantly reduces the state space. We instantiate the action function for cost-aware caching decisions with low complexity so that the cached videos can be updated continuously and instantly with dynamic video popularity. We used video traces from Tencent, one of the largest online video providers in China, to evaluate the performance of our DQN-OVC and to compare it with state-of-the-art solutions. The results demonstrate that DQN-OVC significantly outperforms the baseline algorithms in the edge caching context.

Background. Given that immune-related rash was the most frequently reported PD-1 or PD-L1-related skin toxicity, this systematic review and meta-analysis were conducted to elucidate its incidence risk. Methods. The meta-analysis was carried out according to the PRISMA guidelines. The random effect model was used in the process of all analyses. Skin rash of all grades and grades 3–5 were calculated and gathered in the final comprehensive analyses. Results. The study included 86 clinical trials classified into 15 groups. Compared with chemotherapy, PD-1 or PD-L1 inhibitors significantly strengthened the risk of developing rash across all grades (OR = 1.66, 95% CI: [1.31, 2.11]; p < 0.0001 ). This trend was significantly stronger when the control group was placebo (OR = 2.62, 95% CI: [1.88, 3.65]; p < 0.00001 ). Similar results were observed when PD-1 or PD-L1 inhibitors were given together with chemotherapy (OR = 1.87, 95% CI: [1.59, 2.20]; p < 0.00001 ), even in patients with grades 3–5. As with other combination therapies, the risk of developing rash for all grades was enhanced when PD-1 or PD-L1 was given together with chemotherapy as the second-line option (OR = 2.98, 95% CI: [1.87, 4.75]; p = 0.05 ). No statistically significant differences could be found in skin rash between the PD-1 and PD-L1-related subgroups. Conclusion. Whether PD-1 or PD-L1 inhibitors were given alone or together with others, the risk of developing rash would be enhanced. Furthermore, the risk of developing rash appeared to be higher when PD-1 or PD-L1 inhibitors together with other antitumor drugs were given as the second-line options. No statistically significant results of developing rash between PD-1 and PD-L1 subgroups were obtained owing to the participation of PD-1 or PD-L1 inhibitors.

Ethereum is one of the most popular blockchain systems that support more than half a million transactions every day and foster miscellaneous decentralized applications with its Turing-complete smart contract machine. Whereas it remains mysterious what the transaction pattern of Ethereum is and how it evolves over time. In this article, we study the evolutionary behavior of Ethereum transactions from a temporal graph point of view. We first develop a data analytic platform to collect external transactions associated with users as well as internal transactions initiated by smart contracts. Three types of temporal graphs, user-to-user, contract-to-contract, and user-contract graphs, are constructed according to trading relationships and are segmented with an appropriate time window. We observe a strong correlation between the size of the user-to-user transaction graph and the average Ether price in a time window, while no evidence of such linkage is shown at the average degree, average edge weights, and average triplet closure duration. The macroscopic and microscopic burstiness of Ethereum transactions are validated. We analyze the Gini indexes of the transaction graphs and the user wealth in which Ethereum is found to be very unfair since the very beginning, in a sense, ``the rich is already very rich.''

Scaling blockchain efficiency is crucial to its widespread usage in which the payment channel is one of the most prominent approaches. With payment channels and the network they construct, two users can move some transactions off-chain in a predetermined duration to avoid expensive and time-consuming on-chain settlements. Existing work is devoted to designing high-throughput payment channel networks (PCNs) or efficient PCN routing policies to reduce the fee. In this paper, we investigate the PCN routing from a different perspective by answering whether the routing fee of transactions can be saved by being a bit more patient. The key idea is to reorder the processing sequence of atomic transactions, other than to handle each of them separately and immediately. We present two mechanisms, one is periodic transaction processing assisted by a PCN broker and the other is purely strategic waiting. In the former, all the incoming transactions in a short time interval are processed collectively. We formulate an optimization model to minimize their total routing fee and derive the optimal permutation of processing transactions as well as the routing policy for each of them. A Shapley value based scheme is presented to redistribute the benefit of reordering among the transactions efficiently and fairly. In the latter, we model the waiting time of a strategic transaction on a single payment channel as the first passage time problem in queueing theory when the transaction value is higher than the edge capacity upon its arrival. By capturing the capacity dynamics, we are able to calculate the recursive expression of waiting for time distribution that is useful to gauge a user's cost of patience. Experimental results manifest that our cost redistribution mechanism can effectively save routing fees for all the transactions, and the waiting time distribution coincides with the model well., 11 pages, 12 figures

Packet routing is one of the fundamental problems in computer networks in which a router determines the next-hop of each packet in the queue to get it as quickly as possible to its destination. Reinforcement learning (RL) has been introduced to design autonomous packet routing policies with local information of stochastic packet arrival and service. However, the curse of dimensionality of RL prohibits the more comprehensive representation of dynamic network states, thus limiting its potential benefit. In this article, we propose a novel packet routing framework based on multiagent deep RL (DRL) in which each router possess an independent long short term memory (LSTM) recurrent neural network (RNN) for training and decision making in a fully distributed environment. The LSTM RNN extracts routing features from rich information regarding backlogged packets and past actions, and effectively approximates the value function of Q-learning. We further allow each route to communicate periodically with direct neighbors so that a broader view of network state can be incorporated. The experimental results manifest that our multiagent DRL policy can strike the delicate balance between congestion-aware and shortest routes, and significantly reduce the packet delivery time in general network topologies compared with its counterparts.

The past few years have witnessed a surge of interest towards deep reinforcement learning (DRL) in computer networks. With extraordinary ability of feature extraction, DRL has the potential to re-engineer the fundamental resource allocation problems in networking without relying on pre-programmed models or assumptions about dynamic environments. However, such black-box systems suffer from poor robustness, showing high performance variance and poor tail performance. In this work, we propose a unified Teacher-Student learning framework that harnesses rich domain knowledge to improve robustness. The domain-specific algorithms, less performant but more trustable than DRL, play the role of teachers providing advice at critical states; the student neural network is steered to maximize the expected reward as usual and mimic the teacher’s advice meanwhile. The Teacher-Student method comprises of three modules where the confidence check module locates wrong decisions and risky decisions, the reward shaping module designs a new updating function to stimulate the learning of student network, and the prioritized experience replay module to effectively utilize the advised actions. We further implement our Teacher-Student framework in existing video streaming (Pensieve), load balancing (DeepLB), and TCP congestion control (Aurora). Experimental results manifest that the proposed approach reduces the performance standard deviation of DeepLB by 37%; it improves the 90th, 95th, and 99th tail performance of Pensieve by 7.6%, 8.8%, and 10.7% respectively; and it accelerates the growth rate of Aurora by 2x at the initial stage, and achieves a more stable performance in dynamic environments.