Search

Temporal prefetchers are powerful because they can prefetch irregular sequences of memory accesses, but temporal prefetchers are commercially infeasible because they store large amounts of metadata in DRAM. This paper presents Triage, the first temporal data prefetcher that does not require off-chip metadata. Triage builds on two insights: (1) Metadata are not equally useful, so the less useful metadata need not be saved, and (2) for irregular workloads, it is more profitable to use portions of the LLC to store metadata than data. We also introduce novel schemes to identify useful metadata, to compress metadata, and to determine the fraction of the LLC to dedicate for metadata.

Inappropriate service migrations can lead to undesirable situations, such as high traffic overhead, long service latency, and service disruption. In this paper, we propose an application-aware migration algorithm (AMA) with prefetching. In AMA, a mobile device sends a service offloading request to the controller. After receiving this request, the controller determines the initial service cloud where virtual machine (VM) of the service initially operates by considering the application type. In addition, it periodically decides where to migrate VM and prefetch its core part considering the mobility of the mobile device and application type. To minimize the generated traffic volume while satisfying the requirements of the application, a constraint Markov decision process (CMDP) is formulated and its optimal policy is obtained via linear programming. Evaluation results demonstrate that AMA with the optimal policy can reduce the generated traffic volume while satisfying the requirements of the application (i.e., service latency and probability of service disruption).

Viewport prediction and prefetch have an important influence on VR video streaming performance. This work proposes a novel federated learning-based viewport prediction model training algorithm, ComPer-FedAvg. The proposed algorithm leverages a VR video’s common viewing pattern and users’ personal viewing patterns to train the prediction model in a distributed and privacy-preserving manner. Further, considering the VR video viewport prediction accuracy, a stochastic game is formulated to solve the VR streaming network’s communication resource allocation problem, where limited communication resource blocks are auctioned to users to achieve the optimal overall VR viewing experience. For each user, the auction is decomposed into two disjoint subproblems, namely, the optimal number of data rate requesting and true value claiming (bidding). The optimal true value claiming has been analytically proved to be equal to the VR viewing reward with given data rate. Due to the lack of global information when users request data rate, we reformulate users’ data rate requesting problem as a POMDP problem. A novel deep reinforcement learning algorithm is adopted to solve the problem. Evaluation and simulation results show the proposed viewport prediction and VR streaming schemes outperform conventional solutions in terms of prediction accuracy and VR viewing experience.

Multicasting does not work well for video applications such as video on demand, where individual users request video content at discrete and unpredictable moments. In this paper, we study a concurrent video multicast orchestration solution for the caching-assisted and centrally controlled mobile networks. The proposed solution strives to obtain rapid video prefetch and effective traffic reduction. By rapid video prefetch, user devices can cache required contents in advance, which can effectively improve the streaming fluency in unsteady wireless environments, especially the highly mobile vehicular environment. We introduce a maximum-rate concurrent multicast (MRCM) problem whose solution is intended to maximize the total video delivery rate by a concurrent multicast manner, and prove that the MRCM problem is NP-hard. Then, we propose a dynamic concurrent multicast tree construction algorithm and a global concurrent multicast tree optimization algorithm to approximately solve the MRCM problem in polynomial time. We also introduce a cache competition mechanism that strives to maximize the total cache hit rate when the cache space is insufficient. Performance evaluations show that compared with the existing cache-assisted multicast delivery solutions, the proposed solution significantly reduces the video delivery duration and the traffic stress of backhaul links.

360° video is becoming an increasingly popular technology on commercial social platforms and vital part of emerging Virtual Reality/Augmented Reality (VR/AR) applications. However, the delivery of 360° video content in mobile networks is challenging because of its size. The encoding of 360° video into multiple quality layers and tiles and edge cache-assisted video delivery have been proposed as a remedy to the excess bandwidth requirements of 360° video delivery systems. Existing works using the above tools have shown promising performance for Video-on-Demand (VoD) 360° delivery, but they cannot be straightforwardly extended in a live-streaming setup. Motivated by the above, we study edge cache-assisted 360° live video streaming to increase the overall quality of the delivered 360° videos to users and reduce the service cost. We employ Long Short-Term Memory (LSTM) networks to forecast the evolution of the content requests and prefetch content to caches. To further enhance the delivered video quality, users located in the overlap of the coverage areas of multiple Small Base Stations (SBSs) are allowed to receive data from any of these SBSs. We evaluate and compare the performance of our algorithm with Least Frequently Used (LFU), Least Recently Used (LRU), and First In First Out (FIFO) algorithms. The results show the superiority of the proposed approach in terms of delivered video quality, cache-hit-ratio and backhaul link usage.

Modern multiprocessor systems running multiple applications concurrently exhibit irregular memory access pattern during different phases of execution. The principle of locality is hard to exploit in the presence of such irregular memory requests and may result in additional delays due to resource conflicts throughout memory hierarchy. Prefetching is a promising technique to reduce the memory access latency where data is speculatively fetched ahead of time and stored in a faster memory structure like cache or dedicated prefetch buffer. The emergence of 3D-stacked DRAM provides huge internal bandwidth that makes memory-side prefetching an effective approach to improving system performance. Leveraging the unique architecture of 3D-stacked DRAM, we introduce a memory-side prefetching scheme that works in conjunction with dynamic page mode to reduce memory access latency. We introduce a novel prefetch buffer management scheme that makes intelligent replacement decision based on the utilization and recency of the prefetched data, which also serves as a guidance for future prefetching. Simulation results indicate that our approach improves performance by 21.8 percent on average, compared to a baseline scheme that prefetches a whole row on consecutive hits and implements static open page policy. Our scheme also outperforms an existing memory-side prefetching scheme by 13.2 percent on average, which dynamically adjusts the prefetch degree based on the usefulness of prefetched data.

360-degree video streaming shows great potential to revolutionize the streaming market, by providing much better immersive experience than standard video streams. However, its wide adoption is hindered by the surging demand of network bandwidth due to multi-screen video transmission. To reduce the bandwidth cost, one promising approach is to predict a user’s field of view (FoV), and then prefetch video tiles that a user will view a few seconds ahead. The challenge lies in that user behaviors cannot be properly captured with very limited information, especially the viewing time spent on each tile and the FoV switching behavior are hard to predict. In this paper, we propose a novel 360-degree video streaming algorithm called TVG-Streaming to optimize user experiences by learning user view behaviors. Different from previous approaches, our idea is to exploit tile-view graphs (TVGs) generated by real user behaviors and accurately estimate the probability that each tile falls in the FoV. With the tile view probability, we can determine the bitrate of each tile for delivery and buffering with limited bandwidth budget so as to maximize users’ quality of experience (QoE). For evaluation, we conduct extensive experiments using real traces and the results show that our proposed TVG-Streaming algorithm significantly outperforms other algorithms by at least 20% improvement in terms of users’ QoE.

Hardware data prefetching is a latency hiding technique that mitigates the memory wall problem by fetching data blocks into caches before the processor demands them. For high performing state-of-the-art data prefetchers, this increases dynamic and static energy in memory hierarchy, due to increase in number of requests. A trivial way to improve energy-efficiency of hardware prefetchers is to prefetch instructions on the critical path of execution. As criticality-based data prefetching does not degrade performance significantly; this is an ideal approach to solve the energy-efficiency problem. We discuss limitations of existing critical instruction detection techniques and propose a new technique that uses re-order buffer occupancy as a metric to detect critical instructions and performs prefetcher-specific threshold tuning. With our detector, we achieve maximum memory hierarchy energy savings of 12.3% with 1.4% higher performance, for PPF, and average as follows: (i) SPEC CPU 2017 benchmarks: 2.04% lower energy, 0.3% lower performance, for IPCP at L1D, (ii) client/server benchmarks: 4.7% lower energy, 0.15% lower performance, for PPF, (iii) Cloudsuite benchmarks: 2.99% lower energy, 0.36% higher performance, for IPCP at L1D. IPCP and PPF are state-of-the-art data prefetchers.

This research is to design an effective prefetching method required for hybrid main memory systems consisting of dynamic random-access memory (DRAM) and phase-change memory (PCM) components, which can be especially used for big data applications and massive-scale computing environment. Conventional prefetchers perform adequately for regular memory access patterns. However, graph processing applications show extremely irregular memory access characteristics, causing some difficulty in predicting accurate prefetching operation. Therefore, an effective dynamical prefetching algorithm based on the regression method is proposed in this study. We have designed an intelligent prefetch engine that can identify any dynamic accessing characteristics in memory accessing sequences. Specifically, it can select regular, linear, or polynomial regression predictive analysis based on the memory access sequence characteristics, and also dynamically determine the number of pages required for any selected prefetching. We also present a DRAM-PCM hybrid memory structure that can reduce the energy consumption and resolve the thermal issue that hampers conventional DRAM memory systems. Experimental results indicate that the performance can increase by around 40%, compared to that of conventional DRAM memory structures.

Emerging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this article, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51 percent and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81 percent in average (up to 34.28 percent), and power savings of 35.07 percent over a standard 3D-stacked memory.

The 360-degree video streaming has higher bandwidth requirements compared with traditional video to achieve the same user-perceived playback quality. Since users only view part of the entire videos, viewport-adaptive streaming is an effective approach to guarantee video quality. However, the performance of viewport-adaptive schemes is highly dependent on the bandwidth estimation and viewport prediction. To overcome these issues, we propose a novel reinforcement learning (RL) based viewport-adaptive streaming framework called RLVA, which optimizes the 360-degree video streaming in viewport prediction, prefetch scheduling and rate adaptation. Firstly, RLVA adopts ${t}$ location-scale distribution rather than Gaussian distribution to describe the viewport prediction error characteristic more accurately and achieve the tile viewing probability based on the distribution. Besides, a tile prefetch scheduling algorithm is proposed to update the tiles according to the latest prediction results, which further reduces the adverse effect of prediction error. Furthermore, the tile viewing probabilities are treated as input status of RL algorithm. In this way, RL can adjust its policy to adapt to both of the network conditions and viewport prediction error. Through extensive evaluations, the simulation results show that the proposed RLVA outperforms other viewport-adaptive methods by about 4.8%-66.8% improvement of Quality of Experience (QoE) and effectively reduces the impact of viewport prediction errors.

Docker, as a mainstream container solution, adopts the Copy-on-Write (CoW) mechanism in its storage drivers. This mechanism satisfies the need of different containers to share the same image. However, when a single container performs operations such as modification of an image file, a duplicate is created in the upper read-write layer, which contributes to the runtime overhead. When the accessed image file is fairly large, this additional overhead becomes non-negligible. Here we present the concept of Dynamic Prefetching Strategy Optimization (DPSO), which optimizes the CoW mechanism for a Docker container on the basis of the dynamic prefetching strategy. At the beginning of the container life cycle, DPSO pre-copies up the image files that are most likely to be copied up later to eliminate the overhead caused by performing this operation during application runtime. The experimental results show that DPSO has an average prefetch accuracy of greater than 78% in complex scenarios and could effectively eliminate the overhead caused by the CoW mechanism.

This research is to design a history table-based linear analysis method for a DRAM-PCM (phase change memory) hybrid memory system supporting irregular and complex memory accessing patterns in graph processing applications. This linear analysis method can analyze complex memory access patterns in real time through our history table structure, classifying address values requested from the LLC (last-level cache) as the prefetch candidates. Also, the proposed method is to support a prefetching operation from the storage onto the hybrid main memory, causing significant latency losses. Specifically, we proposed a new memory pattern analysis method, which is called a linear analysis based on a history table, to detect any linear memory access patterns from workloads showing irregular accessing characteristics. For the DRAM-PCM hybrid memory, dynamic data that are frequently accessed are stored in the DRAM. Thus, our linear analysis not only categorizes memory access patterns in the form of line patterns but also separates dynamic and static data so that the hybrid memory systems can be managed adaptively and efficiently. Our experiments show that our model can improve performance and energy consumption by 60% and 30%, respectively, compared to those of the traditional DRAM-only memory model. Moreover, our approach can enhance the entire system performance and energy consumption by 11.3%, 6.8%, compared to streamlined prefetcher-based models.

With large volumes of data being generated in recent years and the inception of big data analytics on social media necessitates accurate user query processing with minimum time complexity. Several research works have been conducted in this area, to address accuracy and time complexity involved in query processing, in this work, Wald Adaptive Prefetched Boosting Classification based Czekanowski Similarity MapReduce (WAPBC–CSMR) technique is introduced. The WAPBC–CSMR technique uses the big dataset for processing large number of user queries. First, a technique called, Wald Adaptive Prefetched Boosting is employed with the objective of classifying the big dataset into different classes. To reduce the time involved in classification, in this paper a classifier called Gaussian distributive Rocchio is used that achieves significant classification in minimum time. With the classified results, a Likelihood Radio Test is applied to integrate the weak learner results into strong classification results. Then the classified and refined data are stored on the prefetcher cache. Upon reception of multi-dimensional user queries by the prefetch manager, the queries are now split into multiple keywords and are fed into the map phase, where mapping function is performed using Czekanowski Similarity Index with the objective of identifying the repeated jobs with maximum query processing accuracy. Followed by which the relevant data are retrieved from the prefetcher cache and repeated user query task is removed in the reduce phase via statistical function, therefore contributing to minimum time. Result analysis of WAPBC–CSMR is performed with big dataset using different metrics such as query processing accuracy, error rate and processing time for varied number of user queries. The result shows that WAPBC–CSMR technique enhances query processing accuracy and lessens the time as well as the error rate than the conventional methods.

Data movement between the CPU and main memory is a first-order obstacle against improv ing performance, scalability, and energy efficiency in modern systems. Computer systems employ a range of techniques to reduce overheads tied to data movement, spanning from traditional mechanisms (e.g., deep multi-level cache hierarch ies, aggressive hardware prefetcher s) to emerging techniques such as Near-Data Processing (NDP), where some computation is moved close to memory. Prior NDP works investigate the root causes of data movement bottlenecks using different profiling methodologies and tools. However, there is still a lack of understanding about the key metrics that can identify different data movement bottlenecks and their relation to traditional and emerging data movement mitigation mechanisms. Our goal is to methodically identify potential sources of data movement over a broad set of applications and to comprehensively compare traditional compute-centric data movement mitigation techniques (e.g., cach ing and prefetch ing) to more memory-centric techniques (e.g., NDP), thereby developing a rigorous understanding of the best techniques to mitigate each source of data movement. With this goal in mind, we perform the first large-scale characterization of a wide variety of applications, across a wide range of application domains, to identify fundamental program properties that lead to data movement to/from main memory. We develop the first systematic methodology to classify applications based on the sources contributing to data movement bottlenecks. From our large-scale characterization of 77K functions across 345 applications, we select 144 functions to form the first open-source benchmark suite (DAMOV) for main memory data movement studies. We select a diverse range of functions that (1) represent different types of data movement bottlenecks, and (2) come from a wide range of application domains. Using NDP as a case study, we identify new insights about the different data movement bottlenecks and use these insights to determine the most suitable data movement mitigation mechanism for a particular application. We open-source DAMOV and the complete source code for our new characterization methodology at https://github.com/CMU-SAFARI/DAMOV., IEEE Access, 9, ISSN:2169-3536

Caching is one of the most promising areas in the field of future internet architecture like Information-centric Networking, Software Defined Networking, and IoT. In Web caching, most of the web content is readily available across the network, even if the webserver is not reachable. Several existing traditional caching methods and cache replacement strategies are evaluated based on the metrics like hit ratio and byte hit Ratio. However, these metrics have not been improved over the period because of the traditional caching policies. So, in this paper, we have used an intelligent function like locally normalized intervals of page visit, website duration, users’ interest between user groups is proposed. These intervals are combined with multiple distance metrics like Manhattan, squared Euclidean, and 3-,4-,5-norm Minkowski. In order to obtain significant common user navigation patterns, the clustering relation between the users using different intervals and distances is thoroughly analyzed. These patterns are successfully coupled with greedy web cache replacement strategies to improve the efficiency of the proposed web cache system. Particularly for improving the caching metrics more, we used an AI-based intelligent approach like Random Forest classifier to boost the prefetch buffer performance and achieves the maximum hit rate of 0.89, 0.90, and byte hit rate of 0.87, 0.89 for Greedy Dual Size Frequency and Weighted Greedy Dual Size Frequency algorithms, respectively. Our experiments show good hit/byte hit rates than the frequently used algorithms like least recently used and least frequently used.

In today’s information-driven world, data access latency accounts for the expensive part of processing user requests. One potential solution to access latency is prefetching, a technique to speculate and move future requests closer to the processing unit. However, the block access requests received by the storage device show poor spatial locality because most file-related locality is absorbed in the higher layers of the memory hierarchy, including the CPU cache and main memory. Besides, the utilization of multithreading results in an interleaved access request making prefetching at the storage level more picky using existing prefetching techniques. Toward this, we propose and assess DeepPrefetcher, a novel deep neural network inspired context-aware prefetching method that adapts to arbitrary memory access patterns. DeepPrefetcher learns the block access pattern contexts using distributed representation and leverage long short-term memory learning model for context-aware data prefetching. Instead of using the logical block address (LBA) value directly, we model the difference between successive access requests, which contains more patterns than LBA value for modeling. By targeting access pattern sequence in this manner, the DeepPrefetcher can learn the vital context from a long input LBA sequence and learn to predict both the previously seen and unseen access patterns. The experimental result reveals that DeepPrefetcher can increase an average prefetch accuracy, coverage, and speedup by 21.5%, 19.5%, and 17.2%, respectively, contrasted with the baseline prefetching strategies. Overall, the proposed prefetching approach surpasses other schemes in all benchmarks, and the outcomes are promising.

Since Windows Vista, Microsoft has offered us a new life companion called SysMain or Superfetch from its old name. This is a service which analyzes and records the user daily software use to increase the speed of his or her experience on the operating system. However, this service provides the opportunity to track software used and private files seen such as movies or confidential files, reveal his or her lifetime activities and map directories. More than just a privacy issue, this constitutes a reliable approach in forensic analysis. Furthermore, this service is often misunderstood due to its little documentation and myths surrounding it, which makes things soon complicated to investigate. This paper is an extended version of the talk presented at Black Hat USA 2020: it aims at debunking partial and fake news about SysMain and its files. This paper will examine in detail its architecture, analyze its mechanisms and explain its operating method. It will detail the format of all the prefetch files which has been undocumented or obsolete so far. In addition, this paper will illustrate forensic concrete cases in which SysMain turns out to be useful.

In order to enable large-scale molecular simulations, algorithms must efficiently utilize multi-core processors that continue to increase in total core count over time with relatively stagnant clock speeds. Although parallelized molecular dynamics (MD) software has taken advantage of this trend in computer hardware, single-particle perturbations with Monte Carlo (MC) are more difficult to parallelize than system-wide updates in MD using domain decomposition. Instead, prefetching reconstructs the serial Markov chain after computing multiple MC trials in parallel. Canonical ensemble MC simulations of a Lennard-Jones fluid with prefetching resulted in up to a factor of 1.7 speedup using 2 threads, and a factor of 3 speedup using 4 threads. Strategies for maximizing efficiency of prefetching simulations are discussed, including the potentially counter-intuitive benefit of reduced acceptance probabilities. Determination of the optimal acceptance probability for a parallel simulation is simplified by theoretical prediction from serial simulation data. Finally, complete open-source code for parallel prefetch simulations was made available in the Free Energy and Advance Sampling Simulation Toolkit (FEASST).

Mobile edge caching (MEC) and device-to-device (D2D) communications are two potential technologies to resolve traffic overload problems in the Internet of Things. Previous works usually investigate them separately with MEC for traffic offloading and D2D for information transmission. In this article, a joint framework consisting of MEC and cache-enabled D2D communications is proposed to minimize the energy cost of systematic traffic transmission, where file popularity and user preference are the critical criteria for small base stations (SBSs) and user devices, respectively. Under this framework, we propose a novel caching strategy, where the Markov decision process is applied to model the requesting behaviors. A novel scheme based on reinforcement learning (RL) is proposed to reveal the popularity of files as well as users’ preference. In particular, a $Q$ -learning algorithm and a deep $Q$ -network algorithm are, respectively, applied to user devices and the SBS due to different complexities of status. To save the energy cost of systematic traffic transmission, users acquire partial traffic through D2D communications based on the cached contents and user distribution. Taking the memory limits, D2D available files, and status changing into consideration, the proposed RL algorithm enables user devices and the SBS to prefetch the optimal files while learning, which can reduce the energy cost significantly. Simulation results demonstrate the superior energy saving performance of the proposed RL-based algorithm over other existing methods under various conditions.

With the development of 5G mobile networks, people’s demand for network response speed and services has increased to meet the needs of a large amount of data traffic, reduce the backhaul load caused by frequently requesting the same data (or content). The file is pre-stored in the base station by the edge device, and the user can directly obtain the requested data in the local cache without remotely. However, changes in popularity are difficult to capture, and data is updated more frequently through the backhaul. In order to reduce the number of backhauls and provide caching services for users with specific needs, we can provide proactive caching with users without affecting user activity. We propose a content caching strategy based on mobility prediction and joint user prefetching (MPJUP). The policy predicts the prefetching device data by predicting the user’s movement position with respect to time by the mobility of the user and then splits the partial cache space for prefetching data based on the user experience gain. Besides, we propose to reduce the backhaul load by reducing the number of content backhauls by cooperating prefetch data between the user and the edge cache device. Experimental analysis shows that our method further reduces the average delay and backhaul load, and the prefetch method is also suitable for more extensive networks.

Recent research revisits pairwise-correlating data prefetching due to its extremely low overhead. Pairwise-correlating data prefetching, however, cannot accurately detect where data streams end. As a result, pairwise-correlating data prefetchers either expose low accuracy or they lose timeliness when they are performing multi-degree prefetching. In this letter, we propose a novel technique to detect where data streams end and hence, control the multi-degree prefetching in the context of pairwise-correlated prefetchers. The key idea is to have a separate metadata table that operates one step ahead of the main metadata table. This way, the runahead metadata table harnesses the degree of prefetching by allowing/disallowing the main metadata table to issue prefetch requests. We evaluate our proposal in the context of a four-core chip multiprocessor and show that it significantly reduces erroneous prefetches, providing up to 16.1 percent performance improvement on top of a state-of-the-art pairwise-correlating prefetcher.

With the increasing complexity of graph structures, the current real-world large-scale graphs are being represented by a considerable amount of vertex and edge data. Furthermore, the analysis of a large number of computing nodes has become a very complicated job that requires a large amount of hardware resources. Moreover, in large-scale graph processing, the vertex and edge data show random and sequential memory access patterns at the same time, and this is a major bottleneck in graph processing. In this paper, we present a high-capacity main memory system with an intelligent pattern-aware prefetching engine to overcome the scalability problem and the memory inefficiency of single-machine graph processing. The proposed intelligent pattern-aware prefetching engine is designed to predict and handle sequential or regular patterns and random-access patterns simultaneously. Experimental results demonstrated that the proposed model exhibited performance improvements of 60% over conventional DRAM models, approximately 40% over the existing prefetch models, and about 12.5% over the latest prefetch models.

Prefetching instructions is a fundamental technique for designing high-performance computers. There are three key properties to consider when designing an efficient and effective prefetcher: timeliness, coverage, and accuracy. Timeliness is an essential property, as bringing instructions too early increases the risk of the instructions being evicted from the cache before their use while requesting them too late can lead to the instructions arriving past their designated execution time. Coverage is important to reduce the number of instruction cache misses (there is enough prefetching), and accuracy to ensure that the prefetcher does not pollute the cache or interacts negatively with the other hardware mechanisms (there is not too much prefetching). This letter presents the Entangling instruction prefetcher that entangles instructions to provide timeliness. The prefetcher works by finding which instruction should trigger the prefetch for a subsequent instruction, accounting for the latency of each cache miss. The prefetcher is carefully adjusted to account for both coverage and accuracy. Our evaluation shows that the Entangling I-prefetcher increases performance by 29.3 percent on average, with a coverage of 94.9 percent and accuracy of 77.4 percent.

Internet dan teknologi web telah menyediakan sarana yang dapat diandalkan untuk berbagi data dan aplikasi. Namun, latensi pemuatan data masih menjadi kendala. Dalam beberapa literatur, metode cache terbukti meningkatkan waktu penyajian halaman web dengan mengurangi proses permintaan pada database. Namun, metode ini tidak mencegah web mengakses server untuk mengambil data cache dalam database, latensi dalam memuat data masih terjadi. Makalah ini menyajikan metode prefetch dengan mengubah data menjadi file data yang diformat JSON sebagai cache, data diperoleh dengan mudah tanpa harus melakukan proses permintaan. Hasil pengujian menunjukkan bahwa aplikasi web menjadi lebih responsif dalam menyajikan data.

This research proposes a memory-disk integrated system (MDIS) with pattern analysis based data management method. The proposed system consists of a pattern adaptive prefetcher with DRAM-based dual buffers and a PCM-based persistent memory structure. The PCM-based persistent memory module is exploited as conventional main memory and storage layers simultaneously. In this system, to compensate the PCMs unfavorable characteristics, a pattern analysis based prefetching method by using dual DRAM buffers with a small amount of space are designed as a sort of caching layer. The proposed pattern analysis based data management method monitors the miss rate to determine the prefetch policy, e.g., when and what to prefetch. These schemes enable flexible adaptation to the application execution characteristics showing unpredictable and inconsistent workload patterns. In order to evaluate our system, we implemented a trace-driven simulator system and launched the YCSB on Redis, Apache Storm, Apache Spark, and OpenStack Swift. The experimental result shows that the MDIS with a pattern adaptive prefetcher can reduce the total access time by 13.9% compared to the conventional scheme and thus the overall performance of the memory-disk integrated system improves.

The advancement of the Internet of Things has enabled numerous applications ranging from smart wearable to connected vehicles. However, the limited energy and memory resources of low-end IoT devices significantly impede their further flourish. In this article, we consider a system that consists of an IoT device and an edge server. The edge server stores code blocks for the IoT device and loads required blocks to the IoT device for execution thereby alleviates the latter from the limited memory resource. Furthermore, the IoT device can harvest energy from the ambient energy sources to achieve a sustainable operation. To deal with the dynamic energy harvesting process and block request process, we propose a stochastic block prefetching framework (BPF) to optimize the user experienced delay. The BPF assists the IoT device to intelligently prefetch blocks from the edge server according to the historical user behaviors. The BPF consists of three modules, i.e., estimation module, prefetching module, and dual learning module. The estimation module measures the probability of block being requested in the future. The prefetching module requests blocks from the edge server according to the available energy and memory. The dual learning module helps to accelerate the convergence of the framework. The numerous simulation results are provided to verify the effectiveness of the proposed framework.

In partially run-time reconfigurable (PRR) FPGAs, hardware tasks should be configured before their execution. The configuration delay imposed by the reconfiguration process increases the total execution time of the hardware tasks and task graphs. In this paper, a new technique named forefront-fetch is presented to improve the makespan of hardware task graphs running on PRR FPGAs via alleviating the adverse effects of the configuration delays. In this technique, which is applied to a sequence of task graphs, the configuration of some tasks is carried out within the execution phase of the previous task graph. This strategy leads to hide the configuration delay of the forefront-fetched tasks that as a result improves the execution time. The proposed solution modifies the schedules of the task graphs at design time to obtain a set of schedule pairs for the run-time environment. Experiments on actual and synthesized task graphs demonstrate the ability of the proposed technique in improving the makespan of hardware task graphs. The obtained results show that for a set of task graphs running on Xilinx™ Virtex-5 XUPV5LX110T FPGA, makespan is improved by 37.81% on average. Moreover, the proposed solution outperforms state-of-the-art prefetch-aware scheduling strategies by 14.78% makespan improvement.

As the demand for embedded-vision grows, solving large optimization problems in real-time with energy and cost budget is a challenge. We present BAX, a hardware accelerator of bundle adjustment (BA), which solves the least-squares problem of state estimation in visual odometry (VO). BAX consists of a frontend and a backend for control and computation, respectively. The frontend generates instructions on-the-fly executed at the backend to perform the BA algorithm. The backend adopts decoupled access/execute (DAE) architecture, which separates the memory access unit (MAU) from the pipeline. The MAU can prefetch vectors and matrices ahead of computations. To further reduce the latency of data reorganization, three transpose-free dataflows are proposed for matrix multiplication operations on the vector processing unit (VPU). Besides, a unified architecture for both forward and backward substitution is designed for matrix decomposition in the linear solver. All the data are stored in 442kB on-chip memory, and the local map is maintained efficiently by the hierarchical graph memory. Compared with the baseline architecture, the processing time is reduced by 53.9% through the above techniques. BAX is implemented in 32-bit floating-point precision with data normalization on FPGA. It completes a full BA in about 63.44ms at 200MHz, consuming 1.12W power. BAX is $1.73\times $ and $22.38\times $ faster than the desktop and embedded CPUs, respectively, and achieves 90% performance of the GPU at much less power consumption.

Cloud and fog computing along with network function virtualization technology have significantly shifted the development of network architectures. They yield in reduced capital and operating expenditures, while achieving network flexibility and scalability to accommodate the massive growth in data traffic volumes from user terminals requesting various services and applications. Now cloud solutions here offer abundant computing and storage resources, at the detriment of high end-to-end delays, hence limiting quality of service for delay-sensitive applications. Meanwhile, fog solutions offer reduced delays, at the detriment of limited resources. Existing efforts focus on merging the two solutions and propose multi-tier hybrid fog-cloud architectures to leverage their both saliencies. However, these approaches can be inefficient when the applications are delay-sensitive and require high resources. Hence this work proposes a novel standalone heterogeneous fog architecture that is composed of high-capacity and low-capacity fog nodes, both located at the terminals proximity. Thereby, realizing a substrate network that offers reduced delays and high resources, without the need to relay to the cloud nodes. Moreover, the work here leverages and deploys a deep learning network to propose a service function chain provisioning scheme implemented on this architecture. The scheme predicts the popular network functions, and maps them on the high-capacity nodes, whereas it predicts the unpopular network functions and maps them on the low-capacity nodes. The goal is to predict the next incoming function and prefetch it on the node. Hence, when a future request demands the same function, it can be cached directly from the node, at reduced resources consumption, processing times, cost, and energy consumption. Also, this yields in higher number of satisfied requests and increased capacity. The deep learning network yields reduced loss model and high success rates.

Data prefetching at fog nodes that significantly reduces latency is critical for video streaming applications. In the framework of dynamic video streaming over fog, some segments (parts of video) can be prefetched by the network provider. Sharing the dynamic information about prefetched node locations to all of the users is not a scalable approach, due to signalling and performance overheads. However, at the time of downloading, the client devices can search for the availability of DASH video segments in the fog nodes present in their vicinity. There is, however, a dilemma of how many fog nodes can be queried, without affecting performance such as continuous playback of the video. \par In this paper, we propose an efficient mechanism for searching video segments over different fog nodes. This search mechanism is formulated as a sequential stochastic modeling framework known as Multi-Armed Bandit~(MAB). While the state space of this model is a countably infinite set, we propose an algorithmic approach to transform to a finite state space model without loss of optimality. With extensive simulation results, the analytical results are validated. We study the different parameters influencing the improved optimal DASH performance, in terms of segment prefetch probability, and number of search attempts made by the client.

Past research has proposed numerous hardware prefetching techniques, most of which rely on exploiting one specific type of program context information (e.g., program counter, cacheline address, or delta between cacheline addresses) to predict future memory accesses. These techniques either completely neglect a prefetcher’s undesirable effects (e.g., memory bandwidth usage) on the overall system, or incorporate system-level feedback as an afterthought to a system-unaware prefetch algorithm. We show that prior prefetchers often lose their performance benefit over a wide range of workloads and system configurations due to their inherent inability to take multiple different types of program context and system-level feedback information into account while prefetching. In this paper, we make a case for designing a holistic prefetch algorithm that learns to prefetch using multiple different types of program context and system-level feedback information inherent to its design. To this end, we propose Pythia, which formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to make a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth usage. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and system-aware prefetch requests in the future. Our extensive evaluations using simulation and hardware synthesis show that Pythia outperforms two state-of-the-art prefetchers (MLOP and Bingo) by 3.4% and 3.8% in single-core, 7.7% and 9.6% in twelve-core, and 16.9% and 20.2% in bandwidth-constrained core configurations, while incurring only 1.03% area overhead over a desktop-class processor and no software changes in workloads. The source code of Pythia can be freely downloaded from https://github.com/CMU-SAFARI/Pythia.

Deep learning recommendation models (DLRMs) are typically composed of two sets of parameters: large embedding tables to handle sparse categorical inputs, and neural networks such as multi-layer perceptrons (MLPs) to handle dense non-categorical inputs. Current DLRM training practices keep both these parameters in GPU memory. But as the size of the embedding tables grow, this practice of storing model parameters in GPU memory requires dozens or even hundreds of GPUs. This is an unsustainable trend with severe environmental consequences. Furthermore, such a design forces only a few conglomerates to be the gate keepers of model training. In this work, we propose cDLRM which democratizes recommendation model training by allowing a user to train on a single GPU regardless of the size of embedding tables by storing all embedding tables in CPU memory. A CPU based pre-processor analyzes training batches to prefetch embedding table slices accessed by those batches and caches them in GPU memory just-in-time. An associated caching protocol on the GPU enables efficiently updating the cached embedding table parameters. cDLRM decouples the embedding table size demands from the number of GPUs needed for compute. We first demonstrate that with cDLRM it is possible to train a large recommendation model using a single GPU regardless of model size. We then demonstrate that with its unique caching strategy, cDLRM enables pure data parallel training. We use two publicly available datasets to show that a cDLRM achieves identical model accuracy compared to a baseline trained completely on GPUs, while benefiting from large reduction in GPU demand.

This study about data prefetching focuses on maximizing the performance of modern processors by hiding cache misses. This paper suggests that improving prefetch coverage is an effective approach to achieve the goal. This work proposes to employ two simple buffers, block offset buffer and block address buffer, that leverage prefetch coverage. The offset buffer stores the offset of cache blocks that are accessed recently, while the block address buffer stores the address of cache blocks that are prefetch-issued lately. For the offset buffer, we propose to adopt multiple lengths of delta history in searching offset patterns, compared to using a single length of delta history in global history buffer method. Each offset buffer is used to find a local optimum offset of a page. Besides, all the offset buffers are employed to refer to each other, mining offset patterns from others. Our second proposed buffer, the block address buffer, is hired for aggressive prefetching that can switch to adaptive burst mode occurred in the case of successive accesses to the same page. Our Buffer-referred data Prefetcher (BRP) improves an IPC speedup of 23.2% over a no-prefetching baseline. BRP outperforms the state-of-the-art Signature Path prefetcher by an IPC speedup of 4.2%.

Microarchitectural side channel attacks have been very prominent in security research over the last few years. Caches proved to be an outstanding side channel, as they provide high resolution and generic cross-core leakage. All major cryptographic libraries provide countermeasures to hinder key extraction via cross-core cache attacks by now. In this paper, we analyze implementations protected by prefetch-based countermeasures aimed at preventing well-known cache attacks, and highlight the circumstances causing them to remain vulnerable. Further, we craft a novel attack technique that precisely synchronizes the attacking and the victim processes, enabling the attacking process to evict the target data from the cache at the desired instants. One key improvement of our approach is that it provides unprivileged attackers with a method to remove specific data from the cache with a single memory access and in absence of shared memory by leveraging the transient capabilities of TSX and relying on the L3 replacement policy. We show the feasibility of our approach by extracting an RSA key from the latest wolfSSL library and an AES key from the T-Table and S-Box implementations included in OpenSSL with CacheSniper. Both libraries implement prefetch-based methods as a protection against cache attacks.

We describe a new method of game streaming designed to address the limitations of video-based cloud gaming and graphics API command streaming approaches. The method is implemented as a game-engine protocol and plug-in called GPEG (Geometry Pump Engine Group). GPEG streams the game engine content as sub-assets, not video of gameplay or graphics API commands. GPEG encoding uses unique pre-processing algorithms which intelligently subdivide textured mesh assets into much smaller sub-assets based on their potential geometric and perceptual visibility. A thin, CPU-only, server software process interactively streams the pre-encoded sub-asset packets to the client-side game engine using navigation-driven, visibility-based prefetch. Like command streaming, the method does not require a GPU on the server. By using predictive prefetch, the method overcomes network latency. In contrast to progressive download, the method is a true content stream, with adaptive scalability and intelligent caching that together enable interactive game engine content streaming over broadband.

In Recently, social media networks are popularly emerging through world. This has been a great platform for information sharing through network among people. Being ubiquitous in nature, social media are accessible anywhere and at any point of time. To provide Quality of Experience support, a learning-based model for social media is been proposed. This is mainly used to improve user's usage satisfaction and to reduce access delay in Online Social Networks (OSN). For this, an analysis of Twitter traces for over fourteen months is conducted. Over 2,800 users Twitter data is collected using Twitter API for the analysis of social media friendship. And a cluster-based analysis for these set of user friends is made for the prefetch prediction. A mechanism using XGBoost algorithm and Decision Tree Regressor algorithm, the performance of this framework is been determined. The performance of this framework using trace-drive emulations on the social media network has been evaluated. Evaluation results could do superior performance with XG Boost algorithm of around 85.1% accuracy of access delay reduction.

Conventional tile-based 360° video streaming methods, including deep reinforcement learning (DRL) based, ignore the interactive nature of 360° video streaming and download tiles following fixed sequential orders, thus failing to respond to the user's head motion changes. We show that these existing solutions suffer from either the prefetch accuracy or the playback stability drop. Furthermore, these methods are constrained to serve only one fixed streaming preference, causing extra training overhead and the lack of generalization on unseen preferences. In this paper, we propose a dual-queue streaming framework, with accuracy and stability purposes respectively, to enable the DRL agent to determine and change the tile download order without incurring overhead. We also design a preference-aware DRL algorithm to incentivize the agent to learn preference-dependent ABR decisions efficiently. Compared with state-of-the-art DRL baselines, our method not only significantly improves the streaming quality, e.g., increasing the average streaming quality by 13.6% on a public dataset, but also demonstrates better performance and generalization under dynamic preferences, e.g., an average quality improvement of 19.9% on unseen preferences.

Graphics Processing Units (GPUs) employ large register files to accommodate all active threads and accelerate context switching. Unfortunately, register files are a scalability bottleneck for future GPUs due to long access latency, high power consumption, and large silicon area provisioning. Prior work proposes hierarchical register file to reduce the register file power consumption by caching registers in a smaller register file cache. Unfortunately, this approach does not improve register access latency due to the low hit rate in the register file cache. In this article, we propose the Latency-Tolerant Register File (LTRF) architecture to achieve low latency in a two-level hierarchical structure while keeping power consumption low. We observe that compile-time interval analysis enables us to divide GPU program execution into intervals with an accurate estimate of a warp’s aggregate register working-set within each interval. The key idea of LTRF is to prefetch the estimated register working-set from the main register file to the register file cache under software control, at the beginning of each interval, and overlap the prefetch latency with the execution of other warps. We observe that register bank conflicts while prefetching the registers could greatly reduce the effectiveness of LTRF. Therefore, we devise a compile-time register renumbering technique to reduce the likelihood of register bank conflicts. Our experimental results show that LTRF enables high-capacity yet long-latency main GPU register files, paving the way for various optimizations. As an example optimization, we implement the main register file with emerging high-density high-latency memory technologies, enabling 8× larger capacity and improving overall GPU performance by 34%.

The introduction of unified memory and demand paging has simplified programming of graphics processing units (GPUs). It has also enabled oversubscribing the memory for a GPU. However, the overhead of page management makes page faults a performance bottleneck. Sometimes the page eviction policy is unable to mitigate performance slowdown caused by page faults and memory oversubscription. On average, eviction policies such as Random and CAR are not competitive with a traditional least recently used (LRU) policy. Other policies, such as CLOCK-Pro, are designed to overcome challenges with LRU, but they only achieve limited speedup. Even enhancing LRU with page walk hit information does not lead to notable performance improvement. Based on these observations, we propose optimization opportunities to mitigate performance degradation caused by page faults and memory oversubscription. These optimization opportunities include an effective page eviction policy that retains LRU’s advantages while addressing LRU’s inability to deal with thrashing access patterns, page prefetch and pre-eviction, memory-aware throttling, and capacity compression.

Memory prefetchers are designed to identify and prefetch specific access patterns, including spatiotemporal locality (e.g., strides, streams), recurring patterns (e.g., varying strides, temporal correlation), and specific irregular patterns (e.g., pointer chasing, index dereferencing). However, existing prefetchers can only target premeditated patterns and relations they were designed to handle and are unable to capture access patterns in which they do not specialize. In this article, we propose a context-based neural network (NN) prefetcher that dynamically adapts to arbitrary memory access patterns. Leveraging recent advances in machine learning, the proposed NN prefetcher correlates program and machine contextual information with memory accesses patterns, using online-training to identify and dynamically adapt to unique access patterns exhibited by the code. By targeting semantic locality in this manner, the prefetcher can discern the useful context attributes and learn to predict previously undetected access patterns, even within noisy memory access streams. We further present an architectural implementation of our NN prefetcher, explore its power, energy, and area limitations, and propose several optimizations. We evaluate the neural network prefetcher over SPEC2006, Graph500, and several microbenchmarks and show that the prefetcher can deliver an average speedup of 21.3% for SPEC2006 (up to 2.3×) and up to 4.4× on kernels over a baseline of PC-based stride prefetcher and 30% for SPEC2006 over a baseline with no prefetching.

In-memory techniques keep data into faster and more expensive storage media for improving performance of big data processing. However, existing mechanisms do not consider how to expedite the data processing applications that access the input datasets only once. Another problem is how to reclaim memory without affecting other running applications. In this paper, we provide scheduling-aware data prefetching and eviction mechanisms based on Spark, Alluxio, and Hadoop. The mechanisms prefetch data and release memory resources based on the scheduling information. A mathematical method is proposed for maximizing the reduction of data access time. To make the mechanisms applicable in large-scale environments, we propose a heuristic algorithm to reduce the computational time. Furthermore, an enhanced version of the heuristic algorithm is also proposed to increase the amount of prefetched data. Finally, we perform real-testbed and simulation experiments to show the effectiveness of the proposed mechanisms.

We present an overview of SAP HANA's Native Store Extension (NSE). This extension substantially increases database capacity, allowing to scale far beyond available system memory. NSE is based on a hybrid in-memory and paged column store architecture composed from data access primitives. These primitives enable the processing of hybrid columns using the same algorithms optimized for traditional HANA's in-memory columns. Using only three key primitives, we fabricated byte-compatible counterparts for complex memory resident data structures (e.g. dictionary and hash-index), compressed schemes (e.g. sparse and run-length encoding), and exotic data types (e.g. geo-spatial). We developed a new buffer cache which optimizes the management of paged resources by smart strategies sensitive to page type and access patterns. The buffer cache integrates with HANA's new execution engine that issues pipelined prefetch requests to improve disk access patterns. A novel load unit configuration, along with a unified persistence format, allows the hybrid column store to dynamically switch between in-memory and paged data access to balance performance and storage economy according to application demands while reducing Total Cost of Ownership (TCO). A new partitioning scheme supports load unit specification at table, partition, and column level. Finally, a new advisor recommends optimal load unit configurations. Our experiments illustrate the performance and memory footprint improvements on typical customer scenarios.

Modern embedded systems are packed with dedicated field-programmable gate arrays (FPGAs) to accelerate the overall system performance. But the main drawback in using FPGA as a reconfigurable system is that a lot of reconfiguration overheads are generated in the reconfiguration process. The reconfiguration overheads are mainly because of the configuration data being fetched from the off-chip memory and also due to the improper management of tasks during execution. This work focusses mainly on the prefetch heuristics, reuse technique, and the available memory hierarchy to provide an efficient management of tasks over the available resources. This short communication proposes a new optimal replacement policy which reduces the overall time and energy reconfiguration overheads for static systems in their subsequent iterations. It is evident from the results that most of the time and energy reconfiguration overheads are eliminated.

This paper proposes a new architecture, called Adaptive PREfetching and Scheduling (APRES), which improves cache efficiency of GPUs. APRES relies on the observation that GPU loads tend to have either high locality or strided access patterns across warps. APRES schedules warps so that as many cache hits are generated as possible before the generation of any cache miss. Without directly predicting future cache hits/misses for each warp, APRES creates a warp group that will execute the same static load shortly and prioritizes the grouped warps. If the first executed warp in the group hits the cache, grouped warps are likely to access the same cache lines. Unless, APRES considers the load as a strided type and generates prefetch requests for the grouped warps. In addition, APRES includes a new dynamic L1 prefetch and data cache partitioning to reduce contentions between demand-fetched and prefetched lines. In our evaluation, APRES achieves 27.8 percent performance improvement.

360° video on-demand streaming is a key component of the emerging virtual reality and augmented reality applications. In such applications, sending the entire 360° video demands extremely high network bandwidth that may not be affordable by today’s networks. On the other hand, sending only the predicted user’s field of view (FoV) is not viable as it is hard to achieve perfect FoV prediction in on-demand streaming, where it is better to prefetch the video multiple seconds ahead, to absorb the network bandwidth fluctuation. This paper proposes a two-tier solution, where the base tier delivers the entire 360° span at a lower quality with a long prefetching buffer, and the enhancement tier delivers the predicted FoV at a higher quality using a short buffer. The base tier provides robustness to both network bandwidth variations and FoV prediction errors. The enhancement tier improves the video quality if it is delivered in time and FoV prediction is accurate. We study the optimal rate allocation between the two tiers and buffer provisioning for the enhancement tier to achieve the optimal trade-off between video quality and streaming robustness. We also design periodic and adaptive optimization frameworks to adapt to the bandwidth variations and FoV prediction errors in realtime. Through simulations driven by real LTE and WiGig network bandwidth traces and user FoV traces, we demonstrate that the proposed two-tier systems can achieve a high-level of quality-of-experience in the face of network bandwidth and user FoV dynamics.

Fault tolerance has become increasingly critical for virtualized systems as growing amount of mission-critical applications are now deployed on virtual machines rather than directly on physical machines. However, prior hardware-based fault-tolerant systems require extensive modification to existing hardware, which makes them infeasible for industry practitioners. Although software-based techniques realize fault tolerance without any hardware modification, they suffer from significant latency overhead that is often orders of magnitude higher than acceptable. To realize practical low-latency fault tolerance in the virtualized environment, we first identify two bottlenecks in prior approaches, namely the overhead for tracking dirty pages in software and the long sequential dependency in checkpointing system states. To address these bottlenecks, we design a novel mechanism to asynchronously prefetch the dirty pages without disrupting the primary VM execution to shorten the sequential dependency. We then develop Phantasy, a system that leverages page-modification logging (PML) technology available on commodity processors to reduce the dirty page tracking overhead and asynchronously prefetches dirty pages through direct remote memory access via RDMA. Evaluated on 25 real-world applications, we demonstrate Phantasy can significantly reduce the performance overhead by 38 percent on average, and further reduce the latency by 85 percent compared to a state-of-the-art virtualization-based fault-tolerant system.