Search

Nowadays, embedded systems are provided with cache memories that are large enough to influence in both performance and energy consumption as never occurred before in this kind of systems. In addition, the cache memory system has been identified as a component that improves those metrics by adapting its configuration according to the memory access patterns of the applications being run. However, given that cache memories have many parameters which may be set to a high number of different values, designers are faced with a wide and time-consuming exploration space. In this paper, we propose an optimization framework based on Grammatical Evolution (GE) which is able to efficiently find the best cache configurations for a given set of benchmark applications. This metaheuristic allows an important reduction of the optimization runtime obtaining good results in a low number of generations. Besides, this reduction is also increased due to the efficient storage of evaluated caches. Moreover, we selected GE because the plasticity of the grammar eases the creation of phenotypes that form the call to the cache simulator required for the evaluation of the different configurations. Experimental results for the Mediabench suite show that our proposal is able to find cache configurations that obtain an average improvement of 62 % versus a real world baseline configuration.

Wireless Mesh Networking (WMN) is the latest Internet framework that provides a comprehensive range for Subsequent Internet (SI) prototype. Despite significant advantages provided by WMN, its practical distribution to connect Internet of Things (IoT) networks caused exorbitant congestion and restricted bandwidth. Motivated by this, a novel mechanism that ensures control in the manifestation of mobbing, end-to-end delay, energy consumption for enhancing the network performance of IoT-enabled WMN is presented. The proposed method is called as an Integrated Markov State Transition and Open Loop Smart Caching (MST-OLSC) for congestion control in IoT-enabled WMN. The proposed method uses Markov state transition scheduling model to differentiate the states of the incoming data packets from the host computer. This is performed by applying the State Betweenness centrality. Next, Congestion Control Token Caching mechanism is applied with the objective of controlling the congestion by means of caching via overflow with well-balanced isolation between regulated and unregulated flow of data packet. Finally, Open Loop Smart Caching is presented to ensure constant data rate, thereby providing fair inflow and outflow between the incoming and outgoing data packets. The evaluation results of MST-OLSC ensure higher network performance with minimum end-to-end delay, energy consumption and higher packet delivery rate is achieved with respect to inflated IoT nodes in WMN.

As the recommendation systems behind commercial services scale up and apply more and more sophisticated machine learning models, it becomes important to optimize computational cost (capacity) and runtime latency, besides the traditional objective of user engagement. Caching recommended results and reusing them later is a common technique used to reduce capacity and latency. However, the standard caching approach negatively impacts user engagement. To overcome the challenge, this paper presents an approach to optimizing capacity, latency and engagement simultaneously. We propose a smart caching system including a lightweight adjuster model to refresh the cached ranking scores, achieving significant capacity savings without impacting ranking quality. To further optimize latency, we introduce a prefetching strategy which leverages the smart cache. Our production deployment on Facebook Marketplace demonstrates that the approach reduces capacity demand by 50% and p75 end-to-end latency by 35%. While Facebook Marketplace is used as a case study, the approach is applicable to other industrial recommendation systems as well.

We consider a basic caching system, where a single server with a database of N files (e.g. movies) is connected to a set of K users through a shared bottleneck link. Each user has a local cache memory with a size of M files. The system operates in two phases: a placement phase, where each cache memory is populated up to its size from the database, and a following delivery phase, where each user requests a file from the database, and the server is responsible for delivering the requested contents. The objective is to design the two phases to minimize the load (peak or average) of the bottleneck link. We characterize the rate-memory tradeoff of the above caching system within a factor of 2.00884 for both the peak rate and the average rate (under uniform file popularity), where the best proved characterization in the current literature gives a factor of 4 and 4.7 respectively. Moreover, in the practically important case where the number of files (N) is large, we exactly characterize the tradeoff for systems with no more than 5 users, and characterize the tradeoff within a factor of 2 otherwise. We establish these results by developing novel information theoretic outer-bounds for the caching problem, which improves the state of the art and gives tight characterization in various cases.

The integration of caching and ultra-dense network (UDN) can not only improve the efficiency of content retrieval by reducing duplicate content transmissions but also improve the network throughput and system energy efficiency (EE) of the UDN. In this paper, we focus on energy consumption aspects of cache-aided UDN (CUDN) and develop a novel caching strategy to improve the system EE. Different from the existing researches, we consider a more realistic scenario where the popularity of the cache content is dynamic and unknown. The proposed caching strategy is a deep reinforcement learning (DRL)-based approach that uses a deep Q-network to approximate the Q action-value function. We optimize the structure and corresponding parameters of the deep Q-network according to the latest findings in the field of DRL and deep learning (DL) to improve the performance of this caching strategy. The simulation results show that the performance in terms of EE of the CUDN can be significantly improved by using our proposed caching strategy.

This thesis evaluates an innovative cache design called, prime-mapped cache. The performance analysis on various application s and programs shows that the primemapped cache performs better than the conventional cache organizations. The performance gain will increase with the increase of the speed gap between processors and memories. The exact cache behavior of numerical applications namely: matrix multiplication and SPEC benchmarks is st udied by varying the cache parameters such as cachesize, linesize and associativi ty. Traces are collected from these programs and miss ratios for instructions and data accesses are compared. Based on the experimental results and depending on the algorithm used, the miss ratios of the prime-mapped cache are found to be 50 to 100% less than for conventional caches. Depending upon the speed difference between processors and memories, with the prime-mapped cache these algorithms can run 30% to 2 times faster than they do on conventional caches.

This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the Q value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%–64% improved in terms of the cache hit ratio and 56%–59% decreased in terms of the average latency.

The demand for higher and higher wireless data rates is driven by the popularity of mobile video content delivery through wireless devices such as tablets and smartphones. To achieve unprecedented mobile content delivery speeds while reducing backhaul cost and delay, in this paper we propose a new system architecture that combines two recent ideas, distributed caching of content in small cells (FemtoCaching), and, cooperative transmissions from nearby base stations (Coordinated Multi-Point). A key characteristic of the proposed architecture is the interdependence between the caching strategy and the physical layer coordination. Specifically, the caching strategy may cache different content in nearby base stations (BSs) to maximize the cache hit ratio, or cache the same content in multiple nearby BSs such that the corresponding BSs can transmit concurrently, e.g., to multiple users using zero-forcing beamforming, and achieve multiplexing gains. Such interdependency allows a joint cross-layer optimization. Given the popularity distribution of the content, the available cache size, and the network topology, we devise near-optimal strategies of caching such that the system throughput is maximized or the system delay is minimized. Under realistic scenarios and assumptions, our analytical and simulation results show that our system yields significantly faster content delivery, which can be one order of magnitude faster than that of legacy systems.

Cache replacement policies and cache partitioning are well-known cache management techniques which aim to eliminate inter- and intra-application contention caused by co-running applications, respectively. Since replacement policies can change applications’ behavior on a shared last-level cache, they have a massive impact on cache partitioning. Furthermore, cache partitioning determines the capacity allocated to each application affecting incorporated replacement policy. However, their interoperability has not been thoroughly explored. Since existing cache partitioning methods are tailored to specific replacement policies to reduce overheads for characterization of applications’ behavior, they may lead to suboptimal partitioning results when incorporated with the up-to-date replacement policies. In cache partitioning, miss curve estimation is a key component to relax this restriction which can reflect the dependency between a replacement policy and cache partitioning on partitioning decision. To tackle this issue, we propose a replacement policy adaptable miss curve estimation (RME) which estimates dynamic workload patterns according to any arbitrary replacement policy and to given applications with low overhead. In addition, RME considers asymmetry of miss latency by miss type, thus the impact of miss curve on cache partitioning can be reflected more accurately. The experimental results support the efficiency of RME and show that RME-based cache partitioning cooperated with high-performance replacement policies can minimize both inter- and intra-application interference successfully.

Reuse distance is an important metric for analytical estimation of cache miss rate. To find the miss rate of a particular cache, the reuse distance profile has to be measured for that particular level and configuration of the cache. Significant amount of simulation time and overhead can be reduced if we can find the miss rate of higher level cache like L2 cache from the RD profile with respect to a lower level cache (i.e., cache that is closer to the processor) such as L1. The objective of this paper is to give an analytical method to find the miss rate of L2 cache for various configurations from the RD profile with respect to L1 cache. We consider all three types of cache inclusion policies namely (i) Strictly Inclusive, (ii) Mutually Exclusive and (iii) Non-Inclusive Non-Exclusive policy. We first prove some general results relating the RD profile of L1 cache to that of L2 cache. We use probabilistic analysis for our derivations. We validate our model against simulations, using the multi-core simulator Sniper with the PARSEC and the SPLASH benchmark suites.

The world is more connected than it ever has been before, and with our pocket supercomputers and IoT (Internet of Things) future, the next generation of the web might just be delivered in a peer-to...

Energy becomes an inevitable challenge when using a large die-stacking dynamic random access memory (DRAM) cache. Although emerging spin-transfer-torque-RAM (STT-RAM) technology can efficiently reduce the static energy of large cache, it cannot completely replace DRAM cache due to the high write energy of STT-RAM. Recently, researchers have observed that there are many redundant bits written in the row buffer and futile bits written back to STT-RAM cells, which do not change the cells’ value but still cost high write energy. In this paper, we first design a large hybrid cache architecture with the DRAM region and the STT-RAM region to reduce the high static energy of DRAM cache. The selective write back to row buffer and selective write back to cell array optimizations are proposed to reduce high write energy of the STT-RAM region by removing the unnecessary bit-writes. Furthermore, we propose reuse distance-oriented data movement to reduce write operations in the STT-RAM region. Finally, we propose a novel tag design for the hybrid cache by moving all tag arrays to the STT-RAM region. The SPEC CPU2006 benchmarks show an average 28.3% energy reduction and 6.7% performance improvement for the write optimizations and 7.3% energy savings and 27.5% instructions per cycle speedups for the proposed tag design.

In recent years, researches focus on addressing the query bottleneck issue using semantic cache. However, the challenges of this method are how to increase cache hit ratio, decrease the query processing time, and address cache consistency issue. In this paper, we construct segment access-aware dynamic semantic cache for relational databases. Some definitions of semantic segment, probe query, and remainder query are proposed to describe the semantic cache. Then, estimation of the query result is proposed. Next, cache access algorithm of our proposed segment access-aware dynamic semantic cache is presented in case of cache exact hit, cache extended hit, cache partial hit and cache miss. Cache item with effective lifecycle tag is proposed to address cache consistency issue. Finally, experimental results show that this approach performs better than regular semantic cache and decisional semantic cache.

In this paper, we evaluate compressibility of L1 data caches and L2 cache in general-purpose graphics processing units (GPGPUs). Our proposed scheme is geared toward improving performance and power of GPGPUs through cache compression. GPGPUs are throughput-oriented devices which execute thousands of threads simultaneously. To handle working set of this massive number of threads, modern GPGPUs exploit several levels of caches. GPGPU design trend shows that the size of caches continues to grow to support even more thread level parallelism. We propose using cache compression to increase effective cache capacity, improve performance, and reduce power consumption in GPGPUs. Our work is motivated by the observation that the values within a cache block are similar, i.e., the arithmetic difference of two successive values within a cache block is small. To reduce data redundancy in L1 data caches and L2 cache, we use low-cost and implementation-efficient base-delta-immediate (BDI) algorithm. BDI replaces a cache block with a base and an array of deltas where the combined size of the base and deltas is less than the original cache block. We also study locality of fields in integer and floating-point numbers. We found that entropy of fields varies across different data types. Based on entropy, we offer different BDI compression schemes for integer and floating-point numbers. We augment a simple, yet effective, predictor that determines type of values dynamically in hardware and without the help of a compiler or a programmer. Evaluation results show that on average, cache compression improves performance by 8% and saves energy of caches by 9%.

The aim of the hierarchical cache memories that are equipped for GPUs is the management of irregular memory access patterns for general purpose workloads. The level-1 data cache (L1D) of the GPU plays an important role for its ability in the provision of high bandwidth and low-latency data accesses. Unfortunately, the GPU L1D may become a performance bottleneck due to facing many performance challenges such as cache contention and resource congestion. These critical issues come from a large number of simultaneous requests from the SIMT cores to the limited-capacity L1D. We observe that many applications have a large number of requests with a very low reuse probability, resulting in the GPU performance degradation. To overcome these challenges, we propose an efficient cache bypassing mechanism that can periodically filter the access stream and make an accurate bypassing decision to improve the efficiency of the L1D. The proposed technique uses a small storage amount to save the tag array of the L1D for the early miss prediction before it makes the bypassing decision. The experiment results reveal that the proposed technique significantly increases the cache efficiency and the GPU performance.

For cache analytical modeling, the stack distance theory is widely utilized to predict LRU-cache behaviors. Typically, the stack distance histogram collecting is implemented by profiling memory references. However, the profiled memory references merely reflect instruction fetching and load/store executions, which only represent the memory accesses to first-level (L1) caches. That is why these traces cannot be applied directly to construct stack distance histograms for downstream (L2 and L3) caches. Therefore, this paper proposes a stack distance probability model to extend the stack distance theory to the multi-level LRU cache behavior predictions. The inputs of our model are the L1 cache stack distance histograms and the multi-level LRU cache configurations. The outputs are the L2 and L3 cache stack distance histograms, with which the conflict misses in L2 and L3 caches can be quantified quickly and precisely. 15 benchmarks chosen from Mobybench 2.0, Mibench I and Mediabench II are used to evaluate the accuracy of our model. Compared to the simulation results from Gem5 in AtomicSimpleCPU mode, the average absolute error of predicting cache misses in the I/D shared L2 cache is less than 5% while that of estimating the L3 cache misses is less than 7%. Furthermore, contrast to the time overhead of Gem5 AtomicSimpleCPU simulations, our model can speed up the cache miss prediction about x100 on average.

With the rapid growth of Internet services in smart cities, large network flow has caused network congestion among users, thereby deteriorating network service quality and user experience. In this study, a distributed proxy cache management platform based on autonomous decision is proposed. The platform realizes highly efficient automatic cache management by taking the advantages of autonomous perception and decision. First, the platform establishes the autonomous management framework of proxy cache on the basis of the idea of feedback loop. Next, the URL is distributed evenly among cache nodes through the hash ring storage mechanism with virtual nodes. Cache status adjustment strategies are made based on historical status sequences and the gray prediction model. A method that is used to predict content hot-rank based on domain name set is proposed to reduce loads of cache nodes. Second, the platform designs a distributed proxy cache management system by depending on the autonomous management framework of proxy cache and the cache management mechanism of autonomous decision, and the Cache Hot Spots Migrate algorithm is suggested for the dynamic migration and integration of virtual nodes on the hash ring. A system knowledge base is established through status sequence and corresponding adjustment strategies. Finally, experimental results on the hash ring, content hot-rank prediction, and adjustment of autonomous decision of cache status are analyzed, thereby confirming that the distributed proxy cache management mechanism based on autonomous decision could manage cache nodes effectively and automatically, and could improve the overall performance of the system.

Flash memory-based SSD-RAIDs are swiftly replacing conventional hard disk drives by exhibiting improved performance and stability, especially in I/O-intensive environments. However, the variations in latency and throughput occurring due to uncoordinated internal garbage collection cripples further boosting of performance. In addition, the unwanted variations in each SSD can influence the overall performance of the entire flash storage adversely. This performance bottleneck can be essentially reduced by an internal write cache in the RAID controller designed prudently by considering the crucial device characteristics. The state-of-the-art cache write for the RAID controller fails to incorporate device characteristics of flash memory-based SSDs and mitigates the performance gain. In this paper, we propose a novel cache design namely Layout-Aware Write Cache (LAWC) to overcome the performance barrier inculcated by independent garbage collections. LAWC implements (i) improved I/O scheduling for logically partitioned write caches, (ii) a destage write synchronization mechanism to allow individual write caches to flush write blocks into the SSD array in a coordinated manner, and (iii) a two-level hybrid cache algorithm utilizing small front level cache for the improved write cache efficiency. LAWC shows significant reduction in response time by 82.39 percent on RAID-0 and 68.51 percent on RAID-5 types of SSDs when compared with state-of-the-art write cache algorithms.

In wireless mobile Ad Hoc networks, cooperative cache management is considered as an efficient technique to increase data availability and improve access latency. This technique is based on coordination and sharing of cached data between nodes belonging to the same area. In this paper, we studied the cooperative cache management strategies. This has enabled us to propose a collaborative cache management scheme for mobile Ad Hoc networks, based on service cache providers (SCP), called cooperative caching based on service providers (CCSP). The proposed scheme enabled the election of some SCPs mobile nodes, which receive cache’s summaries of neighboring nodes. Thus, nodes belonging to the same zone can locate easily cached documents of that area. The election mechanism used in this approach is executed periodically to ensure load balancing. We further provided an evaluation of the proposed solution, in terms of request hit rate, byte hit rate and time gains. Compared with other caching management schemes, the simulation results show that the proposed CCSP scheme improves significantly the cache effectiveness and the network performances. This is achieved by improving data availability and reducing both overall network load and latencies perceived by end users.

Named-data Networking (NDN) is a new networking regime for efficient content distribution. NDN employs name-based access and in-network caching to implement easy access to content. For that, NDN routers perform named-based routing, packet-level caching, and cache look-up, but these operations impose huge overhead on routers. To solve the feasibility issue of NDN router, in this paper, we propose an alternative architecture where caching and routing are separated. Routers are dedicated to packet routing while the cache server that is installed at every domain takes charge of in-network caching. Requests that have visited the cache server are marked and forwarded without further cache-lookup within the domain. This approach fundamentally relieves routers of heavy burden. Additionally, it introduces following benefits: (1) Huge amount of resource for caching is saved; (2) The effectiveness of in-network caching is improved; (3) The access delay of time-sensitive traffic is minimized. For the proposed architecture, cache look-up is performed via reserved links between the cache server and routers. Therefore, it is important to locate the cache server from the point of installation cost. We address how to place the cache server with the minimum cost, which is proved NP-complete, and present an approximation algorithm. Our simulation study shows that the proposed scheme reduces caching operations by up to 80% while achieving the same caching benefit only with 40% caching space.

To improve the performance of mobile video delivery, caching layered videos at a site near to mobile end users (e.g., at the edge of mobile service provider's backbone) was advocated because cached videos can be delivered to mobile users with a high quality of experience, e.g., a short latency. How to optimally cache layered videos based on caching price, the available capacity of cache nodes, and the social features of mobile users, however, is still a challenging issue. In this paper, we propose a novel edge caching scheme to cache layered videos. First, a framework to cache layered videos is presented in which a cache node stores layered videos for multiple social groups, formed by mobile users based on their requests. Due to the limited capacity of the cache node, these social groups compete with each other for the number of layers they request to cache, aiming at maximizing their utilities while all mobile users in each group share the cost involved in the cache of video contents. Second, a Stackelberg game model is developed to study the interaction among multiple social groups and the cache node, and a noncooperative game model is introduced to analyze the competition among mobile users in different social groups. Third, leveraging the backward induction method, the optimal strategy of each player in the game model is proposed. Finally, simulation results show that the proposed method outperforms the exiting counterparts with a higher hit ratio and lower delay of delivering video contents.

Cache has long been used to minimize the latency of main memory accesses by storing frequently used data near the processor. Processor performance depends on the underlying cache performance. Therefore, significant research has been done to identify the most crucial metrics of cache performance. Although the majority of research focuses on measuring cache hit rates and data movement as the primary cache performance metrics, cache utilization is significantly important. We investigate the application’s locality using cache utilization metrics. Furthermore, we present cache utilization and traditional cache performance metrics as the program progresses providing detailed insights into the dynamic application behavior on parallel applications from four benchmark suites running on multiple cores. We explore cache utilization for APEX, Mantevo, NAS, and PARSEC, mostly scientific benchmark suites. Our results indicate that 40% of the data bytes in a cache line are accessed at least once before line eviction. Also, on average a byte is accessed two times before the cache line is evicted for these applications. Moreover, we present runtime cache utilization, as well as, conventional performance metrics that illustrate a holistic understanding of cache behavior. To facilitate this research, we build a memory simulator incorporated into the Structural Simulation Toolkit (Rodrigues et al. in SIGMETRICS Perform Eval Rev 38(4):37–42, 2011). Our results suggest that variable cache line size can result in better performance and can also conserve power.

In this paper, we present a distributed multi-level cache system based on cloud storage, which is aimed at the low access efficiency of small spatio-temporal data files in information service system of Smart City. Taking classification attribute of small spatio-temporal data files in Smart City as the basis of cache content selection, the cache system adopts different cache pool management strategies in different levels of cache. The results of experiment in prototype system indicate that multi-level cache in this paper effectively increases the access bandwidth of small spatio- temporal files in Smart City and greatly improves service quality of multiple concurrent access in system.

Data-center servers require large capacity main memory to run multiple workloads simultaneously. However, the scalability and power consumption of DRAM limit its capability of constructing large capacity memory. Emerging non-volatile memories (e.g. PCM and STT-RAM) provide better scalability and lower power leakage than DRAM. Especially, hybrid memory consisting of DRAM and NVM is able to exploit advantages of different memory medias. However, NVMs have a few drawbacks, such as relatively longer read and write latency. Cache miss at the shared last level cache (LLC) suffers from longer latency if the missing data resides in NVM. Current LLC policies manage the cache space without being aware of the underlying heterogeneous medias. This results in cache performance degradation if a large number of missing data come from NVM. Taking the asymmetric cache miss cost into account, we first propose a new performance metric -TMPKI, which can exactly reflect the LLC performance on the top of hybrid memories. Then we propose a hybrid memory aware cache partitioning technique (HAP) to dynamically adjust the cache spaces for DRAM and NVM data based on TMPKI. Experimental results show that HAP improves performance against the traditional LRU policy by up to 54.3% (19.6% on average) while it incurs a little storage overhead (0.2%).

Caching is an effective optimization in search engine storage architectures. Many caching algorithms have been proposed to improve retrieval performance. The data selection policy of search engine cache management plays an important role, which carefully places the data in memory or other storage, such as solid state disks (SSDs). Considering that the historical query log has a guiding role for the future query, we present an Efficient Data Selection (EDS) policy for search engine cache management, which views cache media as a knapsack, and views results and posting lists as items. The best benefit of EDS can be computed by greedy algorithms. We carry out a series of experiments to study the essential factors of the data selection in different architectures, including hard disk drive (HDD), SSD, and SSD-based hybrid storage architectures. The hybrid storage architecture is a two-level cache architecture, which uses SSD as a secondary cache for the memory. Our main goal is to improve the performance of the search engines and reduce the cost of the servers on two-level cache architecture. The experimental results demonstrate that our proposed policy improves the hit ratio by 20.04% as well as the retrieval performance on HDD, SSD, and hybrid architecture by 31.98%, 28.72% and 23.24%, respectively.

Key-value store accelerators based on field-programmable gate arrays (FPGAs) have been proposed to achieve higher performance per watt than software-based processing. However, because their cache capacity is strictly limited by DRAMs implemented on FPGA boards, their application domains are also limited. To address this issue, the authors propose a multilevel NoSQL cache architecture that utilizes both FPGA-based hardware cache and in-kernel software cache in a complementary style. This motivates them to explore various design options. Simulation results show that their design reduces the cache miss ratio and improves the throughput compared to the nonhierarchical design.

Classic caching algorithms leverage recency, access count, and/or other properties of cached blocks at per-block granularity. However, for media such as flash which have performance and wear penalties for small overwrites, implementing cache policies at a larger granularity is beneficial. Recent research has focused on buffering small blocks and writing in large granularities, sometimes called containers, but it has not explored the ramifications and best strategies for caching compound blocks consisting of logically distinct, but physically co-located, blocks. Containers may have highly diverse blocks, with mixtures of frequently accessed, infrequently accessed, and invalidated blocks. We propose and evaluate Pannier, a flash cache layer that provides high performance while extending flash lifespan. Pannier uses three main techniques: (1) leveraging block access counts to manage cache containers, (2) incorporating block liveness as a property to improve flash cache space efficiency, and (3) designing a multi-step feedback controller to ensure a flash cache reaches its desired lifespan while maintaining performance. Our evaluation shows that Pannier improves flash cache performance and extends lifespan beyond previous per-block and container-aware caching policies. More fundamentally, our investigation highlights the importance of creating new policies for caching compound blocks in flash.

Mobile devices are quickly becoming the most widely used processors in consumer devices. Since their major power supply is battery, energy-efficient computing is highly desired. In this article, we focus on energy-efficient cache design in emerging mobile platforms. We observe that more than 40% of L2 cache accesses are OS kernel accesses in interactive smartphone applications. Such frequent kernel accesses cause serious interferences between the user and kernel blocks in the L2 cache, leading to unnecessary block replacements and high L2 cache miss rate. We first propose to statically partition the L2 cache into two separate segments, which can be accessed only by the user code and kernel code, respectively. Meanwhile, the overall size of the two segments is shrunk, which reduces the energy consumption while still maintaining the similar cache miss rate. We then find completely different access behaviors between the two separated kernel and user segments and explore the multi-retention STT-RAM-based user and kernel segments to obtain higher energy savings in this static partition-based cache design. Finally, we propose to dynamically partition the L2 cache into the user and kernel segments to minimize overall cache size. We also integrate the short-retention STT-RAM into this dynamic partition-based cache design for maximal energy savings. The experimental results show that our static technique reduces cache energy consumption by 75% with 2% performance loss, and our dynamic technique further shows strong capability to reduce cache energy consumption by 85% with only 3% performance loss.

In recently emerged Named Data Networking (NDN) technology, each router has a cache to store and provide frequently requested contents. Hence, an efficient cache management scheme is essential for effective content distribution and high cache utilization. As a new cache management scheme, this paper proposes the sharing of cache summaries with neighboring routers to increase the diversity of cached contents in NDN. Our proposed scheme defines a summary packet using a Bloom filter and a method to share the summary. When a Data packet is received, a router makes the decision of whether or not to save the Data depending on query results of the cache summaries of neighboring routers. If any of the neighboring routers have the Data, the Data has not necessarily been saved. When an Interest packet is received, a router can forward the Interest to a neighboring router that has the requested content by checking the summaries. The proposed scheme is evaluated using ndnSIM, which is an NS-3 based Named Data Networking simulator. Simulation results show that the sharing of cache summaries enhances the content diversity and accordingly the cache hit ratio, and reduces the average content delivery time.

To the best of our knowledge, our work is therst in literature to apply machine learning techniques to the result caching problem in search engines, for both static, dynamic, and state-of-the-art static-dynamic cache organizations.We evaluate a large set of features and illustrate that they can be exploited to increase the hit rate of result caches.We evaluate various oracle caching strategies to illustrate the potential room for improvement in the result caching problem.We show that the proposed machine learning framework can improve the hit rate of result caches, potentially reducing the energy consumption in search engines. A commonly used technique for improving search engine performance is result caching. In result caching, precomputed results (e.g., URLs and snippets of best matching pages) of certain queries are stored in a fast-access storage. The future occurrences of a query whose results are already stored in the cache can be directly served by the result cache, eliminating the need to process the query using costly computing resources. Although other performance metrics are possible, the main performance metric for evaluating the success of a result cache is hit rate. In this work, we present a machine learning approach to improve the hit rate of a result cache by facilitating a large number of features extracted from search engine query logs. We then apply the proposed machine learning approach to static, dynamic, and static-dynamic caching. Compared to the previous methods in the literature, the proposed approach improves the hit rate of the result cache up to 0.66%, which corresponds to 9.60% of the potential room for improvement.

Advancement in semiconductor technology increases power density in recent Chip Multi-Processors (CMPs) which significantly increases the leakage energy consumptions of on-chip Last Level Caches (LLCs). Performance linked dynamic tuning in LLC size is a promising option for reducing the cache leakage. This paper reduces static power consumption by dynamically shutting down or turning on cache banks based upon system performance and cache bank usage statistics. Shutting down of a cache bank remaps its future requests to another active bank, called as target bank. The proposed method is evaluated on three different implementation policies, viz (1) The system can decide to shutdown or turn-on some cache banks periodically throughout the process execution. (2) The system allows to shutdown banks initially and once the bank restarting initiates, no more shutdown is permitted further. (3) This policy resizes cache like first policy with some predefined time slices, in which cache cannot be resized. For a 4MB 4 way set associative L2 cache, experimental analysis shows 66% reduction in static energy with 29% gain in Energy Delay Product (EDP) for first strategy; for the second policy, static power is reduced by 59% with 27% savings in EDP. Finally, last policy saves 65% in static power and 30% in EDP with minimal performance penalty.

Dead blocks are handled inefficiently in multi-level cache hierarchies because the decision as to whether a block is dead has to be taken locally at each cache level. This paper introduces runtime-assisted global cache management to quickly deem blocks dead across cache levels in the context of task-based parallel programs. The scheme is based on a cooperative hardware/software approach that leverages static and dynamic information about future data region reuse(s) available to runtime systems for task-based parallel programming models. We show that our proposed runtime-assisted global cache management approach outperforms previously proposed local dead-block management schemes for task-based parallel programs.

In cache-based side channel attacks, a spy that shares a cache with a victim probes cache locations to extract information on the victim's access patterns. For example, in evict+reload, the spy repeatedly evicts and then reloads a probe address, checking if the victim has accessed the address in between the two operations. While there are many proposals to combat these cache attacks, they all have limitations: they either hurt performance, require programmer intervention, or can only defend against some types of attacks. This paper makes the following observation for an environment with an inclusive cache hierarchy: when the spy evicts the probe address from the shared cache, the address will also be evicted from the private cache of the victim process, creating an inclusion victim. Consequently, to disable cache attacks, this paper proposes to alter the line replacement algorithm of the shared cache, to prevent a process from creating inclusion victims in the caches of cores running other processes. By enforcing this rule, the spy cannot evict the probe address from the shared cache and, hence, cannot glimpse any information on the victim's access patterns. We call our proposal SHARP (Secure Hierarchy-Aware cache Replacement Policy). SHARP efficiently defends against all existing cross-core shared-cache attacks, needs only minimal hardware modifications, and requires no code modifications. We implement SHARP in a cycle-level full-system simulator. We show that it protects against real-world attacks, and that it introduces negligible average performance degradation.

In this paper, we propose a cooperative caching scheme for multimedia data via clusters based on peers connectivity in mobile P2P networks. In the proposed scheme, a cluster is organized for cache sharing among mobile peers with long-term connectivity, and metadata are disseminated to neighbor peers for efficient multimedia data search performance. It reduces data duplication and uses cache space efficiently through integrative cache management of peers inside the cluster. The proposed scheme reduces data replacement time in the event of changes in topology or cache data replacement using the concept of temporal cache. It performs data recovery and cluster adjustment through cluster management in the event of an abrupt disconnection of a peer. In this scheme, metadata of popular multimedia data are disseminated to neighbor peers for efficient data searching. In a data search, queries are processed in the order of local cache, metadata, the cluster to which it belongs, and neighbor clusters, in accordance with cooperative caching strategy. Performance evaluation results show that the proposed scheme has a higher cache hit ratio, and lower cost for data replacement and query processing than existing schemes.

Sensor nodes in Wireless Sensor Networks (WSNs) are battery-powered devices that consume energy during data transmission and processing. One of the most critical tasks in a sensor network is dealing with reliable end-to-end transmissions and optimizing the power consumption. WSNs are error prone due to the constrained nature of the nodes and the interference with other wireless technologies such as Wifi and Bluetooth. One possible way to minimize this problem is caching the data. Data caching is one technique of improving the performance of a transport protocol. Since a typical data transmission consumes more energy than processing in a sensor network, the use of caching enables quick access to data. Therefore, caching, if used efficiently, could reduce overall network traffic and hence bandwidth can be optimally utilized. Unfortunately, a systematic analysis of caching in WSNs was until now lacking. Given that WSNs are expected to play an important role in Machine Type Communications (MTC) and Internet of Things (IoT), the authors believe that now is time to collect the results of years of research on this important topic. This paper presents a comprehensive survey on the state-of-the-art cache-based transport protocols in wireless sensor networks. We classify the transport protocols by presenting a thematic taxonomy of the current cache management mechanisms in wireless sensor networks. Moreover, the critical aspects of the existing cache-ware schemes in sensor networks are analyzed to determine the strengths and weaknesses of such protocols. The similarities and differences of the transport protocols based on the important parameters, such as cache insertion/replacement policy, cache size requirement, cache location, cache partition, and cache decision are investigated in this paper. In addition to that, we discuss open research issues and challenges of cache-based transport protocols in wireless sensor networks. We strongly believe that this study can serve as a basis so that future implementations can choose the combination of caching mechanisms that best fits their target application scenario.

Cache locking is a cache management technique to preclude the replacement of locked contents. Cache locking methods have been proposed to improve predictability and worst-case execution time (WCET) previously. Recently, instruction cache locking has also been applied to improve average-case execution time (ACET). However, we observe that the previous ACET-driven instruction cache locking technique shows very limited improvement on performance when applied in data cache. The underlying reason lies in that object similarity of data accesses in data memory blocks are relatively low. This paper presents a data re-allocation enabled cache locking framework where data objects are first re-allocated to enhance data object similarity in memory blocks and then a data cache locking is well motivated. In this way, locking efficiency for data cache can be enhanced and thus system performance can be improved. The experimental results show that the miss rate, memory access cycles and dynamic energy can obtain good improvements across a suite of benchmarks.

Ray tracing is a computationally intensive task required by movie-makers to create the highly realistic images they require for motion pictures. GPUs currently dominate as hardware accelerators in the multi-billion dollar movie industry. However, FPGAs with their flexibility have the potential to be more power efficient accelerators. This paper investigates and proposes an FPGA acceleration platform to easily and efficiently explore parallelism in ray tracing on FPGAs. It is integrated with LuxRender, a modern and open rendering engine. A major focus is on the proposal of a cache architecture, given that memory bandwidth is identified as a bottleneck. For the design of the domain-specific cache, we study the typical memory access patterns to ray tracing data structures, i.e. acceleration hierarchy and scene primitives. The results lead to the proposal of a data structure specific cache (separating nodes from primitives) and a novel node cache replacement policy. We demonstrate that the proposed cache architecture successfully alleviates the memory bandwidth bottleneck and that the novel replacement policy can provide a good performance increase for small cache sizes and may work well as a complimentary cache along side a direct mapped cache. The evaluation further shows that scaling the number of traversal units with a cache provides an increase in performance over all scenes. We also demonstrate the energy and bandwidth efficiency of the proposed platform in comparison to a CPU and a GPU platform.

The last level cache (LLC) in private configurations offer lower latency and isolation but extinguishes the possibility of sharing underutilized cache resources. Cooperative Caching (CC) provides capacity sharing by spilling a line evicted from one cache to another. However, CC proposals did not pay enough attention to the natural problem of private LLC, replication. The static policies either indulging the replicated blocks (replicas) in or excluding them out of LLC invariably are deficient for the complex cache capacity situations in manycore environment. In this paper, we present replication-aware cache management (RACMan) to optimize replication for private configurations. RACMan relies on a novel coarse-grained low-overheard mechanism PBFP that monitors and predicts the replica reusability to dynamically adjust LLC insertion policies giving replicas different positions of LRU chain and chances of survival in LLC according to the prediction. Experiment results show our proposal is competent to optimize replication by performing better than two baseline systems in the respects of L2 Hit Rate, Network Traffics, IPC, and Dynamic Energy. RACMan fulfils the requirements of manycore CMPs with private LLC for increasing system performance, area efficiency, and scalability.

International audience; Cloud computing and Network Function Virtualization (NFV) use cases proposed by ETSI can improve network caching when an adequate optimization algorithm is used. A cache placement algorithm is proposed based on these virtualization tools. Moreover, a cache placement protocol is presented to support the cache migration. Optimal cache and Virtual Machine (VM) placement and migration, and optimal routing based on several new constraints such as system load, network infrastructure, user satisfaction and migration cost is designed. Extensive evaluations and comparison to state of the art techniques are carried out. The results show that our cache techniques outperform other solutions and give better satisfaction to the three constituents: users, network operators and content providers.

Very Long Instruction Word (VLIW) architectures are commonly used in application-specific domains due to their parallelism and low-power characteristics. Recently, parameterization of such architectures allows for runtime adaptation of the issue-width to match the inherent Instruction Level Parallelism (ILP) of an application. One implementation of such an approach is that the event of the issue-width switching dynamically triggers the reconfiguration of the data cache at runtime. In this paper, the relationship between cache resizing and issue-width is well investigated. We have observed that the requirement of the cache does not always correlate with the issuewidth of the VLIW processor. To further coordinate the cache resizing with the changing issue-width, we present a novel feedback mechanism to "block" the low yields of cache resizing when the issue-width changes. In this manner, our feedback cache mechanism has a coordinated effort with the issue-width changes, which leads to a noticeable improvement of the cache performance. The experiments show that there is 10% energy savings as well as a 2.3% cache misses decline on average achieved, compared with the cache without the feedback mechanism. Therefore, the feedback mechanism is proven to have the capability to ensure more benefits are achieved from the dynamic and frequent reconfiguration.

Modern graphical processing units (GPUs) are equipped with general-purpose L1 and L2 caches to reduce the memory bandwidth demand and improve the performance of some irregular general-purpose GPU (GPGPU) applications. However, due to the massive multithreading, GPGPU caches suffer from severe resource contention and low data-sharing which may lead to performance degradation instead. This paper proposes a low-cost streaming-aware conflict-avoiding thrashing-resistant (SACAT) GPGPU cache management scheme that efficiently utilizes the GPGPU cache resources and addresses all the problems associated with GPGPU caches. The proposed scheme employs three orthogonal techniques. First, it dynamically detects and bypasses streaming applications at fine granularity. Second, a dynamic warp throttling via cores sampling (DWT-CS) is proposed to alleviate cache thrashing. DWT-CS runs an exhaustive search over cores to find the best number of warps that achieves the highest performance. Third, it employs pseudo random interleaving cache (PRIC), which is an improved cache indexing function based on polynomial modulus mapping, to mitigate associativity stalls and eliminate conflict misses. Experimental results demonstrate that the proposed scheme achieves a 1.87 $\times$ and a 1.5 $\times$ performance improvement over the cache-conscious wavefront scheduler (CCWS) and the memory request prioritization buffer (MRPB), respectively.

Worst-Case Execution Time (WCET) is an important metric for programs running on real-time systems, and finding precise estimates of a program’s WCET is crucial to avoid wastage of hardware resources and to improve the schedulability of task sets. Caches have a major impact on a program’s execution time, and accurate estimation of a program’s cache behavior can lead to significant reduction in its estimated WCET. The traditional approach to cache analysis generally targets the worst-case cache behavior of individual cache accesses and provides a safe hit-miss classification for every individual access. In this work, we show that these classifications are not sufficient to precisely capture cache behavior, since they apply to individual accesses, and often, more precise predictions can be made about groups of accesses. Further, memory accesses inside loops may show the worst-case behavior only for a subset of the iteration space. In order to predict such behavior in a scalable fashion, we use the fact that the cache behavior of an access mostly depends only on the memory accesses made in the immediate vicinity, and hence we analyze a small, fixed-size neighborhood of every access with complete precision and summarize the resulting information in the form of cache miss paths. A variety of analyses are then performed on the cache miss paths to make precise predictions about cache behavior. We also demonstrate precision issues in Abstract Interpretation-based Must and Persistence cache analysis that can be easily solved using cache miss paths. Experimental results over a wide range of benchmarks demonstrate precision improvement in WCET of multipath programs over previous approaches, and we also show how to integrate our approach with other microarchitectural analysis such as pipeline analysis.

As the number of on-chip cores and memory demands of applications increase, judicious management of cache resources has become not merely attractive but imperative. Cache partitioning, that is, dividing cache space between applications based on their memory demands, is a promising approach to provide capacity benefits of shared cache with performance isolation of private caches. However, naively partitioning the cache may lead to performance loss, unfairness, and lack of quality-of-service guarantees. It is clear that intelligent techniques are required for realizing the full potential of cache partitioning. In this article, we present a survey of techniques for partitioning shared caches in multicore processors. We categorize the techniques based on important characteristics and provide a bird’s eye view of the field of cache partitioning.