Search

Adaptive routing algorithms can improve performance by balancing load across network channels in the presence of non-uniform traffic patterns. However, out-of-order packets can be introduced due to multi-path transmission of adaptive routing. With out-of-order transmission in the network, packets need to be reordered at the destination before being absorbed. Increasing network size with adaptive routing makes the time when a packet arrives at the destination extremely uncertain, which requires a large buffer to reorder the packets and this can exceed design space. Therefore, the challenge is to balance the trade-off between multi-path transmission and packet reordering. In this paper, we propose a novel packet reordering metric-OOD to quantify the degree of out-of-order. To minimize the OOD of packets, we propose DancerFly, an order-aware network-on-chip router that mitigates out-of-order packets caused by adaptive routing. DancerFly achieves this goal by providing two-level reordering. First, it performs in-buffer reordering by reordering packets queuing in the input buffer. Second, packets from different input ports are reordered before traversing through the router. We evaluate our design and the results show that the OOD can be reduced by 36.3% with comparable performance to the baseline. [ABSTRACT FROM AUTHOR]

The introduction of unified memory and demand paging has simplified programming of graphics processing units (GPUs). It has also enabled oversubscribing the memory for a GPU. However, the overhead of page management makes page faults a performance bottleneck. Sometimes the page eviction policy is unable to mitigate performance slowdown caused by page faults and memory oversubscription. On average, eviction policies such as Random and CAR are not competitive with a traditional least recently used (LRU) policy. Other policies, such as CLOCK-Pro, are designed to overcome challenges with LRU, but they only achieve limited speedup. Even enhancing LRU with page walk hit information does not lead to notable performance improvement. Based on these observations, we propose optimization opportunities to mitigate performance degradation caused by page faults and memory oversubscription. These optimization opportunities include an effective page eviction policy that retains LRU's advantages while addressing LRU's inability to deal with thrashing access patterns, page prefetch and pre-eviction, memory-aware throttling, and capacity compression. [ABSTRACT FROM AUTHOR]

Memory access scheduling schemes, often performed in memory controllers, have a marked impact on alleviating the heavy burden placed on memory systems of GPGPUs. Existing out-of-order scheduling schemes, like FR-FCFS, improve memory access efficiency by reordering memory request sequences at the destination. Their effectiveness, however, is at the expense of complex logics and high power consumption. In this paper, we propose a NoC-side memory access scheduling based on the key insight that the transmission of on-chip networks is the dominating factor in destroying the row access locality and causing poor memory access efficiency. With appropriate NoC-side optimization, the straight-forward in-order scheduling can be used in memory controllers to simplify scheduling logics and alleviate the tight power envelope. Moreover, we introduce several light-weight optimizations to further improve the system performance. Experimental results on memory-intensive applications show that, comparing with FR-FCFS, our proposed scheme increases the overall system performance by 10.5%, reduces the power consumption by 20% and improves the energy efficiency by 36.9%. [ABSTRACT FROM AUTHOR]

The “memory wall” problem has necessitated the introduction of more accurate memory metrics. C-AMAT, an extension of AMAT that considers memory access concurrency, can evaluate the performance of modern memory systems more accurately and comprehensively. However, compared to AMAT, calculating C-AMAT means additional complexity and more hardware resources, which incurs high overhead for this metric. In this paper, we propose FC-AMAT, an analysis model based on C-AMAT. We first explore the numerical relations between factors’ C-AMAT and overall C-AMAT, then try to use factor’s-first (changed factor or leading factor) C-AMAT to evaluate effects of changes applied to a memory system, which can not only reduce measurement overhead, but also guarantee acceptable evaluation accuracy through a rigorous check. Furthermore, we provide the method to determine the leading factor of a memory system. Simulations with varied configurations are conducted to study the numerical relations between factors’ C-AMAT and overall C-AMAT, as well as to evaluate the effects of FC-AMAT. Experimental results show that (1) parameters of factors’ C-AMAT have different influence degrees on overall C-AMAT, (2) the leading factor of a memory system remains unchanged across various configurations and (3) using factor’s C-AMAT as the evaluation metric reduces the hardware overhead for measurement, including detecting logic and storage cost, which demonstrates the effectiveness of FC-AMAT. [ABSTRACT FROM AUTHOR]