Search

Due to complex interactions among various deep neural network (DNN) optimization techniques, modern DNNs can have weights and activations that are dense or sparse with diverse sparsity degrees. To offer a good trade-off between accuracy and hardware performance, an ideal DNN accelerator should have high flexibility to efficiently translate DNN sparsity into reductions in energy and/or latency without incurring significant complexity overhead. This paper introduces hierarchical structured sparsity (HSS), with the key insight that we can systematically represent diverse sparsity degrees by having them hierarchically composed from multiple simple sparsity patterns. As a result, HSS simplifies the underlying hardware since it only needs to support simple sparsity patterns; this significantly reduces the sparsity acceleration overhead, which improves efficiency. Motivated by such opportunities, we propose a simultaneously efficient and flexible accelerator, named HighLight, to accelerate DNNs that have diverse sparsity degrees (including dense). Due to the flexibility of HSS, different HSS patterns can be introduced to DNNs to meet different applications’ accuracy requirements. Compared to existing works, HighLight achieves a geomean of up to 6.4 × better energy-delay product (EDP) across workloads with diverse sparsity degrees, and always sits on the EDP-accuracy Pareto frontier for representative DNNs.

Sparse tensor algebra is a challenging class of workloads to accelerate due to low arithmetic intensity and varying sparsity patterns. Prior sparse tensor algebra accelerators have explored tiling sparse data to increase exploitable data reuse and improve throughput, but typically allocate tile size in a given buffer for the worst-case data occupancy. This severely limits the utilization of available memory resources and reduces data reuse. Other accelerators employ complex tiling during preprocessing or at runtime to determine the exact tile size based on its occupancy. This paper proposes a speculative tensor tiling approach, called overbooking, to improve buffer utilization by taking advantage of the distribution of nonzero elements in sparse tensors to construct larger tiles with greater data reuse. To ensure correctness, we propose a low-overhead hardware mechanism, Tailors, that can tolerate data overflow by design while ensuring reasonable data reuse. We demonstrate that Tailors can be easily integrated into the memory hierarchy of an existing sparse tensor algebra accelerator. To ensure high buffer utilization with minimal tiling overhead, we introduce a statistical approach, Swiftiles, to pick a tile size so that tiles usually fit within the buffer’s capacity, but can potentially overflow, i.e., it overbooks the buffers. Across a suite of 22 sparse tensor algebra workloads, we show that our proposed overbooking strategy introduces an average speedup of 52.7 × and 2.3 × and an average energy reduction of 22.5 × and 2.5 × over ExTensor without and with optimized tiling, respectively.

Lire Battre les cartes, c'est partir à la découverte du Shanghai des années cinquante à soixante-dix à travers les souvenirs de l'auteur qui y vit depuis toujours. Une collection d'histoires vraies sur la vie des petites gens dans les ruelles, le salon de coiffure, les ouvrières dans les usines, les promendes en barque, et les angoisses aussi de ces temps de grands bouleversements, avec un foisonnement de détails concrets qui nous font toucher du doigt les objets de la vie quotidienne et leurs métamorphoses au fil des ans, meubles, machines, nourriture, vêtements, couettes, valises, outils... et une langue recherchée, subtile et sensuelle, qui nous fait frémir et vibrer en partageant les mésaventures de ces personnages d'une Chine disparue, tour à tour cocasses, tragiques, émouvantes.... Né en 1952 à Shanghai, Jin Yucheng a 16 ans quand il est envoyé travailler dans une ferme, où il restera huit ans. Rentré à Shanghai, il doit travailler à l'usine pour vivre mais réussit à publier une nouvelle en 1985 dans le magazine Mengya, qui obtient un prix. En 1989 il devient rédacteur en chef de la prestigieuse revue Littérature de Shanghai. En 2006 paraît Battre les cartes, qui préfigure son grand roman Blossoms, paru en 2012.

Lire Battre les cartes, c'est partir à la découverte du Shanghai des années cinquante à soixante-dix à travers les souvenirs de l'auteur qui y vit depuis toujours. Une collection d'histoires vraies sur la vie des petites gens dans les ruelles, le salon de coiffure, les ouvrières dans les usines, les promendes en barque, et les angoisses aussi de ces temps de grands bouleversements, avec un foisonnement de détails concrets qui nous font toucher du doigt les objets de la vie quotidienne et leurs métamorphoses au fil des ans, meubles, machines, nourriture, vêtements, couettes, valises, outils... et une langue recherchée, subtile et sensuelle, qui nous fait frémir et vibrer en partageant les mésaventures de ces personnages d'une Chine disparue, tour à tour cocasses, tragiques, émouvantes.... Né en 1952 à Shanghai, Jin Yucheng a 16 ans quand il est envoyé travailler dans une ferme, où il restera huit ans. Rentré à Shanghai, il doit travailler à l'usine pour vivre mais réussit à publier une nouvelle en 1985 dans le magazine Mengya, qui obtient un prix. En 1989 il devient rédacteur en chef de la prestigieuse revue Littérature de Shanghai. En 2006 paraît Battre les cartes, qui préfigure son grand roman Blossoms, paru en 2012.

Processing-in-memory (PIM) deep neural network (DNN) accelerators, which aim to improve energy/area efficiency of DNN processing by integrating computation into data storage, have gained popularity in recent years. Therefore, it is attractive to have a generally applicable framework that is able to quickly provide insights into the various trade-offs involved in PIM accelerator designs. We present an architecture-level design estimation framework for PIM accelerators that allows easy representations of the designs with provided architecture templates and component design templates, performs analytical runtime simulations, and produces technology-dependent area and energy estimations. We show that the framework can be easily used to evaluate state of the art PIM accelerator designs; it achieves 95% accurate total energy estimations and reproduces exact area breakdowns of the components in the design. Related open-source code is available at http://accelergy.mit.edu/.

With Moore’s law slowing down and Dennard scaling ended, energy efficient domain-specific accelerators, such as deep neural network (DNN) processors for machine learning and programmable network switches for cloud applications, have become a promising way for hardware designers to continue bringing energy efficiency improvements to data and computation-intensive applications. To ensure the fast exploration of the accelerator design space, architecture level energy estimators, which perform energy estimations without requiring complete hardware description of the designs, are critical to designers. However, it is difficult to use existing architecture level energy estimators to obtain accurate estimates for accelerator designs, as accelerator designs are diverse and sensitive to data patterns. This paper presents Accelergy, a generally applicable energy estimation methodology for accelerators that allows design specifications comprised of user-defined high-level compound components and user-defined low-level primitive components, which can be characterized by third-party energy estimation plug-ins. An example with primitive and compound components for DNN accelerator designs is also provided as an application of the proposed methodology. Overall, Accelergy achieves 95% accuracy on Eyeriss, a well-known DNN accelerator design, and can correctly capture the energy breakdown of components at different granularities. The Accelergy code is available at http://accelergy.mit.edu., United States. Defense Advanced Research Projects Agency (Contract HR0011-18-3-0007)