Optimization of Microarchitecture and Dataflow for Sparse Tensor CNN Acceleration

The inherent sparsity present in convolutional neural networks (CNNs) offers a valuable opportunity to significantly decrease the computational workload during inference. Nevertheless, leveraging unstructured sparsity typically comes with the trade-off of increased complexity or substantial hardware overheads for accelerators. To address these challenges, this research introduces an innovative inner join aimed at effectively reducing the size and power consumption of the sparsity-handling circuit. Additionally, a novel dataflow named Channel Stacking of Sparse Tensors (CSSpa) is presented, focusing on maximizing data reuse to minimize memory accesses – an aspect that significantly contributes to overall power consumption. Through comprehensive simulations, CSSpa demonstrates a $1.6\times $ speedup and a $5.6\times $ reduction in SRAM accesses when executing inference on the ResNet50 model, compared to the existing Sparten architecture. Furthermore, the implementation results reveal a notable $2.32\times $ enhancement in hardware resource efficiency and a $3.3\times $ improvement in energy efficiency compared to Sparten.