Search

Nearest neighbor (NN) search computations are at the core of many applications such as few-shot learning, classification, and hyperdimensional computing. As such, efficient hardware support for NN search is highly desired. In-memory computing using emerging devices offers attractive solutions for NN search. Solutions based on ternary content-addressable memories (TCAMs) offer high energy and latency improvements for NN search at the expense of accuracy. In this work, we propose a novel distance function that can be natively evaluated with multi-bit content-addressable memories (MCAMs) based on ferroelectric FETs (FeFETs) to perform a single-step, in-memory NN search. We evaluate the efficacy of FeFET MCAMs in the context of few-shot learning applications with different datasets. As an example, we achieve a 78.54% accuracy for a 5-way, 5-shot classification task for the mini-ImageNet dataset (only 1.5% lower than software-based implementations) when using a 3-bit MCAM for NN search. We consider the effects of FeFET threshold voltage variations on the application accuracy and analyze the area and search energy requirements of FeFET MCAMs for accurate operations. Our results indicate that MCAMs require 2× lower area and search energy than TCAMs to achieve the same accuracy. Furthermore, we experimentally demonstrate a 2-bit implementation of FeFET MCAM using AND arrays from GLOBALFOUNDRIES to further validate the design concept. [ABSTRACT FROM AUTHOR]

Transformer networks have outperformed recurrent and convolutional neural networks in terms of accuracy in various sequential tasks. However, memory and compute bottlenecks prevent transformer networks from scaling to long sequences due to their high execution time and energy consumption. Different neural attention mechanisms have been proposed to lower computational load but still suffer from the memory bandwidth bottleneck. In-memory processing can help alleviate memory bottlenecks by reducing the transfer overhead between the memory and compute units, thus allowing transformer networks to scale to longer sequences. We propose an in-memory transformer network accelerator (iMTransformer) that uses a combination of crossbars and contentaddressable memories to accelerate transformer networks. We accelerate transformer networks by (1) computing in-memory, thus minimizing the memory transfer overhead, (2) caching reusable parameters to reduce the number of operations, and (3) exploiting the available parallelism in the attention mechanism computation. To reduce energy consumption, the following techniques are introduced: (1) a configurable attention selector is used to choose different sparse attention patterns, (2) a contentaddressable memory aided locality sensitive hashing helps to filter the number of sequence elements by their importance, and (3) FeFET-based crossbars are used to store projection weights while CMOS-based crossbars are used as an attentional cache to store attention scores for later reuse. Using a CMOS-FeFET hybrid iMTransformer introduced a significant energy improvement compared to the CMOS-only iMTransformer. The CMOS-FeFET hybrid iMTransformer achieved an 8.96× delay improvement and 12.57× energy improvement for the Vanilla transformers compared to the GPU baseline at a sequence length of 512. Implementing BERT using CMOS-FeFET hybrid iMTransformer achieves 13.71× delay improvement and 8.95× delay improvement compared to the GPU baseline at sequence length of 512. The hybrid iMTransformer also achieves a throughput of 2.23 K samples/sec and 124.8 samples/s/W using the MLPerf benchmark using BERT-large and SQuAD 1.1 dataset, an 11× speedup and 7.92× energy improvement compared to the GPU baseline. [ABSTRACT FROM AUTHOR]