Search

In natural language processing (NLP), transformer based models have surpassed recurrent neural networks (RNN) as state of the art, being introduced specifically to address the limitations of RNNs originating from its sequential nature. As a similar sequence modeling problem, transformer methods can be readily adapted for deep learning time series prediction. This paper proposes a sparse transformer based approach for electricity load prediction. The layers of a transformer addresses the shortcomings of RNNs and CNNs by applying the attention mechanism on the entire time series, allowing any data point in the input to influence any location in the output of the layer. This allows transformers to incorporate information from the entire sequence in a single layer. Attention computations can also be parallelized. Thus, transformers can achieve faster speeds, or trade this speed for more layers and increased complexity. In experiments on public datasets, the sparse transformer attained comparable accuracy to an RNN-based SOTA method (Liu et al., 2022) while being up to 5× faster during inference. Moreover, the proposed model is general enough to forecast the load from individual households to city levels as shown in the extensive experiments conducted. [ABSTRACT FROM AUTHOR]

Artificial neural networks, inspired by the biological networks of the human brain, have become game-changing computing models in modern computer science. Inspired by their wide scope of applications, synthetic biology strives to create their biological counterparts, which we denote synthetic biological neural networks (SYNBIONNs). Their use in the fields of medicine, biosensors, biotechnology, and many more shows great potential and presents exciting possibilities. So far, many different synthetic biological networks have been successfully constructed, however, SYNBIONN implementations have been sparse. The latter are mostly based on neural networks pretrained in silico and being heavily dependent on extensive human input. In this paper, we review current implementations and models of SYNBIONNs. We briefly present the biological platforms that show potential for designing and constructing perceptrons and/or multilayer SYNBIONNs. We explore their future possibilities along with the challenges that must be overcome to successfully implement a scalable in vivo biological neural network capable of online learning. [ABSTRACT FROM AUTHOR]