1. Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity
- Author
-
Wu, Xundong, Zhao, Pengfei, Yu, Zilin, Ma, Lei, Yip, Ka-Wa, Tang, Huajin, Pan, Gang, and Huang, Tiejun
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,FOS: Biological sciences ,Quantitative Biology - Neurons and Cognition ,Computer Science - Neural and Evolutionary Computing ,Neurons and Cognition (q-bio.NC) ,Neural and Evolutionary Computing (cs.NE) ,Machine Learning (cs.LG) - Abstract
Our comprehension of biological neuronal networks has profoundly influenced the evolution of artificial neural networks (ANNs). However, the neurons employed in ANNs exhibit remarkable deviations from their biological analogs, mainly due to the absence of complex dendritic trees encompassing local nonlinearity. Despite such disparities, previous investigations have demonstrated that point neurons can functionally substitute dendritic neurons in executing computational tasks. In this study, we scrutinized the importance of nonlinear dendrites within neural networks. By employing machine-learning methodologies, we assessed the impact of dendritic structure nonlinearity on neural network performance. Our findings reveal that integrating dendritic structures can substantially enhance model capacity and performance while keeping signal communication costs effectively restrained. This investigation offers pivotal insights that hold considerable implications for the development of future neural network accelerators.
- Published
- 2023