Back to Search Start Over

Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity

Authors :
Wu, Xundong
Zhao, Pengfei
Yu, Zilin
Ma, Lei
Yip, Ka-Wa
Tang, Huajin
Pan, Gang
Huang, Tiejun
Publication Year :
2023

Abstract

Our comprehension of biological neuronal networks has profoundly influenced the evolution of artificial neural networks (ANNs). However, the neurons employed in ANNs exhibit remarkable deviations from their biological analogs, mainly due to the absence of complex dendritic trees encompassing local nonlinearity. Despite such disparities, previous investigations have demonstrated that point neurons can functionally substitute dendritic neurons in executing computational tasks. In this study, we scrutinized the importance of nonlinear dendrites within neural networks. By employing machine-learning methodologies, we assessed the impact of dendritic structure nonlinearity on neural network performance. Our findings reveal that integrating dendritic structures can substantially enhance model capacity and performance while keeping signal communication costs effectively restrained. This investigation offers pivotal insights that hold considerable implications for the development of future neural network accelerators.

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2306.11950
Document Type :
Working Paper