Feature and structural learning of memory sequences with recurrent and gated spiking neural networks using free-energy: application to speech perception and production I

We propose a unified framework for modeling the cortico-basal system (CX-BG) and the fronto-striatal system (PFC-BG) for the generation and recall of audio memory sequences; ie, sound perception and speech production. Our genuine model is based on the neural architecture called INFERNO standing for Iterative Free-Energy Optimization of Recurrent Neural Networks. Free-energy (noise) minimization is used for exploring, selecting and learning in PFC the optimal choices of actions to perform in the BG network (eg sound production) in order to reproduce and control the most accurately possible the spike trains representing sounds in CX. The difference between the two working memories relies in the neural coding itself, which is based on temporal ordering in the CX-BG networks (Spike Timing-Dependent Plasticity) and on the rank ordering in the sequence in the PFC-BG networks (gating or gain-modulation). We detail in this paper only the CX-BG system responsible to encode the audio primitives at few milliseconds order, while the PFC-BG system responsible for the learning of temporal structure in sequences will be presented in a complementary paper. Two experiments done with a small and a big audio database show the capabilities of exploration, generalization and robustness to noise of the neural architecture to retrieve audio primitives.