Recognition of Novelty Made Easy: Constraints of Channel Capacity on Generative Networks

We subseribe to the idea that the brain employs generative networks. In turn, we conclude that channel capacity constraints form the main obstacle for effective information transfer in the brain. Robust and fast information flow processing methods warranting efficient information transfer, e.g. grouping of inputs and information maximization principles need to be applied. For this reason, indepent component analyses on groups of patterns were conducted using (a) model labyrinth, (b) movies on highway traffic and (c) mixed acoustical signals. We found that in all cases ‘familiar’ inputs give rise to cumulated firing histograms close to exponential distributions, whereas ‘novel’ information are better deseribed by broad, sometimes truncated Gaussian distributions. It can be shown that upon minimization of mutual information between processing channels, noise can reveal itself locally. Therefore, we conjecture that novelty - as opposed to noise - can be recognized by means of the statistics of neuronal firing in brain areas.