The transmission of information and the effect of local feedback in theoretical and neural networks

Summary For pattern recognition, linear separation networks have been proposed in which the contribution of an input to an association unit is made proportional to the logarithm of the conditional probability of that input given the output of the association unit. Such networks achieve useful discrimination for only the limited condition in which inputs are independent. In practice the output of such a network exceeds the log conditional probability of its corresponding pattern by a large uncalculable quantity; consequently selection cannot be made by comparing the outputs of networks with a calculable significance threshold-one can only select the largest output; a further consequence is that local feedback from output to input does not give the association unit any useful property of autonomous classification. This paper examines a network in which the contribution of an input is made proportional to the Shannon information between that input and the output. Both the above difficulties then disappear, and local feedback causes an association unit to construct a useful class of inputs without the need of an external teacher which ‘knows the answers’. The classification behaviour of such an information network is examined for a number of simple examples in which inputs do not obey an Independence Relation. The results bear some resemblance to recently discovered properties of neurones in the cerebral cortex. The theory suggests that there must be an effect between neurones which has not so far been observed, and a way of looking for it. The theory offers an explanation of some known perceptual phenomena.