The relational bottleneck as an inductive bias for efficient abstraction.

Human learners acquire abstract concepts from limited experience. The effort to explain this capacity has fueled debate between symbolic and connectionist approaches and motivated proposals for neuro-symbolic systems. The relational bottleneck principle suggests a novel way to bridge the gap. By restricting information processing to focus only on relations, the approach encourages abstract symbol-like mechanisms to emerge in neural networks. We present an information theoretic formulation and review neural network architectures that implement the principle, enabling rapid learning and systematic generalization of relational patterns. The approach can explain phenomena ranging from the development of numerical abstractions to capacity limits in cognition; is consistent with findings from cognitive neuroscience; and offers a principle for designing more powerful artificial learning systems. A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain. [ABSTRACT FROM AUTHOR]