1. Predictive coding feedback results in perceived illusory contours in a recurrent neural network
- Author
-
Callum Biggs O'May, Zhaoyang Pang, Bhavin Choksi, Rufin VanRullen, Centre de recherche cerveau et cognition (CERCO UMR5549), Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Centre National de la Recherche Scientifique (CNRS)-Toulouse Mind & Brain Institut (TMBI), Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées, Artificial and Natural Intelligence Toulouse Institute (ANITI), ANR-19-P3IA-0004,ANITI,Artificial and Natural Intelligence Toulouse Institute(2019), ANR-19-NEUC-0004,OsCiDeep,US-France Research Proposal: Oscillatory processes for visual reasoning in deep neural networks(2019), VANRULLEN, Rufin, Artificial and Natural Intelligence Toulouse Institute - - ANITI2019 - ANR-19-P3IA-0004 - P3IA - VALID, US-France Research Proposal: Oscillatory processes for visual reasoning in deep neural networks - - OsCiDeep2019 - ANR-19-NEUC-0004 - NEUC - VALID, Centre de recherche cerveau et cognition (CERCO), Institut des sciences du cerveau de Toulouse. (ISCT), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-CHU Toulouse [Toulouse]-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3), and Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-CHU Toulouse [Toulouse]-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Centre National de la Recherche Scientifique (CNRS)
- Subjects
FOS: Computer and information sciences ,[INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI] ,Computer science ,Computer Vision and Pattern Recognition (cs.CV) ,Cognitive Neuroscience ,media_common.quotation_subject ,Computer Science - Computer Vision and Pattern Recognition ,Illusion ,Convolutional neural network ,050105 experimental psychology ,Feedback ,[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI] ,03 medical and health sciences ,0302 clinical medicine ,Artificial Intelligence ,Perception ,Illusory contours ,medicine ,Humans ,0501 psychology and cognitive sciences ,Visual Cortex ,media_common ,Artificial neural network ,business.industry ,[SCCO.NEUR]Cognitive science/Neuroscience ,Deep learning ,[SCCO.NEUR] Cognitive science/Neuroscience ,05 social sciences ,Pattern recognition ,Illusions ,Form Perception ,Recurrent neural network ,Visual cortex ,medicine.anatomical_structure ,FOS: Biological sciences ,Quantitative Biology - Neurons and Cognition ,Neurons and Cognition (q-bio.NC) ,Neural Networks, Computer ,Artificial intelligence ,business ,Photic Stimulation ,030217 neurology & neurosurgery - Abstract
Modern feedforward convolutional neural networks (CNNs) can now solve some computer vision tasks at super-human levels. However, these networks only roughly mimic human visual perception. One difference from human vision is that they do not appear to perceive illusory contours (e.g. Kanizsa squares) in the same way humans do. Physiological evidence from visual cortex suggests that the perception of illusory contours could involve feedback connections. Would recurrent feedback neural networks perceive illusory contours like humans? In this work we equip a deep feedforward convolutional network with brain-inspired recurrent dynamics. The network was first pretrained with an unsupervised reconstruction objective on a natural image dataset, to expose it to natural object contour statistics. Then, a classification decision layer was added and the model was finetuned on a form discrimination task: squares vs. randomly oriented inducer shapes (no illusory contour). Finally, the model was tested with the unfamiliar ''illusory contour'' configuration: inducer shapes oriented to form an illusory square. Compared with feedforward baselines, the iterative ''predictive coding'' feedback resulted in more illusory contours being classified as physical squares. The perception of the illusory contour was measurable in the luminance profile of the image reconstructions produced by the model, demonstrating that the model really ''sees'' the illusion. Ablation studies revealed that natural image pretraining and feedback error correction are both critical to the perception of the illusion. Finally we validated our conclusions in a deeper network (VGG): adding the same predictive coding feedback dynamics again leads to the perception of illusory contours., Manuscript under review
- Published
- 2021