1. Building partially understandable convolutional neural networks by differentiating class-related neural nodes
- Author
-
Dawei Dai, Chengfu Tang, Shuyin Xia, and Guoyin Wang
- Subjects
0209 industrial biotechnology ,Artificial neural network ,Computer science ,business.industry ,Cognitive Neuroscience ,02 engineering and technology ,Convolutional neural network ,Field (computer science) ,Computer Science Applications ,Task (project management) ,020901 industrial engineering & automation ,Artificial Intelligence ,Encoding (memory) ,0202 electrical engineering, electronic engineering, information engineering ,Feature (machine learning) ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,Neural coding ,Coding (social sciences) - Abstract
In recent years, convolutional neural networks (CNNs) have been successfully applied in the field of image processing, and have been deployed to a variety of artificial intelligence systems. However, such neural models are still considered to be “black box” for most tasks. Two of fundamental issues underlying this problem are as follows: 1. What type of knowledge learned by a neural network in a task was not understandable and predictable? 2. The decision made by a neural model was generally not evaluable. Like neural coding in the brain, some neurons only participated in encoding a particular task. Inspired by this, in this paper, we propose a method to modify traditional CNN models into understandable CNNs, to clarify the information coding in high conv-layers of CNNs and further evaluate the decisions made by a neural model. In our understandable CNN models, each neural node (feature map) in a selected conv-layer was assigned to participate in encoding only one class in a classification task. Our models use the same training data as ordinary models without the need for additional annotations for supervision. We applied our method to the ResNet and DenseNet models. The experiments showed that new models can learn the information coding mode that we expected in an image-recognition task, and, using the pre-assigned coding mode, we can interpret why a neural model makes a right or wrong decision, which decisions are credible, and which are not.
- Published
- 2021