Back to Search
Start Over
DCP–NAS: Discrepant Child–Parent Neural Architecture Search for 1-bit CNNs.
- Source :
-
International Journal of Computer Vision . Nov2023, Vol. 131 Issue 11, p2793-2815. 23p. - Publication Year :
- 2023
-
Abstract
- Neural architecture search (NAS) proves to be among the effective approaches for many tasks by generating an application-adaptive neural architecture, which is still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binary weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework, while searching the 1-bit CNNs is more challenging due to the more complicated processes involved. In this paper, we introduce Discrepant Child–Parent Neural Architecture Search (DCP–NAS) to efficiently search 1-bit CNNs, based on a new framework of searching the 1-bit model (Child) under the supervision of a real-valued model (Parent). Particularly, we first utilize a Parent model to calculate a tangent direction, based on which the tangent propagation method is introduced to search the optimized 1-bit Child. We further observe a coupling relationship between the weights and architecture parameters existing in such differentiable frameworks. To address the issue, we propose a decoupled optimization method to search an optimized architecture. Extensive experiments demonstrate that our DCP–NAS achieves much better results than prior arts on both CIFAR-10 and ImageNet datasets. In particular, the backbones achieved by our DCP–NAS achieve strong generalization performance on person re-identification and object detection. [ABSTRACT FROM AUTHOR]
- Subjects :
- *CONVOLUTIONAL neural networks
*NEWTON-Raphson method
Subjects
Details
- Language :
- English
- ISSN :
- 09205691
- Volume :
- 131
- Issue :
- 11
- Database :
- Academic Search Index
- Journal :
- International Journal of Computer Vision
- Publication Type :
- Academic Journal
- Accession number :
- 172329465
- Full Text :
- https://doi.org/10.1007/s11263-023-01836-4