Back to Search
Start Over
A Hardware Prototype Targeting Distributed Deep Learning for On-device Inference
- Source :
- CVPR Workshops
- Publication Year :
- 2020
- Publisher :
- IEEE, 2020.
-
Abstract
- This paper presents a hardware prototype and a framework for a new communication-aware model compression for distributed on-device inference. Our approach relies on Knowledge Distillation (KD) and achieves orders of magnitude compression ratios on a large pre-trained teacher model. The distributed hardware prototype consists of multiple student models deployed on Raspberry-Pi 3 nodes that run Wide ResNet and VGG models on the CIFAR10 dataset for real-time image classification. We observe significant reductions in memory footprint (50×), energy consumption (14×), latency (33×) and an increase in performance (12×) without any significant accuracy loss compared to the initial teacher model. This is an important step towards deploying deep learning models for IoT applications.
Details
- Database :
- OpenAIRE
- Journal :
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Accession number :
- edsair.doi...........b1e4c185ee3b3f1d308294c1ca792fbd