1. PipeCNN:一种基于软件流水线的并行化卷积神经网络方法.
- Author
-
吴鹏 and 周宁宁
- Subjects
- *
CONVOLUTIONAL neural networks , *BACK propagation , *DEEP learning , *PROBLEM solving , *COMPUTER software - Abstract
Aiming at the traditional model parallel methods for accelerating convolution neural network (CNN) tend to have low utilization, this paper proposed PipeCNN, which accelerated CNN with software pipeline. Firstly, this paper studied the forward propagation and back propagation, and then explored data correlation during training. Secondly, it parallelized CNN with the support of software pipeline, and then analyzed two feasible gradient updating methods in PipeCNN. Finally, it used circular queue to realize communication between two layers and then proposed a task allocation algorithm to divide CNN into working parts. Experiments show that the method can obtain good speedup and utilization while ensuring the accuracy of the model. It shows that software pipeline can effectively solve the problem of low utilization in model parallel, and accelerate the training of CNN. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF