1. Random-based networks with dropout for embedded systems
- Author
-
Rodolfo Zunino, Paolo Gastaldo, C. Gianoglio, and Edoardo Ragusa
- Subjects
Embedded systems ,Internet of Things ,Random-based neural networks ,Speedup ,Computational complexity theory ,Artificial neural network ,Computer science ,business.industry ,020206 networking & telecommunications ,02 engineering and technology ,Regularization (mathematics) ,Artificial Intelligence ,Embedded system ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,business ,Classifier (UML) ,Software - Abstract
Random-based learning paradigms exhibit efficient training algorithms and remarkable generalization performances. However, the computational cost of the training procedure scales with the cube of the number of hidden neurons. The paper presents a novel training procedure for random-based neural networks, which combines ensemble techniques and dropout regularization. This limits the computational complexity of the training phase without affecting classification performance significantly; the method best fits Internet of Things (IoT) applications. In the training algorithm, one first generates a pool of random neurons; then, an ensemble of independent sub-networks (each including a fraction of the original pool) is trained; finally, the sub-networks are integrated into one classifier. The experimental validation compared the proposed approach with state-of-the-art solutions, by taking into account both generalization performance and computational complexity. To verify the effectiveness in IoT applications, the training procedures were deployed on a pair of commercially available embedded devices. The results showed that the proposed approach overall improved accuracy, with a minor degradation in performance in a few cases. When considering embedded implementations as compared with conventional architectures, the speedup of the proposed method scored up to 20× in IoT devices.
- Published
- 2020
- Full Text
- View/download PDF