1. Boltzmann machines reduction by high-order decimation
- Author
-
Farguell, Enric, Mazzanti, Ferran, and Gomez-Ramirez, Eduardo
- Subjects
Algorithms -- Usage ,Neural networks -- Design and construction ,Simulated annealing (Mathematics) -- Methods ,Algorithm ,Neural network ,Business ,Computers ,Electronics ,Electronics and electrical industries - Abstract
Decimation is a common technique in statistical physics that is used in the context of Boltzmann machines (BMs) to drastically reduce the computational cost at the learning stage. Decimation allows to analytically evaluate quantifies that should otherwise be statistically estimated by means of Monte Carlo (MC) simulations. However, in its original formulation, this method could only be applied to restricted topologies corresponding to sparsely connected neural networks. In this brief, we present a generalization of the decimation process and prove that it can be used on any BM, regardless of its topology and connectivity. We solve the Monk problem with this algorithm and show that it performs as well as the best classification methods currently available. Index Terms--Boltzmann machines (BMs), decimation, neural networks, simulated annealing (SA).
- Published
- 2008