A phase transition in sampling from restricted Boltzmann machines
Y.-J. Kwon et al.
Abstract
Restricted Boltzmann machines are a class of undirected graphical models that play a key role in deep learning and unsupervised learning. In this study, we prove a phase transition phenomenon in the mixing time of the Gibbs sampler for a one-parameter restricted Boltzmann machine. Specifically, the mixing time varies logarithmically, polynomially, and exponentially with the number of vertices depending on whether the parameter c is above, equal to, or below a critical value c⋆≈−5.87. A key insight from our analysis is the link between the Gibbs sampler and a dynamic system, which we use to quantify the former based on the behavior of the latter. To study the critical case c=c⋆, we develop a new isoperimetric inequality for the sampler’s stationary distribution by showing that the distribution is nearly log-concave. Conditions for rapid and torpid convergence of a class of generic binomial chain are also provided.
Evidence weight
Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40
| F · citation impact | 0.50 × 0.4 = 0.20 |
| M · momentum | 0.50 × 0.15 = 0.07 |
| V · venue signal | 0.50 × 0.05 = 0.03 |
| R · text relevance † | 0.50 × 0.4 = 0.20 |
† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.