Search

Convolutional neural networks (CNNs) have been employed along with Variational Monte Carlo methods for finding the ground state of quantum many-body spin systems with great success. In order to do so, however, a CNN with only linearly many variational parameters has to circumvent the ``curse of dimensionality'' and successfully approximate a wavefunction on an exponentially large Hilbert space. In our work, we provide a theoretical and experimental analysis of how the CNN optimizes learning for spin systems, and investigate the CNN's low dimensional approximation. We first quantify the role played by physical symmetries of the underlying spin system during training. We incorporate our insights into a new training algorithm and demonstrate its improved efficiency, accuracy and robustness. We then further investigate the CNN's ability to approximate wavefunctions by looking at the entanglement spectrum captured by the size of the convolutional filter. Our insights reveal the CNN to be an ansatz fundamentally centered around the occurrence statistics of $K$-motifs of the input strings. We use this motivation to provide the shallow CNN ansatz with a unifying theoretical interpretation in terms of other well-known statistical and physical ansatzes such as the maximum entropy (MaxEnt) and entangled plaquette correlator product states (EP-CPS). Using regression analysis, we find further relationships between the CNN's approximations of the different motifs' expectation values. Our results allow us to gain a comprehensive, improved understanding of how CNNs successfully approximate quantum spin Hamiltonians and to use that understanding to improve CNN performance., Comment: 27 pages, 6 figures. Yilong Ju and Shah Saad Alam contributed equally to this work

Adversarial Attacks are still a significant challenge for neural networks. Recent work has shown that adversarial perturbations typically contain high-frequency features, but the root cause of this phenomenon remains unknown. Inspired by theoretical work on linear full-width convolutional models, we hypothesize that the local (i.e. bounded-width) convolutional operations commonly used in current neural networks are implicitly biased to learn high frequency features, and that this is one of the root causes of high frequency adversarial examples. To test this hypothesis, we analyzed the impact of different choices of linear and nonlinear architectures on the implicit bias of the learned features and the adversarial perturbations, in both spatial and frequency domains. We find that the high-frequency adversarial perturbations are critically dependent on the convolution operation because the spatially-limited nature of local convolutions induces an implicit bias towards high frequency features. The explanation for the latter involves the Fourier Uncertainty Principle: a spatially-limited (local in the space domain) filter cannot also be frequency-limited (local in the frequency domain). Furthermore, using larger convolution kernel sizes or avoiding convolutions (e.g. by using Vision Transformers architecture) significantly reduces this high frequency bias, but not the overall susceptibility to attacks. Looking forward, our work strongly suggests that understanding and controlling the implicit bias of architectures will be essential for achieving adversarial robustness., Comment: 23 pages, 11 figures, 12 Tables