Spin liquid nature in the Heisenberg J1−J2 triangular antiferromagnet

We investigate the spin-$\frac{1}{2}$ Heisenberg model on the triangular lattice in the presence of nearest-neighbor $J_1$ and next-nearest-neighbor $J_2$ antiferromagnetic couplings. Motivated by recent findings from density-matrix renormalization group (DMRG) claiming the existence of a gapped spin liquid with signatures of spontaneously broken lattice point group symmetry [Zhu and White, Phys. Rev. B 92, 041105 (2015); Hu, Gong, Zhu, and Sheng, Phys. Rev. B 92, 140403 (2015)], we employ the variational Monte Carlo (VMC) approach to analyze the model from an alternative perspective that considers both magnetically ordered and paramagnetic trial states. We find a quantum paramagnet in the regime $0.08\lesssim J_2/J_1\lesssim 0.16$, framed by $120^{\circ}$ coplanar (stripe collinear) antiferromagnetic order for smaller (larger) $J_2/J_1$. By considering the optimization of spin-liquid wave functions of a different gauge group and lattice point group content as derived from Abrikosov mean-field theory, we obtain the gapless $U(1)$ Dirac spin liquid as the energetically most preferable state in comparison to all symmetric or nematic gapped $\mathbb{Z}_{2}$ spin liquids so far advocated by DMRG. Moreover, by the application of few Lanczos iterations, we find the energy to be the same as the DMRG result within error-bars. To further resolve the intriguing disagreement between VMC and DMRG, we complement our methodological approach by the pseudofermion functional renormalization group (PFFRG) to compare the spin structure factors for the paramagnetic regime calculated by VMC, DMRG, and PFFRG. This model promises to be an ideal test-bed for future numerical refinements in tracking the long-range correlations in frustrated magnets.
Comment: Editors' Suggestion. 16 pages, 13 figures, 4 tables