Doped Mott insulators exhibit some of the most intriguing quantum phases of matter, including quantum spin liquids, unconventional superconductors and non-Fermi liquid metals 1-3 . Such phases often arise when itinerant electrons are close to a Mott insulating state, and thus experience strong spatial correlations. Proximity between different layers of van der Waals heterostructures naturally realizes a platform for experimentally studying the relationship between localized, correlated electrons and itinerant electrons. Here we explore this relationship by studying the magnetic landscape of tantalum disulfide 4Hb-TaS 2 , which realizes an alternating stacking of a candidate spin liquid and a superconductor 4 . We report on a spontaneous vortex phase whose vortex density can be trained in the normal state. We show that time-reversal symmetry is broken in the normal state, indicating the presence of a magnetic phase independent of the superconductor. Notably, this phase does not generate ferromagnetic signals that are detectable using conventional techniques. We use scanning superconducting quantum interference device microscopy to show that it is incompatible with ferromagnetic ordering. The discovery of this unusual magnetic phase illustrates how combining superconductivity with a strongly correlated system can lead to unexpected physics., (© 2022. The Author(s), under exclusive licence to Springer Nature Limited.)