Dissolved and colloidal phosphorus (P) represent the mobile P fractions in soils, but their role in P cycling in forests is still largely unclear. In this study of four calcareous forest soil profiles, the elemental compositions of different size fractions of water dispersible colloids (WDC) were investigated by asymmetric field flow fractionation. Nuclear magnetic resonance spectroscopy (NMR) was applied to identify the organic P compounds in soils, WDC, and soil solutions. Carbon was the dominant element in WDC of all soil horizons, including mineral soils that were rich in Ca or Si. Although chemical composition of P varied dramatically with increasing depth, the colloidal P composition remained unchanged. This contrasting difference between mineral soil and its WDC fraction indicated that the colloids were not locally generated but originated from the overlying organic soil horizons. Carbonate minerals were unlikely involved in colloid formation under acidic condition. Instead, Ca2+ probably drove colloid formation by bridging organic matter, including P-containing compounds released from litter degradation. Colloid formation was influenced by climate, vegetation, and soil characteristics. No dissolved P was detected in deeper mineral soil horizons due to efficient retention by Ca minerals. Colloidal P was still present in deeper soil layers and thus of significance for potential P vertical transfer.