• In this work, we investigate the in- and out-of-plane shifts of the PSHE based on a full-wave theory, revealing that they have the same physical mechanism. • The in- and out-of-plane displacements of the PSHE are determined by momentum-dependent Pancharatnam-Berry (PB) phase, where the in-plane wavevector k x -dependent phase gradient determines the in-plane shift, and the transverse wavevector k y -dependent phase gradient dictates the out-of-plane shift. • We discover that the intrinsic OAM (IOAM) and extrinsic OAM (EOAM) carried by incident off-axis vortex beams can respectively convert into the EOAM and IOAM of abnormal modes in the transmitted beam. • The OAM generated during the spin-reversal process, together with the OAM carried by the incident beam, collectively determines the in- and out-of-plane displacements. • When off-axis vortex beams are incident, the competition between the IOAM and EOAM is more complex than in those without OAM. Our study unifies in- and out-of-plane shifts into a single theoretical framework, which offers an alternative perspective on the spin–orbit interaction of light. The photonic spin-Hall effect (PSHE) is a result of the interaction of spin and orbital angular momentum (OAM). However, previous research on the PSHE has primarily focused on out-of-plane (transverse) shifts. In fact, it encompasses both out-of-plane shifts and relatively weaker in-plane shifts, and the latter of which plays a crucial role in controlling the direction of spin separation. Here, we investigate the in- and out-of-plane shifts of the PSHE based on a full-wave theory, revealing that they have the same physical mechanism. The in- and out-of-plane displacements of the PSHE are determined by a momentum-dependent Pancharatnam-Berry (PB) phase, where the in-plane wavevector k x -dependent phase gradient determines the in-plane shift, and the transverse wavevector k y -dependent phase gradient dictates the out-of-plane shift. We discover that the intrinsic OAM (IOAM) and extrinsic OAM (EOAM) carried by incident off-axis vortex beams can respectively convert into the EOAM and IOAM of abnormal modes in the transmitted beam. The OAM generated during the spin-reversal process, together with the OAM carried by the incident beam, collectively determines the in- and out-of-plane displacements. When off-axis vortex beams are incident, the competition between the IOAM and EOAM is more complex than in those without OAM. Our study unifies in- and out-of-plane shifts into a single theoretical framework, which offers an alternative perspective on the spin–orbit interaction of light. [ABSTRACT FROM AUTHOR]