We examine two set-ups that reveal different operational implications of path-phase complementarity for single photons in a Mach-Zehnder interferometer (MZI). In both set-ups, the which-way (WW) information is recorded in the polarization state of the photon serving as a 'flying which-way detector'. In the 'predictive' variant, using a fixed initial state, one obtains duality relation between the probability to correctly predict the outcome of either a which-way (WW) or which-phase (WP) measurement (equivalent to the conventional path-distinguishability-visibility). In this set-up, only one or the other (WW or WP) prediction has operational meaning in a single experiment. In the second, 'retrodictive' protocol, the initial state is secretly selected for each photon by one party, Alice, among a set of initial states which may differ in the amplitudes and phases of the photon in each arm of the MZI. The goal of the other party, Bob, is to retrodict the initial state by measurements on the photon. Here, a similar duality relation between WP and WW probabilities governs their simultaneous guesses in each experimental run. [ABSTRACT FROM AUTHOR]