A long-term fertilizer experiment was set up on a calcareous chernozem soil with a wheat-maize-maize-wheat crop rotation, as part of the National Long-Term Fertilization Experiments (NLTFE) Network, set up with the same experimental pattern under different soil and agro-climatic conditions in Hungary. The effect of P fertilization on the soil, on maize yields, and on leaf P and Zn contents in the flowering stage were examined in the trials. In certain years, foliar zinc fertilizer was applied, in order to prove that yield losses due to P-induced Zn deficiency can be compensated by Zn application. Calcium-ammonium nitrate, superphosphate and 60% potassium chloride were used as NPK, and Zn-hexamine (in 1991) and Zn-sulphate (in 2006) as foliar Zn fertilizers. In the years since 1970, averaged over 36 maize harvests, treatments N3P1K1 and N4P1K1, involving annual rates of 150 to 200 kg ha−1 N, 100 kg ha−1 K2O and 50 kg ha−1 P2O5, gave the highest yields (8.3 t ha−1 grain on average). As the years progressed, treatments exceeding 50 kg ha−1 P2O5 a year were found to have an increasingly unfavourable effect. Based on the yields of ten cycles (36 maize years), variants P2, P3 and P4 resulted in 16–30–45 t ha−1 grain yield losses in comparison to variant P1. Investigations carried out in 1987, 1991 and 2006 showed that the leaf Zn content on plots with more than 150 to 200 mg kg−1 AL (ammonium lactate)-soluble P2O5 (over 30 mg kg−1 Olsen-P) dropped below 15 mg kg−1 and the P/Zn ratio rose to above 150 or even 250 in the flowering stage in two years. As a consequence of P-induced Zn deficiency, maize grain yields fell by 2 t ha−1 in two of the years investigated and by almost 5 t ha−1 in one year at the P4 level (200 kg ha−1 P2O5 year−1), in comparison to the P1 variant (50 kg ha−1 P2O5 year−1). When 1.2 kg ha−1 foliar Zn was applied in the form of zinc hexamine, 1.7 to 1.8 t ha−1 maize grain yield surpluses were obtained on plots with higher P levels in 1991. In 2006 the P-induced Zn deficiency caused unexpectedly high (almost 5 t ha−1) grain yield losses on plots with higher P levels, so the maize grain yield surpluses obtained in response to 1.2 kg ha−1 foliar Zn application, in the form of zinc sulphate, were as high as 1.6 to 3.8 t ha−1. The data clearly indicate that maize yields are impeded by both poor and excessive P status. Soil and plant analysis may be useful tools for monitoring the nutritional status of plants.