Plastic mulching (PM) is regarded as a promising way to increase crop production. However, its reported that plastic mulching may decrease soil organic carbon content and been unfavorable for sustainable agricultural production. Confirmation of the effects of plastic mulching on soil water and organic carbon balances in a long term, including finding efficient ways to improve these balances, is crucial for the sustainability of agricultural production in semiarid rain-fed areas. We conducted field experiment (2010–2019), with spring maize and wheat as tested crops, using four treatments of plastic mulching (PM), plastic mulching with supplementary irrigation (PMI), plastic mulching with organic fertilizer application (PMO) and without mulching (CK). The results of this 10-year field experiment showed that PM with supplementary irrigation and organic fertilizer application not only significantly increased yield and Water productivity (WP) of wheat and maize, but also had a positive effect on soil water budget. The soil water budget of PMI, PMO and PM significantly increased by 120.4%, 96.1% and 105.8% in wheat and by 149.7%, 28.2% and 53.6% in maize, compared with CK, respectively. The improved soil water condition resulted in a significant increment of yield, PMI, PMO and PM increased yield by 122.8%, 89.7% and 67.0% for wheat, by 237.8%, 183.0% and 148.4% for maize, respectively, compared with CK. The 10 years of continuous PM significantly decreased soil organic carbon content (SOC) in 0–10, 10–20, 30–50 and 50–70 cm profiles by 22.5%, 19.1%, 15.6% and 15.3% for wheat, but had no significant effect for maize. The PMO significantly increased soil organic carbon content in 0–30 cm profiles for both wheat and maize, but the soil organic carbon budgets were negative for all four treatments, the PMO accelerated soil organic carbon loss for maize but had little effect for wheat compared with PM and CK. However, PMI, PMO and PM significantly increases crop bio-carbon production, resulted in the positive total carbon budget and significantly increased by 25.0, 15.0, 11.5 Mg ha−1 in wheat and 105.1, 74.1, 74.0 Mg ha−1 in maize, respectively, as compared with CK. These results suggested that the soil organic carbon budget differed for the two crops, also affected by water or organic carbon supplementation. A more appropriate crop rotation system with organic fertilizer application should be developed, to increase crop production and soil quality under plastic mulched condition in such semiarid rain-fed areas.