Agricultural land use is an important source of CO2 emissions. Therefore, taking the CO2 emissions of China’s agricultural land use during 1995–2020 as a case, we firstly calculated its composition and analyzed the spatiotemporal evolution characteristics. Then, the Tapio decoupling model and logarithmic mean Divisia index (LMDI) were, respectively, used to identify the decoupling relationship between the CO2 emission change and economic growth, and analyze the driving factors for CO2 emissions. (1) The CO2 emissions of China’s agricultural land use were composed of two main phases (fluctuating growth phase (1995–2015) and rapid decline phase (2016–2020)). The total CO2 emissions exhibited a non-equilibrium spatial distribution. The inter-provincial CO2 emissions differences first expanded and then shrank, but the inter-provincial differences of CO2 emissions intensity continuously decreased. (2) The total CO2 emissions of China’s agricultural land use increased from 50.443 Mt in 1995 to 79.187 Mt in 2020, with an average annual growth rate of 1.82%. Fertilizer, agricultural diesel and agricultural (plastic) film were the main sources of anthropogenic agricultural-land-use CO2 emissions. Controlling the use of fertilizer and agricultural diesel and improving the utilization efficiency of agricultural (plastic) film could be an effective way to reduce CO2 emissions. (3) The Tapio decoupling relationship between the CO2 emission change and economic growth was a weak decoupling state during 1995–2015 and a strong decoupling state during 2016–2020. This result indicates that China’s agricultural land use can be effectively controlled. (4) The agricultural economic level is the decisive factor in promoting CO2 emissions increase, and its cumulative contribution was 476.09%. Inversely, the CO2 emission intensity, agricultural structure and agricultural labor force were three key factors, with cumulative contributions of −189.51%, −16.86% and −169.72%, respectively. Collectively, based on the findings obtained from the present research, we have proposed some suggestions to promote the sustainable use of agriculture lands in China.