Microbial necromass plays an essential role in soil organic carbon (SOC) accumulation. Nevertheless, how microbial necromass carbon (C) concentrations and their contributions to SOC sequestration change from litter to mineral soil and what factors influence its accumulation remain poorly understood. To address this knowledge gap, we performed a field experiment to investigate the compositional distribution characteristics of microbial necromass C and its contributions to SOC sequestration in an oak forest (Quercus wutaishanica) litter-mineral soil profile of the Chinese Loess Plateau. The present study estimated the microbial necromass C concentrations based on the microbial cell wall's biomarker amino sugars. Our results demonstrated that microbial necromass C increased from the Oi1 to Oa layers but decreased from the Ah1 to AB horizons. The highest accumulation of microbial necromass C was found at the litter-mineral soil interface (i.e., total microbial necromass in the Oa layer was 39.5 Mg ha−1, and Ah1 was 22.8 Mg ha−1). The contribution of total microbial necromass C to SOC increased from Oi1 to Ah2. Specifically, the total microbial necromass C accounted for 40.7%, 47.7%, and 37.0% of the average bulk SOC in the Ah1, Ah2, and AB horizons of the oak forest mineral soil, respectively. The ratio of fungal to bacterial necromass C decreased from litter to mineral soil, indicating that the relatively higher bacterial necromass C increasingly accumulated in the deeper litter layers and upper mineral soil horizons. Fungal and bacterial necromass C increased with increasing labile organic C, nitrogen (N), and labile inorganic phosphorus (P), suggesting that higher easily accessible soluble nutrients lead to higher microbial biomass levels, which in turn lead to higher microbial necromass accumulation. Overall, our findings suggested that microbial demand for C or N influences the quantity of soluble nutrients and further lead to changes in microbial necromass C decomposition/accumulation.