Purpose: Intervertebral disc degeneration (IVDD), Modic changes, and fatty infiltration in the paraspinal muscles are possible causes of low back pain (LBP). Multifidus has been the most commonly blamed paraspinal muscle in the etiology of LBP. However, it contributes to 20% of the extensor moment on the lumbar spine. In the present study, we aimed to identify whether patients with LBP and asymptomatic subjects differed in terms of intervertebral discs, end‐plates, and fatty infiltration in their paraspinal muscles. Methods: Consecutive women and men, who visited the spine outpatient clinics with chronic LBP and had lumbar spine MRI for their LBP without leg pain were included. Asymptomatic subjects without LBP/leg pain for the last year were recruited. Modic changes, IVDD, and fatty infiltration in the paraspinal muscles were evaluated on lumbar spine magnetic resonance imagings of the patients with LBP and age‐, gender‐ and BMI‐matched asymptomatic controls. Results: Low back pain was closely associated with fatty infiltration in the paraspinal muscles at all lumbar levels whereas it had association with severe IVDD and Modic changes at lower lumbar levels. Multifidus at the lower lumbar levels was the fattiest paraspinal muscle in both asymptomatic subjects and patients with LBP. Patients with LBP had severe fatty infiltration in the erector spinae at the upper lumbar levels. Conclusion: Severe IVDD and Modic changes were more common at lower lumbar levels in patients with LBP. Both asymptomatic subjects and those with LBP had fatty multifidus at lower lumbar levels, whereas those with LBP had fatty infiltration in the erector spinae at upper lumbar levels. We suggest that fatty infiltration could have started in the multifidus. The erector spinae had greater contribution to the lumbar extension compared to the multifidus. Thus, LBP could develop when the quality of the erector spinae at the upper lumbar levels impairs due to fatty infiltration. [ABSTRACT FROM AUTHOR]