Background: Polypharmacy and potentially inappropriate medications (PIMs) are prominent prescribing issues in elderly patients. This study was to investigate the different prevalence of PIM use in elderly inpatients between 65-79 years of age and 80 years or older, who were discharged from Geriatric Department in West China Hospital. Methods: A large-scale cohort of 1796 inpatients aged 65 years or over was recruited. Respectively, 618 patients were 65-79 years and 1178 patients were 80 years or older. Updated 2012 Beers Criteria by the American Geriatric Society was applied to assess the use of PIM among the investigated samples. Results: A review of the prescribed medications identified 686 patients aged 80 years or older consumed at least one PIM giving a rate of 58.2%. Conversely, 268 (43.4%) patients aged 65-79 years consumed at least one PIM (χ2 = 40.18, P < 0.001). Patients aged 80 years or older had higher hospitalization expenses, length of stay, co-morbidities, medical prescription, and mortality than patients aged 65-79 years (all with P < 0.001). Patients aged 80 years or older were prescribed with more benzodiazepines, drugs with strong anticholinergic properties, megestrol, antipsychotics, theophylline, and aspirin. In multiple regression analysis, PIM use was significantly associated with female gender, age, number of diagnostic disease, and number of prescribed medication. Conclusions: The finding from this study revealed that inpatients aged 80 years or older encountered more PIM use than those aged 65-79 years. Anticholinergic properties, megestrol, antipsychotics, theophylline, and aspirin are medications that often prescribed to inpatients aged 80 years or older. Doctors should carefully choose drugs for the elderly, especially the elderly aged 80 years or older.