Does categorizing scale scores with cutoff points affect hypothesis-testing results?

Abstract Background The aim of this study is to evaluate the hypothesis test results after categorizing the scale scores with cut-off points and to assess whether similar results would be obtained in that best represent the categories. Methods This cross-sectional study was conducted between March 15 and 20, 2023 via the Lime Survey. The questionnaire included questions about the sociodemographic and life characteristics of the participants and the Beck Depression Inventory II (BDI-II). Four groups (minimal, mild, moderate, severe depression) were formed using the cutoff points. Data analysis was performed with all participants and referred to as the conventional analysis group. Then, six subanalysis groups were determined to best represent the groups formed according to the BDI-II. In each BDI-II category, six subanalysis groups were created, including those between Q1–Q3 (IQR group), including those within ± 1 std, including those between 5p–95p (90% of the sample), including those between 2.5p–97.5p (95% of the sample). In addition, 100 different samples were randomly selected containing 50% of each group. Results Of the 1950 participants, 84.7% (n = 1652) were female and 15.3% (n = 298) were male. In terms of depression, it was observed that the significance varied in the analysis groups for sex (p = 0.039), medication use (p = 0.009) and age (p = 0.010) variables. However, these variables were not significant in some of the subanalysis groups. On the other hand, a p