Back to Search Start Over

Nonparametric cluster significance testing with reference to a unimodal null distribution.

Authors :
Helgeson, Erika S.
Vock, David M.
Bair, Eric
Source :
Biometrics. Dec2021, Vol. 77 Issue 4, p1215-1226. 12p.
Publication Year :
2021

Abstract

Cluster analysis is an unsupervised learning strategy that is exceptionally useful for identifying homogeneous subgroups of observations in data sets of unknown structure. However, it is challenging to determine if the identified clusters represent truly distinct subgroups rather than noise. Existing approaches for addressing this problem tend to define clusters based on distributional assumptions, ignore the inherent correlation structure in the data, or are not suited for high‐dimension low‐sample size (HDLSS) settings. In this paper, we propose a novel method to evaluate the significance of identified clusters by comparing the explained variation due to the clustering from the original data to that produced by clustering a unimodal reference distribution that preserves the covariance structure in the data. The reference distribution is generated using kernel density estimation, and thus, does not require that the data follow a particular distribution. By utilizing sparse covariance estimation, the method is adapted for the HDLSS setting. The approach can be used to test the null hypothesis that the data cannot be partitioned into clusters and to determine the optimal number of clusters. Simulation examples, theoretical evaluations, and applications to temporomandibular disorder research and cancer microarray data illustrate the utility of the proposed method. [ABSTRACT FROM AUTHOR]

Details

Language :
English
ISSN :
0006341X
Volume :
77
Issue :
4
Database :
Academic Search Index
Journal :
Biometrics
Publication Type :
Academic Journal
Accession number :
154459211
Full Text :
https://doi.org/10.1111/biom.13376