To join this seminar virtually: Please request Zoom connection details from ea [at] stat.ubc.ca
Abstract: Clustering analysis is an area of unsupervised learning that carries great potential in analyzing the increasingly immense amount of available data. Whereas it is straightforward to apply clustering on datasets, it is much less so when it comes to assessing the quality of the clustering results, since we often lack some notions of ground truth. This, coupled with the fact that two samples collected in diļ¬erent settings may not be the same and hence the respective results may exhibit variations, makes this problem both important and challenging at the same time.
In his 1985 paper Confdence Limits on Phylogenies: An Approach Using the Bootstrap, Felsensteins proposes the idea of the bootstrap probability as a tool to quantify this uncertainty in the context of hierarchical clustering. It has since gained much appreciation and become one of the main tools for this problem, so much so that it is often referred to as a p-value.
We review the 1996 paper by Efron et al., Bootstrap confidence levels for phylogenetic trees, which argues that the bootstrap probability lacks a framework of a model and a null hypothesis for it to be formalized into a statistical p-value. We then explore an alternative interpretation based on the idea of bootstrap as a drop-in replacement for the sampling probability if we have access to the population. We run simulations to explore the performance of the bootstrap probability and how it tracks the target sampling probability.
We find that while the bootstrap probability is indeed a very elegant and simple-to-calculate metric, there are situations in which we can have great confidence in the results and others whereas we should be less so and further analyses may be necessary.