I'm planning another post on a recent research paper, but in preparation I want to talk about statistics. Specifically, I want to talk about
Frequentist versus
Bayesian perspectives for the interpretation of experiments. I won't get technical, or go into the actual details about how calculations are done, but rather talk about philosophy.
My casual impression, based on reading papers to stay abreast of the field, is that most experiments use Frequentist methods in analysing their data. In this approach, discovery of new phenomena is based on disproving the
null hypothesis, the assumption that there is nothing to discover. In this sense, Frequentist methods are very
Popperian. Frequentists will argue that this ensures their methods are objective, which is more or less true.
1
The problem with Frequentism is that it has a tendency to be misinterpreted. For example, let's say in a particular experiment we can exclude the null hypothesis at 95% confidence level. What does that mean? It is tempting to interpret it as saying that there is a 95% probability that the null hypothesis is false. However,
this is wrong. The strictly correct statement is: if the null hypothesis is true, the probability of getting this experimental result is 5% or less.