Departmental Seminar: Why all p-values should be banned (and other norms you can ignore)

Science is difficult. To do good science researchers need to know about philosophy of science, learn how to develop theories, become experts in experimental design, study measurement theory, and understand the statistics they use to analyze their data. I used to rely on norms when I made choices when I did research. From the way I phrase my research question, to how I determine the sample size for a study, to the statistical tests I performed, my justifications were typically ‘this is how we do it’. In this talk I will explain that, regrettably, almost all the norms we rely on are wrong. I will provide present some ways to justify aspects of the research cycle, such as sample sizes and choices for statistical tests, and discuss the (im)possibility of individually accumulating sufficient knowledge to be able to justify all important decisions in the research you do, and the possible benefits of a more collaborative science.