Alpha

One of the most frustrating things I have seen as a graduate student was a p-value of 0.06. In fact, I have seen this terrible number on more than one occasion. A p-value of 0.06 means that there is a six percent chance that the results I observed were a result of random chance if there really was no effect. Now, I personally think, when it comes to ecology at least, that six percent is low. Think about all the troubleshooting ecological research requires or how we are often limited to small sample sizes. How different is six percent really from 5 percent?

I am not the only one who is suspicious of the alpha values. I have listened to professors proclaim the death of traditional statistics and the end of an era of α < 0.05. Ecologists are moving towards Bayesian statistics. If you have been around FIU long enough you may remember the prestigious Glaser Seminar speaker in 2014 was Dr. David Anderson.

The other day I was surprised to see an article that was advocating for lowering alpha values to 0.005! What??? 

THEY ADDED AN EXTRA ZERO AFTER THE DECIMAL! 

I was shook.

After reading more I began to understand their reasoning. Benjamin et al. are focused on preventing false positives, or type I error. They are concerned with the lack of reproducible experiments. One example of this is in drug research where a new drug is shown to have a significant effect during the trials but then years later, after released, scientists discover that there is no effect or even a harmful effect. This wastes time and money and increases the general public’s mistrust of science.

Overall, Benjamin et al. did a thorough job at explaining their logic and took time to preemptively respond to anticipated objections. They make it clear that the change to 0.005 is a standard for evidence but should not be a standard for publication or policy. I think this might be what saves us ecologists from perishing.

Their justification for lowering alpha reminded me of how natural gas is often presented as the transition fuel. You know those people who say, “natural gas burns cleaner than coal or oil and will help buy time as we transition to renewable energy”. I interpreted Benjamin et al.’s conclusion along the lines of: lowering the alpha helps mitigate some of the systemic issues in modern scientific research but it is only a temporary fix for a much larger issue that will take more thought and debate.

What do you think about lowering alphas to 0.005? How would that change future FCE findings? 

I would love to hear your thoughts on the value of α and p! Comment below!