Member
Bad Statistics in Sports Science
Interesting take on why so much BS comes out in the sports science journals.
How Shoddy Statistics Found A Home In Sports Research | FiveThirtyEight
TMPHBITEU
Pull quote:
In doing so, those spreadsheets often find effects where traditional statistical methods don’t. Hopkins views this as a benefit because it means that more studies turn up positive findings worth publishing. But others see it as a threat to sports science’s integrity because it increases the chances that those findings aren’t real.
Member
Interesting that, according to the review, Hopkins approach to statistics combines frequentist and Bayesian. Yet neither of those are given pride of place in Richard Hamming's significant work on practical probability "The Art of Probability". In fact, the frequentist approach is consider and dismissed as insufficient by Hamming. Instead, Hamming uniquely posits symmetry as the foundation of probability (and thus stats). Here's a nice review to that point:
The Hamming Lectures on Probability
✓ Member
The most egregious part of all this is that classical null hypothesis testing isn't necessarily all that strong to begin with. The 5% alpha convention is the result of an off-hand comment that was taken as gospel, and confidence intervals are typically far more useful. On top of all that, there is a thin line between data-mining and structural deficiencies that is made even worse by the poor setup (e.g. randomizing legs for leg extensions when n=too small). The point of statistical analysis is to put well thought out theories to the test and see if they stand up to rigour, not to lower the hurdle for them. The problem is not typically that we know something is true and we just can't show it statistically; its that we get spurious results too often and can't be sure if what wasn't reject is real. There's enough ways to justify BS already. If this keeps up, Hopkins will give Baltimore a bad reputation...
CUWdNiL.jpg
Member
I was speaking to a PT recently on how sports medicine and medical practice creates treatments and he basically admitted that correlative evidence is an industry norm (duh), but he was like "what do you want us to do? Not use it if it helps people?" Obviously there's an issue with practitioners and those who write "the literature" forgetting the correlative nature of this information. Whether that's due to simple inertia or something else, I have no idea, but people need to stop saying "things are this way" when they have no way of knowing if they actually are.
Member
Crom's Axiom:
In all "science" there is really only one statistic that matters: "The probability that your statistical inferences will be reviewed by someone who regards himself as a greater statistical expert than yourself."
Corollary: The fact that you have resorted to using statistical analysis means that no matter how perfectly you have designed your experiment, and how rigorously you have correctly applied advanced statistical mathematics, you could still be 100% WRONG.
Carry on.
Member
。。。And that quote is why you can't blindly trust PTs. The data don't actually support that the treatment helps people. Sometimes an unproven treatment or supplement can be justified because it will do no harm even if doesn't work. But, much of the world's billions of dollars of snake oil is sold using the same reasoning.Originally Posted by tfranc
Biological science in general is rife with irreproducible studies (about 60%); even to the point where some researchers are gaining prestige by disproving the claims of studies that were considered groundbreaking truths for years.
Member
This is stunning and unequivocally reduces the statistical analysis utilized in exercise science journals to the level of The Onion.
I’m speechless to read that this claptrap is being accepted in this discipline. I’m fairly savvy in interpreting statistical data as I minored in biostatistics when I got my masters in public health degree (specializing in epidemiology). I also ran our journal club for medical students and residents when I was an assistant professor at the UCLA’s medical school. My take on this is that exercise journals don’t get to determine which statistical applications they wish to use and then arbitrarily claim that their approach isn’t merely expedient but rather, is mainstream.
What will their editors do when they’re called out by the legions of actual, bona fide statisticians aching to make themselves champions of science?
Posting Permissions
