COVID19 Factors We Should Consider/Current Events

[Mark Rippetoe]

What is the source of this figure?

See the work of John Ioannidis: Why Most Published Research Findings Are False

[Shiva Kaul]

One of the most overrated and inappropriately-titled papers, ever.

[jfsully]

See the work of John Ioannidis: Why Most Published Research Findings Are False

This is important work, and indeed the usefulness of science as a whole has been diluted by academic publishing requirements which give incentives for publishing volume over quality.

I think it's important to note, however, that it is possible to become literate in the basics of research and to be able to sort the wheat from the chaff. While you should never believe something just because it is prefaced by "studies have shown..." (in fact that phrase usually precedes something that has been oversimplified to the point of BS), it is true that there is lots of good work being done by smart people. The irreproducibility "crisis" is not random and does not affect all work equally.

The fact is that most solid research makes very incremental progress in complex ways that are uninteresting or impenetrable to lay people, and that true "holy crap" type discoveries in science are far too rare to satisfy the needs of journalists. So cool stuff doesn't get reported because complexity is no fun, and bogus stuff gets overhyped because it generates clicks and pageviews.

It is also important to note that the fact that "most published research findings are false," goes hand in hand with "most published research findings are not read or cared about by anyone, including other scientists." Within each field, people know which journals are decent, or spotty, or are crap, and are well aware of the existence of "vanity" journals which will apparently publish literally anything for a fee. There is an amusing/alarming example of a paper called Get Me Off Your Fucking Mailing List that was accepted for publication by a journal, apparently unread. I don't think they actually published it because the authors didn't want to pay the fee and felt their point had been made by its publication.

So, just like you should not dismiss insights from Rip and Starting Strength based on the overwhelming degree of bullshit that comes from the "fitness industry," you should not assume all "science" is bullshit based on the fact that so much of it is.

[Mark Rippetoe]

One of the most overrated and inappropriately-titled papers, ever.

If you have time, Shiva, would you please tell us why the paper is overrated and inappropriately titled.

[Johnsonville]

I don’t think this has been posted yet, but this article talks about the H3N2 virus that was a pandemic in ‘68-69. It killed over 100,000 Americans, yet life went on, and music festivals such as Woodstock occurred during the peak of the pandemic.
Woodstock Occurred in the Middle of a Pandemic – AIER

I’m surprised rip or any of the guys in their 60’s haven’t brought this up, or maybe did and I missed it. Anyways it is a good read and shows how the same generation who were going to Woodstock during a pandemic are now the pussies afraid to come within six feet of another human.

[Mark Rippetoe]

Couple of things:

Tanzania suspends laboratory head after president questions coronavirus tests - Reuters

Tanzania has suspended the head of its national health laboratory in charge of testing for the coronavirus and ordered an investigation, a day after President John Magufuli questioned the tests’ accuracy. Magufuli said on Sunday the imported test kits were faulty as they had returned positive results on a goat and a pawpaw — among several non-human samples submitted for testing, with technicians left deliberately unaware of their origins.

Dallas salon owner gets 7 days in jail for reopening in defiance of countywide restrictions | FOX 4 News Dallas-Fort Worth Shelley Luther is a hero.

Legacy Barbell - We were “shut down” by people inundating... | Facebook Henry Walker is a hero.



And:

I don’t think this has been posted yet, but this article talks about the H3N2 virus that was a pandemic in ‘68-69. It killed over 100,000 Americans, yet life went on, and music festivals such as Woodstock occurred during the peak of the pandemic.
Woodstock Occurred in the Middle of a Pandemic – AIER

I’m surprised rip or any of the guys in their 60’s haven’t brought this up, or maybe did and I missed it. Anyways it is a good read and shows how the same generation who were going to Woodstock during a pandemic are now the pussies afraid to come within six feet of another human.

Lifespan in the US in those days was 70 whereas it is 78 today. Population was 200 million as compared with 328 million today. It was also a healthier population with low obesity. If it would be possible to extrapolate the death data based on population and demographics, we might be looking at a quarter million deaths today from this virus. So in terms of lethality, it was as deadly and scary as COVID-19 if not more so, though we shall have to wait to see.

I didn't bring it up because I didn't remember it. Perhaps the media didn't have as big an ax to grind with President Nixon as they do with President Trump.

[Robert Santana]

Agreed.
Though there is another factor that doesn't get a lot of attention.
To the left, if your research is backed (paid for) by the government it is deemed righteous and good. However, if your research is funded by private (gasp, corporations) funds, then the research is by default tainted and cannot be trusted, as if a corporation would not do research in a scientifically rigorous way.
What nobody on the left will ever acknowledge is the monopoly Government and Academia have on research, if you want to get research money, your research must be in line with "the consensus" otherwise you have no hope of getting funded. In my humble opinion if government and academia are going to be doling our research funds then there needs to be a dollar for dollar ratio for research, meaning for every global warming conformist research grant there needs to be as much money going to the skeptic.

It's quite funny isn't it? If you work in academia the expectation is that you secure NIH grants and any other grant is deemed inferior. Last I checked all money is green, but I'll concede that no corporation can outspend the federal government. This is just another form of states trying to get in on the printing press. What is even funnier is that corporations exist because of the government, so is it really all that different? Both are incentivized to produced biased results, so I don't really subscribe the existence of a "black hat" vs "white hat" bias.

[Mainstream]

I don’t think this has been posted yet, but this article talks about the H3N2 virus that was a pandemic in ‘68-69. It killed over 100,000 Americans, yet life went on, and music festivals such as Woodstock occurred during the peak of the pandemic.
Woodstock Occurred in the Middle of a Pandemic – AIER

I’m surprised rip or any of the guys in their 60’s haven’t brought this up, or maybe did and I missed it. Anyways it is a good read and shows how the same generation who were going to Woodstock during a pandemic are now the pussies afraid to come within six feet of another human.

I too was too young to remember it, but this is an excellent point....

Modern society has become enslaved to the OMNIPRESENCE of the media and Science's ability to track and quantify....

I constantly hear phrases from meteorologist like the worst storm ever or this storm killed more people than any other....and the hyperbole is true yet totally false at the same time....in the old days tidal waves hit, earthquakes ripped, tornadoes destroyed, wild fires burned, but many areas were unpopulated and there were no news cameras or stations to instantly bring the destruction into your living room

Its like the second year the FBI uniform crime report came out and crime skyrocketed upward across America and what they mostly found out is that crime didn't really go up...its just the counting ,recording and reporting off it did....we became better at tracking it....

Same with autism rates in America...when we were little everybody knew kids in their schools that were a little off in some way....But now those kids end up in the child study teams psychologist office and are given a diagnosis...did autism go up or are we becoming better at identifying and tracking it....

Same thing here...viruses and bacteria kill millions every year but we didn't know the name , where it came from, where its going and who has it like we do now...I love when pseudo-intellectuals start telling me all the unique features of the coronavirus and why its so contagious and deadly...like all those germs in the past were not so dangerous and deadly...yes they were but we didnt have the ability to see them like we do now...

and we didnt have the pervasive 24/7 media that must bring a steady diet of continual death into every living room and personal device in the country

[Mainstream]

People need to remember that the Media's true job is TO SELL ADVERTISING...It accomplishes this by RATINGS....it has found that BAD NEWS has WAY MORE RATINGS than good news...thats just the way people are....if you tell people its a bright sunny day and love is in the air...THEY TURN THE TV SET OFF AND GO LIVE THEIR LIVES....but if you tell them a Tornado, Earthquake, terrorists, disease is Coming to kill them...they stayed glued to the set....so every news person just knows this...they love a good Horror story because they know the money will flow....they don't necessarily lie...they would lie but they are afraid of getting caught in a lie and people stop watching them...so they don't outright lie...but they just make sure they are constantly telling you BAD NEWS the bigger the bad news...the BETTER!

See what I mean..it never stops....A mutant coronavirus has emerged, even more contagious than the original, study says

[Shiva Kaul]

If you have time, Shiva, would you please tell us why the paper is overrated and inappropriately titled.

I think reproducibility is a problem, but mostly for unsexy, non-statistical reasons: overstating conclusions, omitting details, fudging data, etc. Ioannidis discusses these in his section on "bias", but that part of his model is completely tautological and does not explain anything. Literally, let u be the probability that the authors are lying; then, as u increases, be astounded that the false publication rate increases!

The phenomenon he actually investigates: if there are many independent research teams, each with a small sample of data, and a result is published if and only if it is positive, and if legitimate positive results are rare - then there will be many published false positives. At a high level, this is known as the multiple testing problem, and has been known for decades. (That's why this paper is published in PLOS Medicine, not a statistical venue.) His point is that control of false discovery may be done within individual studies, but is not done at the meta-level of multiple studies. The novel contributions are the quantitative estimates obtained by carrying out the napkin math.

Unfortunately, these aren’t good enough to convince me that multiple testing (rather than the "unsexy" stuff) is to blame for the replication crisis. Are there really that many teams working on the exact same utterly hopeless study? His quantitative estimates depend critically on the prior probability that a study should yield a positive - what he calls R/(R+1). This value is unknown, and there is no attempt to estimate them from data. He just uses values that he thinks sound reasonable (Table 4). But I wouldn't trust his intuition: his example value of R=10^{-4}, in a GWAS, is misguided. He considers every comparison within a single study as a study in its own right. In reality, one would employ a statistical method which accounts for the multiple testing. In other words, he is repeating the same intra-study/inter-study mistake that motivates the paper! (As an aside, I am dismayed by the equality R = R/(R+1) = 10^{-4}).

In machine learning, there are concerns about reproducibility and multiple testing. (Conducting and submitting lots of junk studies is similar to trying lots of predictors on a benchmark, hoping one does well by chance). Well, empirically, multiple testing doesn't seem to be as big of a deal as we thought. It is crucial to look at actual data.

Oh well. Ioannidis lets his toy model drag him into baseless conclusions. For example, consider his claim that "the hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true." Conventional understanding is that more scrutiny, bigger datasets, and less stress about funding all lead to better research. But no, the toy model (which doesn’t account for any of those phenomena) says otherwise!

Catchy titles are admissible in sleepy technical fields, like the one I work in, for authors to drum up general interest in their work. This paper does not qualify. The title is absurdly aggressive for these brief musings. "Multiple testing in scientific publication" would have been reasonable. Of course, no one here would have cited that. Because this paper brings popular attention to reproducibility and p-values, lots of statisticians don't scrutinize it. I'm not that kind.