starting strength gym
Page 1 of 4 123 ... LastLast
Results 1 to 10 of 31

Thread: Off-topic discussion of the problem of "peer review" in journals

  1. #1
    Join Date
    Jul 2007
    Location
    North Texas
    Posts
    53,559

    Default Off-topic discussion of the problem of "peer review" in journals

    • starting strength seminar april 2024
    • starting strength seminar jume 2024
    • starting strength seminar august 2024
    Here is a comment on FB:

    John Churchill Rip, im a big fan of and youve taught me a lot about getting strong. so im surprised and disappointed to read an anti-science quote in the lead of your column. Vox Day? A blogger ranting about how all science is BS because of one shite journal's lax oversight? C'mon Rip.
    The lay public has no idea about the realities of journal publication.

  2. #2
    Join Date
    May 2015
    Location
    Melbourne
    Posts
    8

    Default

    OK, I'll bite.
    1) Are academic scientists stupid for providing journal with content, editing and review for free, then buying their own work back with other peoples money through expensive subscription charges? Yes.
    2) Is much of the work published bullshit designed to be a "publication statistic" and lead to personal advancement? Yes.
    3) Does this make "Starting Strength" science? No.

    Where is the data? Of all the people coached, all the progress journaled where is the data? You have one "study" published on the site that seems to convey that starting strength leads to non compliance. i.e. not doing the program. Or is that not recording in the logs doing the program? Who knows? Not you. But it might work for the very small percentage of people who managed to do the program AND record their progress. Maybe, but many of them weren't beginners anyway. Or at least you're not sure they were.

    You have a nice book, well written, good pictures, some Newtonian explanations of the lifts. But that is all you have. It is not the type of science you purport it to be. It isn't a transparent presentation of all the trial and error (engineering) that led to its creation, it is a narrative fiction you choose to believe after the experimentation. Hypothesizing after the fact.

    Let the flaming begin.

  3. #3
    Join Date
    May 2016
    Location
    Finland
    Posts
    327

    Default

    Quote Originally Posted by Mark Rippetoe View Post
    Here is a comment on FB:
    The lay public has no idea about the realities of journal publication.
    As someone who has participated in peer review in high ranking journals and published in some, I have to agree with this. The journal publication system is flawed, and deeply so. Peer review -- meaning, peer review as it is conducted in many if not most journals -- is broken in many ways. It is true that in many journals the oversight of reviewers is often not properly conducted.

    This means that, on average, a peer-reviewed article has a rather high probability of being accepted in a pretty reputable journal if 1) It caters to the expectations of the current trends and 2) if it is of reasonable technical quality, i.e., it goes through the motions of seemingly high-quality research. This means Silly Bullshit(tm) gets published more often than we'd like. To make matters worse, a high quality research paper that goes against one received wisdom or another, has a pretty high probability of being rejected.

    So, the peer review-process, which is supposed to act as a filter or guard against Silly Bullshit, does not filter it as it is supposed to.

    Having said that, the process still gives a higher probability of publication for Good and Honest Research than Silly Bullshit, when it comes to most fields. These include fields such as Computer Science (there are pretty bad journals out there, but most are pretty good), Physics, Chemistry, most engineering fields, most of Biology etc. In experimental sciences, the peer review process is not the last line of defence anyhow; if people systematically fail to replicate the results of already published research, papers do get retracted and people lose their positions. Frauds get outed all the time.

    To the lay public that wants to educate themselves about the problems of peer review, this poses a problem. When they perceive Peer Review as broken, they tend to overcompensate, and draw the conclusion that Science is broken.

    Vox Day is a really really bad example (or good, in the sense that he is extremely informative). Look at his stance on evolutionary biology -- something I know a little about (not my field of science, I am a mathematician and computer scientist, but I've dabbled around). Because some people make errors in peer review, he sees this as *evidence* that the world was created 6000 years ago or so. (I may be misrepresenting and overstating his stance; this is for the purpose of illustrating his logic). Quoting a person who -- by my humble opinion -- is in dire need of psychiatric intervention is a bit of a disappointment to me.

    Not that I disagree with what the article otherwise says. Evidence comes in many many forms, and the sort of evidence that you and all the other people who have seen Starting Strength in action can testify to, is compelling. It is a fundamental misunderstanding of what "science" or "scientific knowledge" is to write this evidence off as something irrelevant. It is an important set of concepts that has been pushed aside in the exercise science community, and they do need an overhaul of their views. Also, the concept of peer review needs a redesign, but these things take time, people are conservative, institutions change very slowly etc ad nauseam.

    But please, Rip, you're better than this. Vox Day, for crying out loud.

  4. #4
    Join Date
    Jul 2007
    Location
    North Texas
    Posts
    53,559

    Default

    Quote Originally Posted by rohano View Post
    Where is the data?
    What is data? Be precise.

    Quote Originally Posted by Tiedemies View Post
    To the lay public that wants to educate themselves about the problems of peer review, this poses a problem. When they perceive Peer Review as broken, they tend to overcompensate, and draw the conclusion that Science is broken.
    Not so much the lay public, but people with bachelor's degrees fail to understand that Science is an investigative process and peer-review is an editorial process. Sometimes PR catches problems, but usually it is used to confer legitimacy on master's theses. Rohano here is a great example of a person who thinks that Isaac Newton was not doing science because his "data" was not peer-reviewed and published in Nature.

    Vox Day is a really really bad example (or good, in the sense that he is extremely informative). Look at his stance on evolutionary biology .....

    But please, Rip, you're better than this. Vox Day, for crying out loud.
    You must realize that I don't care what you think about Vox Day's theology, any more than I care about your AGW Theology. People are free to pursue their faith in any way they deem necessary. I found the quote useful in this context, and I used it without consulting you not because I knew you'd be disappointed, but because I found it useful. As usual, your feelings and opinions are your own business.

  5. #5
    Join Date
    May 2014
    Location
    Ozarks
    Posts
    1,283

    Default

    People really get their nickers twisted over this shit, eh? Maybe philosophy, basic critical thinking and logical reasoning should be taught in schools again- and at younger ages. And maybe for longer than a semester or two. I took nothing more away from that statement than "humans, being fallible by their nature, are subject to having their various endeavors affected to differing degrees by the consequences of said fallibility."

    I've always said that healthy skepticism doesn't make your wrong in your doubt. It doesn't make you right, necessarily, but it doesn't make you wrong. And, correct me if I'm mistaken, but isn't the position of the SSCs and yourself, Rip, that people are more than encouraged to attempt to prove your model is NOT, in fact, the most efficient at increasing force production capacity in a general sense in the trainee when applied correctly and with adequate compliance?

  6. #6
    Join Date
    Oct 2016
    Location
    Seattle, WA
    Posts
    143

    Default

    It is true that the lay public doesn’t know anything about publishing science; that’s why Vox Day has an audience in the first place. Unfortunately, he doesn’t know anything about it either, so it’s like the loudest blind person leading the rest of the blind for no better reason than because he is the only one they hear.

  7. #7
    Join Date
    Oct 2016
    Location
    Seattle, WA
    Posts
    143

    Default

    Vox Day’s comment makes it clear that he forms and expresses strong opinions about topics on which he has very limited, superficial knowledge—a quick ‘google scholar’ search shows that he has never published a scientific article. He didn’t even provide evidence or examples for any of his claims. He is the pretty much the exact opposite of the type of person I respect.

    In the peer-review process, the reviewers (experts in the field) check whether the topic of the article is appropriate for the journal, whether the methods indeed address the topic, whether the results justify the conclusions, and more. What do you mean reviewers looking at the data isn’t the norm? The figures and tables in a publication are the data. Reviewers absolutely look at figures/tables because they are in the field and the data is useful to them. The peer-review process isn’t perfect—some reviewers have biases, some cases of over interpretation are missed, etc. However, it’s kind of like democracy in that it’s the best system we have.

    The scientific community accomplishes major things that benefit the general public: research on cholesterol regulation led to the development of statins, which extend the lives of people with a predisposition to elevated cholesterol by years; research on the cardiac electrical system led to the development of pacemakers that extend the lives of people with a predisposition to arrhythmias by years; research on HIV replication led to the development of drugs that give HIV patients a life expectancy that is ~ equivalent to everyone else...

    None of these accomplishments came from a single paper or even a single lab. The point outsiders are missing is that the scientific literature is a conversation with others in the field. You don’t publish your paper because it is THE CURE for whatever disease you’re studying. You publish your paper because its data and implications are useful to other scientists who are working on the same problem. You address possible implications or applications of your research in the ‘discussion’ to give others ideas about future directions not for ‘propaganda’. It’s a large community with similar goals working on problems that are far too complex and difficult for a single paper to solve.

    Yes. I did get triggered. I have never really been in an internet argument before. However, I have huge admiration for Rip and was crushed to seem him propagate an anti-science message.

  8. #8
    Join Date
    Mar 2014
    Location
    St. Louis, MO
    Posts
    45

    Default

    Peer review might be more significant if journals disclosed the reviewers of the paper. In my brief experience with exercise science research I observed that many study authors would choose to send their papers to their friends at other schools for review. Many times, their friends were co-authors on previously published papers. Perhaps, and hopefully, other journals have higher standards than those of exercise science.

    If a reviewer approves a paper for publication then they should have the courage to put their name on that paper stating that they approved it.

  9. #9
    Join Date
    Jul 2007
    Location
    North Texas
    Posts
    53,559

    Default

    Quote Originally Posted by J-Mo View Post
    In the peer-review process, the reviewers (experts in the field) check whether the topic of the article is appropriate for the journal, whether the methods indeed address the topic, whether the results justify the conclusions, and more. What do you mean reviewers looking at the data isn’t the norm? The figures and tables in a publication are the data. Reviewers absolutely look at figures/tables because they are in the field and the data is useful to them. The peer-review process isn’t perfect—some reviewers have biases, some cases of over interpretation are missed, etc. However, it’s kind of like democracy in that it’s the best system we have.

    The scientific community accomplishes major things that benefit the general public: research on cholesterol regulation led to the development of statins, which extend the lives of people with a predisposition to elevated cholesterol by years; research on the cardiac electrical system led to the development of pacemakers that extend the lives of people with a predisposition to arrhythmias by years; research on HIV replication led to the development of drugs that give HIV patients a life expectancy that is ~ equivalent to everyone else...

    None of these accomplishments came from a single paper or even a single lab. The point outsiders are missing is that the scientific literature is a conversation with others in the field. You don’t publish your paper because it is THE CURE for whatever disease you’re studying. You publish your paper because its data and implications are useful to other scientists who are working on the same problem. You address possible implications or applications of your research in the ‘discussion’ to give others ideas about future directions not for ‘propaganda’. It’s a large community with similar goals working on problems that are far too complex and difficult for a single paper to solve.
    This is a wonderful, lovely, and worshipful homage to the miracles wrought by both Modern Science and the Peer-Review Process. As I mentioned earlier, you may believe that which makes you happiest, even though your child-like faith in the process ignores lots of important problems.


    I have huge admiration for Rip and was crushed to seem him propagate an anti-science message.
    Here is the quote from the evil arch-nemesis of science Vox Day I used to introduce the piece:

    All of the arguments about the presumed reliability of science are ridiculous and easily shown to be false. Science is no more “self-correcting” than accounting. Peer review is more commonly known as “proofreading” by the rest of the publishing industry and is not even theoretically a means of ensuring accuracy or correctness. And scientists are observably less trustworthy than nearly anyone except lawyers, politicians, and used car salesmen; at least prostitutes are honest about their pursuit of “grants” and “funding.”

    These days, the scientific process is mainly honored in the breach by professional, credentialed scientists. And we have a word for testable, reliable science. That word is “engineering.”
    What is it about this entirely accurate summary of the situation within the vast majority of the academic/governmental science establishment that leads you to be "crushed" by my use of this as analogy to what we're doing in contrast to the ExFizz people?

  10. #10
    Join Date
    Aug 2013
    Location
    Lakeland, FL
    Posts
    3,112

    Default

    starting strength coach development program
    What part of guilt by association is scientific?

    I read some other stuff on the blog and found it all rather non controversial as someone who thinks that falsifying things is helpful...and that is what most of the posts and that post in particular did: falsified the idea that peer reviewed = credible.

    But even if he is a nutbag, so what? That quote is either appropriate and accurate or it isn't.

Page 1 of 4 123 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •