I get a Google alert whenever there’s a news story or research article published about “sciatica” or “sitting disabilities.” So I followed the media firestorm-in-a-snow-globe surrounding Dr. Chris Bailey’s article in the New England Journal of Medicine and the accompanying editorial with interest and a healthy sense of skepticism.
But before I could get around to putting my thoughts into order, Gary Schwitzer of HealthNewsReview.org wrote an editorial that put things better than I could have. I was heartened to see that skepticism and second-opinion seeking were skills that are still practiced in some corners of the internet.
I spent six years in various academic publishing roles. And I wasn’t in the industry long before I became extremely skeptical of any health article that contains the phrase “According to a study published in…[insert journal name}” in the first two paragraphs.
For two and a half years, I worked as a Publishing Assistant at a prestigious academic press. Sure, I was as low as you could get on the totem pole. But even I realized that the people in the press office were English majors whose scientific experience added up to a few dozen last-minute papers in college. Even those with some scientific training didn’t have the background necessary to understand every discipline in our publishing portfolio.
What they did have were reporters’ email addresses. They knew what sort of articles made news, and prioritized them accordingly.
The problem, of course, was that the press people didn’t actually understand what the articles were saying, because the articles were cloaked in scientific jargon and had a lot of tables. So the press people read the abstract and perhaps an email from the Editor-in-Chief and wrote a press release with a catchy headline.
That press release would then go to reporters. Most of the reporters know more about reporting than they do about science. They are also busy people who work on a deadline. So many of them are happy to take the slick press release and add a snarky lede that would catch their readers’ attention. As far as I could tell, most of them never read the actual article they were reporting about, and you can guess how many actually run their own statistical analysis or grill the authors with insightful questions.
The reporters are former English majors too, and wouldn’t understand a scientific discovery if it brought them chocolates and a puppy. (I say this as a former English major who has tried to parse out her share of academic papers.)
So what you have here is a system in which people with only a limited understanding of the research are alerting other people with a limited understanding of the research about the newsworthiness of a particular article. And then the alertees assume that, since the press release came from the publisher, it must be accurate.* It is certainly more readable than the article itself.
*To be fair, reputable publishers usually run press releases by the author or a subject matter expert from the editorial staff. So press releases (from reputable publishers) are unlikely to include major factual errors. But they are also not likely to include much debate about the limitations of the study or its scientific merit.
So we have a system that ends up turning the highly cautious, highly qualified conclusions of most research papers into confident declarations of fact.
Photo by Adonyi Gábor form PxHere
And that’s assuming everyone involved is putting in a good-faith effort, which is not always the case. There are plenty of journals out there with impressive-sounding names and no peer-review process to speak of. The perils of this system, combined with too-trusting reporters, were chronicled brilliantly by a science journalist who took part in a sting operation.
I have said many, many times, I am not a scientist. But these days, if I read a health article that might have significance to me, I do my best to find and read the actual paper the article is based on rather than trusting the journalist’s take. Rarely do I understand everything. But at least I come away understanding the limits of my own understanding.
